[{
      "title": "TeleDexter: Towards Human-level Dexterous Teleoperation",
      "url": "/posts/2026/07/teledexter-paper-notes/",
      "date": "2026-07-21",
      "tags": ["Dexterous Manipulation","Teleoperation","Reinforcement Learning","Sim-to-Real","Imitation Learning"],
      "excerpt": "",
      "text": "TeleDexter adds a learned low-level “cerebellum” between an operator’s intent and a dexterous robot hand. The operator and a tracked object provide synchronized fingertip-position and object-pose targets. A reinforcement-learning controller then chooses robot joint commands and contact transitions that realize those targets. This changes dexterous teleoperation from direct pose copying into goal-conditioned, contact-aware execution.  The mechanism is also the right boundary for reading the paper’s “human-level” claim. TeleDexter demonstrates in-hand reorientation, finger gaiting, and multi-stage tool use that conventional retargeting baselines rarely complete. It remains an object-specific controller trained from motion-capture data, requires real-time object tracking, observes no tactile signal, and is evaluated on seven fixed tasks with 15 trials each. The result is a strong contact-execution layer and a valuable demonstration generator; general human-level manipulation remains a much larger goal.  Paper Info  “Towards Human-level Dexterous Teleoperation” is by Puhao Li, Zeyuan Chen, Yingying Wu, Pengkun Wei, Yuyang Li, Tianyu Wang, Jiaxiao Shi, Mingrui Yu, Baoxiong Jia, Song-Chun Zhu, Tengyu Liu, and Siyuan Huang. The authors are affiliated with Tsinghua University, the State Key Laboratory of General Artificial Intelligence at BIGAI, and Peking University. These notes refer to arXiv:2607.11481v1, submitted July 13, 2026.      Paper    PDF    Project page and videos    The Missing Layer in Dexterous Teleoperation  Kinematic teleoperation maps tracked human joints or geometric hand features into robot joint targets. It is intuitive and fast, but the mapping carries no model of contact force, friction, object inertia, or actuator response. This becomes fragile as soon as a finger must release, move around the object, and re-establish contact while other fingers keep the object stable.  TeleDexter inserts a learned dynamics prior into this loop. All quantities are expressed in the robot wrist frame. The arm independently follows the operator’s wrist through inverse kinematics, while the hand policy receives a co-tracking goal  [g_t=\\left(\\hat{p}^{\\mathrm{tip}}_t,\\hat{T}^{o}_t\\right),]  where (\\hat{p}^{\\mathrm{tip}}_t) contains target fingertip positions and (\\hat{T}^{o}_t=(\\hat{x}^{o}_t,\\hat{R}^{o}_t)\\in SE(3)) is the target object pose. The policy outputs joint-position commands  [a_t=\\pi_\\theta(o_t,g_t).]  The goal specifies what geometry should be reached. Simulation-trained control supplies how to move contacts and joints to reach it. Tracking the object target matters as much as tracking the fingertips: identical fingertip positions can correspond to a stable regrasp, a slipping object, or a failed contact transition.  At deployment, motion capture measures an operator-side hand-object interaction while the robot-side state provides the current hand and object configuration. The resulting feedback loop is therefore grounded in two errors simultaneously: fingertip target versus robot fingertip state, and desired object motion versus the robot object’s measured motion.  1. Geometry-Aware Reference Motions  The low-level controller is trained from unscripted human hand-object interaction. For each object, the authors record 150 trajectories of 20 seconds each, totaling about 50 minutes. The recordings include in-hand translation, rotation, free play, finger gaiting, and tool-use motions.  Direct human-to-robot retargeting provides a useful initialization but can place fingers inside the object or destroy intended contacts. TeleDexter uses a two-stage pipeline:      vector-based retargeting aligns human and robot hand geometry;    object-mesh-aware optimization refines the whole trajectory.    The second stage solves  [q^*{1:T}=\\arg\\min{q_{1:T}}\\sum_{t=1}^{T}\\left(L^t_{\\mathrm{vec}}+\\lambda_{\\mathrm{surf}}L^t_{\\mathrm{surf}}+\\lambda_{\\mathrm{pen}}L^t_{\\mathrm{pen}}+\\lambda_{\\mathrm{col}}L^t_{\\mathrm{col}}\\right)+\\lambda_{\\mathrm{smooth}}L_{\\mathrm{smooth}}.]  The surface term attracts near-contact hand points to the object surface; the penetration term pushes hand points out of the object; the collision term prevents fingers from overlapping; and trajectory smoothness suppresses motion-capture jitter. This produces synchronized robot fingertip and object-pose targets with more plausible contact geometry. The later RL rollouts supply physical feasibility by resolving forces and friction that lie outside the geometric optimization.  2. Consecutive Subgoals Give RL Room to Discover Contact  Dense frame-by-frame imitation fixes a target at every instant. For dexterous contact, this can over-constrain exploration: the robot embodiment and dynamics may require a finger sequence that differs from the captured human timing.  TeleDexter samples consecutive co-tracking subgoals from each reference trajectory. For active subgoal (k), it measures per-finger position error, object-position error, and object-rotation error:  [e^f_{t,k}=\\lVert p^{\\mathrm{tip}}{t,f}-\\hat p^{\\mathrm{tip}}{k,f}\\rVert_2,\\qquade^{\\mathrm{pos}}_{t,k}=\\lVert x^o_t-\\hat x^o_k\\rVert_2,]  [e^{\\mathrm{rot}}_{t,k}=\\left\\lVert\\operatorname{Log}!\\left(\\hat R^{o\\top}_kR^o_t\\right)\\right\\rVert_2.]  A subgoal counts as reached after all errors stay below their thresholds for 5–15 consecutive frames. The final tolerances are 3 cm per fingertip, 1 cm object translation, and 10° object rotation. Once the target is reached, the system samples a later subgoal; 10% of transitions switch to a different reference trajectory at the same temporal index. The policy must therefore connect configurations that were never adjacent in a human demonstration.  The reward combines a sparse reach bonus, a small dense tracking term, and a time penalty:  [r_t=\\mathbf{1}{\\mathrm{reach}}(t)w{\\mathrm{step}}(t)r_{\\mathrm{score}}(t)+\\alpha_{\\mathrm{dense}}r_{\\mathrm{dense}}(t)-c_{\\mathrm{time}}.]  The sparse bonus is weighted by subgoal distance, so longer successful transitions earn more. The dense term helps early exploration before the first goal is reached. A curriculum gradually increases gravity, subgoal distance, object-tracking pressure, and action-mask duration. This formulation preserves the reference motion’s geometric intent while allowing RL to discover rolling, regrasping, and finger-gaiting strategies through contact simulation.  The ablation makes this point sharply. On held-out cuboid, hammer, and screwdriver motions, the dense-tracking policy reaches only about 2.6–2.7 sparse subgoals before stalling; TeleDexter reaches 32.6, 186.6, and 178.5, respectively.  3. The Learned Controller and Its Information Boundary  For SharpaWave, the observation has 142 dimensions; LeapHand uses 112. It contains joint positions and their sine/cosine encoding, the current object pose, gravity in the wrist frame, target and error vectors for fingertips and object pose, and the previous low-level command. An LSTM policy produces residual joint-position targets through a soft dead zone that lets the controller explicitly hold its previous command.  Two omissions are important: the controller receives neither contact force nor joint velocity. The authors exclude these signals because they are noisy or unavailable on the hardware. The policy must infer motion and contact consequences from kinematic state history, the previous command, gravity, and object-pose feedback. This makes hardware deployment simpler, while leaving ambiguous contact states unresolved. The appendix identifies tracking stalls caused by exactly this ambiguity: without touch, the controller cannot tell whether a finger is pressing the object or sliding past it.  Training uses SAPG in Isaac Gym with about 62,400 parallel environments on four RTX 5090 GPUs. Each object-specific controller consumes roughly (10^{10}) environment steps and about one day of training. The paper’s “single-stage” description means one RL stage and a shared reward recipe across contact modes. It still requires offline capture, geometry-aware retargeting, large-scale simulation, and a separately trained controller for each object.  4. Random Action Masking Bridges Asynchronous Hardware  Domain randomization covers hand and object dynamics, friction, object scale, external perturbations, observation noise, latency, and wrist initialization. TeleDexter adds random action masking as a stronger action-space regularizer.  With probability 0.15, the simulator selects three joint dimensions and freezes them at their previous commands for 1–10 control steps. Other joints receive the new action normally:  [\\tilde a_t[j]=\\begin{cases}\\tilde a_{t-1}[j], & j\\in\\mathcal{M}_t,a_t[j], & \\text{otherwise}.\\end{cases}]  This exposes the policy to stale and desynchronized actuation, approximating motor lag, backlash, compliance, and missed SDK commands. Removing action masking drops real-world success from 66.7% to 33.3% on HammerUse, from 73.3% to 0% on ScrewdriverUse, a..."
    },{
      "title": "Tactile Genesis: Exploring Tactile Sensors at Scale for Learning Dexterous Tasks",
      "url": "/posts/2026/07/tactile-genesis-paper-notes/",
      "date": "2026-07-16",
      "tags": ["Robot Learning","Tactile Sensing","Dexterous Manipulation","Simulation","Sensor Design"],
      "excerpt": "",
      "text": "Tactile Genesis builds a GPU-parallel tactile simulation platform and uses it to ask a practical hardware-design question: for dexterous manipulation, what tactile signal should a robot hand measure, where should it measure it, and how dense should the sensors be?  The short answer from the paper is unusually concrete. Whole-hand coverage matters more than installing a sophisticated sensor only at the fingertips. Around 200 taxels distributed across the hand are enough for the three studied tasks, and per-taxel force/torque is the strongest general-purpose representation. The policy architecture is mainly an experimental instrument for making this comparison; the central contribution is the simulator and the controlled sensor study.  Paper Info  The paper is “Tactile Genesis: Exploring Tactile Sensors at Scale for Learning Dexterous Tasks” by Trinity Chung, Kashu Yamazaki, Dhruv Patel, Alexis Duburcq, Yiling Qiao, Katerina Fragkiadaki, and Aran Nayebi. It is listed as a CoRL 2026 paper on the project page. The paper and a self-contained code snapshot are publicly available.  What Does the Paper Actually Do?  Existing tactile hardware bundles together several design choices: sensing principle, location, spatial resolution, physical response, and noise. Changing the sensor often changes the robot hand itself, which makes an apples-to-apples comparison almost impossible on real hardware.  Tactile Genesis turns these choices into independent simulation variables. It integrates multiple tactile abstractions into the Genesis physics engine behind a common interface, allows arbitrary placement and resolution on robot surfaces, and adds configurable imperfections such as latency, white noise, bias, random-walk drift, hysteresis, dead taxels, gain variation, and spatial crosstalk. This lets the authors hold the task, hand, policy, and training procedure fixed while changing only the tactile observation.  The work therefore has two layers:      A scalable tactile simulator that implements several sensor families on arbitrary robot geometry.    A controlled policy study that varies signal type, placement, resolution, and noise on three dexterous tasks.    The Simulated Tactile Modalities  The common interface covers seven broad sensing abstractions.      Binary contact: whether each taxel is touching an object.    Contact depth: penetration or distance-to-surface measurements from ray casting or a signed distance field.    Kinematic force/torque: a six-axis estimate at each taxel derived from contact depth, surface normal, and relative linear/angular velocity.    Elastomer displacement: simulated marker motion on a compliant tactile surface, similar to the intermediate signal used by GelSight-style sensors.    Geometry-aware proximity: a pre-contact response obtained by querying tracked object points near each taxel.    Temperature: a voxelized thermal field with diffusion, contact conduction, internal heat generation, radiation, convection, and sensor response lag.    Contact audio: synthesized impact, rolling, sliding, and actuation sounds conditioned on contact physics and material properties.    These models intentionally live at different abstraction levels. KinematicTaxel estimates useful local force without solving full deformable mechanics. ElastomerTaxel models spatially coupled marker displacement. TemperatureGrid solves a coarse heat-transfer field. Contact audio is a parametric signal generator layered over rigid-body contacts. The platform aims to expose policy-relevant sensor outputs at training scale, with parameters that can be calibrated toward real devices.  The implementation reaches more than 20,000 parallel environments and supports more than 1,000 taxels per hand on one GPU. The authors also benchmark regimes above 10,000 taxels. Depending on the sensor and baseline, they report approximately 3x to 20x higher throughput than previous tactile simulators, including TacSL, Tacmap, and HydroShear comparisons.  How the Policy Comparison Works  For each task-hand pair, the authors first train a privileged teacher with PPO. The teacher sees full object state, privileged contact information, proprioception, and the task goal. Random Network Distillation is added to encourage exploration.  They then train a tactile student through DAgger-style behavioral cloning. At deployment, the student receives joint positions, joint velocities, the previous action, one tactile representation, and the goal for reorientation tasks. Small auxiliary heads reconstruct privileged object quantities from the student’s hidden state during training:  [\\mathcal{L}{\\text{student}}=\\mathcal{L}{\\text{BC}}+\\mathcal{L}_{\\text{aux}}.]  The auxiliary decoders are discarded after training. Their role is to encourage the tactile encoder to preserve task-relevant object information. Keeping the student architecture and teacher behavior fixed makes performance differences easier to attribute to tactile sensing.  The comparison includes eight downstream tactile observations: per-taxel binary contact, link-aggregated contact, contact depth, link-aggregated force, per-taxel force, per-taxel force/torque, elastomer displacement, and proximity. A proprioception-only student provides the baseline.  Three Dexterous Tasks  The experiments use three XHand1 tasks with different contact regimes.      In-palm rotate: the thumb must locate and capture an object moving on the palm. Pre-contact location information can help.    In-hand repose: the hand continuously manipulates an object toward a target orientation. Slip and grip force are central.    Screwdriver: the fingers perform fast gaiting to keep a screwdriver spinning. Contacts are brief and rapidly changing.    Across these tasks, the study sweeps fingertip, finger, and whole-hand placement; low, medium, and high spatial resolution; clean and noisy sensing; and the tactile representations above.  Main Findings  1. Proprioception alone is insufficient  Every tactile student beats the proprioception-only baseline on all three tasks, including students that receive only binary contact. Joint state and the previous action cannot reliably reveal object slip, decoupled object motion, or hidden contact events.  2. Coverage matters more than sensor sophistication  On in-palm rotation, sensing only at the fingertips leaves a large gap to the privileged teacher. Adding the palm and proximal/middle finger surfaces closes most of that gap, even when the added taxels carry a relatively simple signal. The marginal value of covering a new hand region exceeds the value of upgrading an already instrumented fingertip.  3. Roughly 200 well-placed taxels are enough here  The medium-resolution XHand1 configuration contains 199 taxels across the whole hand. It performs close to the higher-resolution 667-taxel configuration across the tested tasks. The results suggest that coarse spatial contact distribution carries most of the useful information in this setting.  This is a task-scoped result. Fine texture recognition, tiny object geometry, or very local slip estimation could still benefit from high-resolution sensors; those capabilities are outside this experiment.  4. Per-taxel force/torque is the best default  The strongest signal depends on the task. Force/torque is best for in-hand reposing, where incipient slip and grip strength matter. Proximity is strongest for in-palm rotation because it detects an approaching object and lets the thumb pre-shape before contact. On screwdriver spinning, all tactile types behave similarly and remain below the teacher, suggesting that temporal integration or vision may be the missing ingredient.  Aggregated across tasks, per-taxel force/torque matches or exceeds the alternatives most consistently. The paper recommends it as the default tactile abstraction when the hardware supports it.  5. Elastomer marker motion is not automatically the best control signal  Elastomer displacement performs below force/torque on the two sustained-contact tasks. Marker motion at one location depends on deformation at neighboring locations, so the signal is spatially coupled. That coupling is useful for reconstructing local shape; it makes the local force vector less direct for the policy. This result concerns the chosen representation and control tasks, and does not imply that vision-based tactile sensors are generally weak.  Temperature and Audio Experiments  The temperature module is more than a rendering effect. The simulator models diffusion inside a voxel grid, heat exchange through contact, radiation, convection, and a first-order sensor lag. In a separate task, a hand must find one hot ball among eight geometrically identical balls using only proprioception and finger temperature readings.  The policy succeeds only under highly conductive, sufficiently strong thermal configurations. Parameters matching the sensitivity of current robotic temperature hardware fail. The experiment is valuable as a sensor-spec..."
    },{
      "title": "RynnWorld-Teleop: An Action-Conditioned World Model for Digital Teleoperation",
      "url": "/posts/2026/07/rynnworld-teleop-paper-notes/",
      "date": "2026-07-16",
      "tags": ["Digital Teleoperation","World Models","Video Generation","Dexterous Manipulation","Robot Learning"],
      "excerpt": "",
      "text": "RynnWorld-Teleop replaces the physical robot in a teleoperation session with an action-conditioned video world model. An operator’s hand-pose stream simultaneously drives a robot-view video generator and a human-to-robot retargeting pipeline, producing synchronized RGB observations and 54-dimensional robot actions for imitation learning.  My main takeaway concerns the operator’s “feel.” The system provides a responsive visual control loop: a distilled causal generator produces robot-centric video at 40 FPS with about 25 ms model latency. It does not provide haptic or force feedback. The Manus gloves capture pose; they do not return contact force, weight, friction, or slip to the hands. RynnWorld-Teleop is therefore best understood as a scalable data engine with visual telepresence, not a bilateral force-reflecting teleoperation system.  Paper Info  “RynnWorld-Teleop: An Action-Conditioned World Model for Digital Teleoperation” is by Haoyu Zhao, Xingyue Zhao, Hangyu Li, Biao Gong, Kehan Li, Siteng Huang, Xin Li, Deli Zhao, and Zhongyu Li, from DAMO Academy, Alibaba Group, the Hong Kong Embodied AI Lab, CUHK, Hupan Lab, and Ant Group. The notes below refer to arXiv:2607.06558v2, dated July 12, 2026.      Paper    PDF    Project page    Official code    Models on Hugging Face    Digital Teleoperation as Data Generation  Physical teleoperation couples every demonstration to robot availability, a prepared workspace, real objects, and manual resets. RynnWorld-Teleop moves the interaction into a learned visual world. Given a reference image (I_{\\mathrm{ref}}) and a hand-gesture sequence  [P={p_1,\\ldots,p_T},]  the model generates a robot-centric egocentric video  [V={v_1,\\ldots,v_T}.]  The same gesture stream is retargeted into robot joint positions. The output trajectory is therefore  [\\tau={(v_t,a_t)}_{t=1}^{T},\\qquad a_t\\in\\mathbb{R}^{54},]  where each action concatenates two 7-DoF arms and two 20-DoF dexterous hands. The generated observation and robot action share the same human control signal, which gives the system its action-grounded property.  The paper defines three requirements for useful digital teleoperation:      robot-centric: the generated view depicts the target robot embodiment;    action-grounded: each frame remains tied to recoverable joint-level actions;    real-time: the operator stays in the visual loop and can compose longer skills.    What the Operator Actually Feels  The operator wears five HTC Vive trackers—chest, wrists, and upper arms—and Manus data gloves. In the physical data-collection setup, the trackers provide wrist-to-chest transforms at 100–120 Hz. The gloves are converted into a 21-point MediaPipe hand skeleton and then retargeted to each 20-DoF WUJI hand, with an exponential moving average for smoothing.  These devices are motion sensors. The paper describes no actuators in the gloves, fingertip force display, vibrotactile feedback, exoskeleton resistance, or other haptic channel. The feedback available during digital teleoperation is the generated first-person robot video.                    Feedback or control channel        Present in the paper?        Role                            Hand and arm pose tracking        Yes        Captures operator intent                    Real-time robot-view video        Yes        Visual closed-loop feedback                    Motion smoothing and IK constraints        Yes        Stabilizes retargeted action labels                    Contact-force reflection        No        No physical reaction force reaches the operator                    Weight or inertia sensation        No        Inferred only from generated visual motion                    Friction and slip sensation        No        No tactile slip feedback                    Operator user study        No        No measured comfort, presence, or subjective hand-feel score              “Fluid and responsive” in the paper is supported by throughput and latency measurements. It should not be interpreted as measured haptic realism.  1. Depth-Aware Skeletons Preserve Fine Motion  A standard 2D skeleton loses the hand’s distance from the camera, which is crucial when reaching behind, around, or into contact with an object. RynnWorld-Teleop renders a 21-joint skeletal video whose color and joint/bone diameter vary with camera-space depth. The result remains a 2D image-like signal while carrying explicit 3D cues.  A pretrained VAE maps this skeleton video to a control latent  [c\\in\\mathbb{R}^{C\\times T\\times H\\times W},]  spatially and temporally aligned with the target video latent. This representation gives the video model fine-grained articulation and depth information without requiring full meshes or robot simulation assets.  2. Pose-Conditioned Video DiT  The world model builds on Wan2.2-TI2V-5B. It introduces a control patch-embedding branch alongside the original video branch. Because skeleton and video latents have different statistics, the control latent is normalized into the video latent distribution before fusion:  [\\tilde c=\\frac{c-\\mu_c}{\\sigma_c}\\sigma_z+\\mu_z,]  [x=\\operatorname{PatchEmbed}_z(z_t)+\\alpha\\operatorname{PatchEmbed}_c(\\tilde c).]  The control branch is zero-initialized and (\\alpha) starts small, allowing pose control to enter gradually while preserving the pretrained video prior. Direct concatenation disrupts that distribution: the reported FVD rises from 585 to 1191 in the LoRA ablation.  This conditioning mechanism is the link between an operator gesture and the visual consequence. It improves controllability, although it cannot guarantee that every generated contact obeys true rigid-body dynamics.  3. Human Priors First, Robot Embodiment Second  Training proceeds across two domains. Stage 1 uses large-scale egocentric human video to learn hand-object interaction priors:      VITRA: 30.7 million frames;    EgoDex: 74.0 million frames.    Stage 2 fine-tunes on 1,800 paired human-robot teleoperation episodes, totaling 0.43 million robot frames. The four tasks are dual picking, block pushing, bimanual lifting, and lid placement. Human gestures are paired with their robot executions through IK, teaching the generator how a human intention should appear through a specific robot embodiment.  The human-video stage supplies object permanence, contact patterns, and visual “common sense.” Removing it causes a severe collapse: FVD increases to 2598, with ghosting, blurred structures, and disappearing end effectors or objects. These learned priors make contact look plausible to the operator and make generated trajectories useful for policy training. They remain learned statistical dynamics, without explicit online force or contact solving.  4. Streaming Distillation Creates the Visual Loop  A bidirectional diffusion teacher is too slow for interactive control. The authors convert it into a causal student in two steps.  First, causal flow-matching warm-up teaches frame-causal generation. At time (t), attention can use only frames (1,\\ldots,t). A fixed-size KV cache supports streaming, while the reference-image embedding persists as a sink token to preserve the original scene.  Second, Distribution Matching Distillation (DMD) transfers the teacher’s output distribution into a four-step generator. Training backpropagates through the persistent KV cache across successive chunks, reducing boundary artifacts and improving long-rollout consistency.  On one H100 at (480\\times832), the causal model reports:      40.0 FPS;    approximately 25 ms per generated frame;    about 5% of latency in skeletal encoding;    about 72% in four-step causal DiT denoising;    about 23% in VAE decoding.    This rate matches or exceeds a typical 30 Hz robot camera stream and is the main basis for the paper’s claim of a responsive operator experience. The paper reports model inference latency, not complete motion-to-photon latency including trackers, communication, display, and buffering.  There is also a quality-speed trade-off. On EgoDex-Test, the non-causal SFT model obtains FVD 550 at 2.8 FPS; the causal model reaches 40 FPS with FVD 1226. Real-time interaction comes with reduced visual fidelity.  5. Retargeting Produces Smooth Robot Actions  The operator’s wrist pose is expressed relative to the chest, scaled into the robot workspace, and transformed into the robot base frame:  [T_{\\mathrm{target}}=T_{\\mathrm{base}}\\cdot\\operatorname{Scale}(T_{\\mathrm{chest}}^{-1}T_{\\mathrm{wrist}})\\cdotT_{\\mathrm{ee}}.]  An iterative damped least-squares IK solver maps this pose to robot joints. Adaptive damping handles near-singular configurations, a null-space shoulder prior favors natural arm postures, and hard clipping enforces joint limits. During physical demonstration collection, Ruckig further limits velocity, acceleration, and jerk before sending arm commands at 200 Hz. The robot’s low-level interface runs at 500 Hz; policy commands are reported at 50 Hz with 18–30 ms delay.  These mechanisms improve control smoothnes..."
    },{
      "title": "OmniContact: Chaining Meta-Skills via Contact Flow for Generalizable Humanoid Loco-Manipulation",
      "url": "/posts/2026/07/omnicontact-paper-notes/",
      "date": "2026-07-16",
      "tags": ["Humanoid Robots","Loco-Manipulation","Reinforcement Learning","Motion Control","Skill Chaining"],
      "excerpt": "",
      "text": "OmniContact proposes a compact interface for long-horizon humanoid loco-manipulation. Instead of asking a planner to generate dense whole-body motion, it represents a future skill using sparse body targets and binary contact schedules. A learned controller executes this contact flow, while a rule-based planner regenerates it online as the object state changes.  My read: the central contribution is the boundary between planning and control. CF-Gen converts object-level goals into contact-aware references through hand-designed phase templates, geometry, IK, and interpolation. CF-Track is a unified reinforcement-learning policy that turns those references into low-level humanoid actions. Contact flow is the shared language that makes carrying, pushing, kicking, recovery, and skill chaining fit into one system.  Paper Info  The paper is “OmniContact: Chaining Meta-Skills via Contact Flow for Generalizable Humanoid Loco-Manipulation” by Runyi Yu, Xiaoyi Lin, Ji Ma, Yinhuai Wang, Koukou Luo, Jiahao Ji, Huayi Wang, Wenjia Wang, Runhan Zhang, Ping Tan, Ting Wu, Ruoli Dai, Qifeng Chen, and Lei Han. The paper, project page, official MuJoCo execution repository, and OmniContact dataset are public.  The Problem: What Should a High-Level Planner Command?  Long-horizon loco-manipulation has two coupled problems. The robot needs a robust controller for individual interactions, and it needs a mechanism for composing those interactions while reacting to failed contacts or displaced objects.  Dense human-object trajectories precisely specify behavior, but they are expensive to synthesize and brittle under online changes. A symbolic command such as carry(box, target) is easy to plan, yet gives the low-level controller little information about where and when contact should happen. Latent skill embeddings are compact, although their semantics are difficult for an external planner to inspect and modify.  OmniContact chooses an intermediate representation: preserve the small set of future body targets required to express motion intent, and explicitly attach the contact timing that distinguishes manipulation from ordinary locomotion.  Contact Flow  At time (t), contact flow is defined as  [F_t={(b_{t+k},c_{t+k})}_{k\\in\\mathcal{T}},]  with non-uniform future offsets  [\\mathcal{T}={0,1,2,3,4,8,12,16,24,32,50}.]  Here (b_{t+k}) contains sparse future body targets, primarily torso, wrists, and ankles. The contact vector  [c_{t+k}\\in{0,1}^{4}]  specifies binary contact states for four robot end effectors. Non-uniform sampling allocates more targets to the immediate future and fewer to longer-term intent.  This representation carries two kinds of information:      Where the body should move: sparse future torso and end-effector targets.    When interaction should occur: explicit contact/no-contact state for each end effector.    Contact flow does not contain contact force, pressure, or a dense physics trajectory. It describes intended contact mode and sparse kinematic goals. The low-level policy retains freedom to choose the remaining whole-body motion needed for balance and compliance.  The ablation strongly supports this design. Tracking torso and end effectors without contact reaches only 11.5% success on the CF-Gen Carry Box evaluation. Adding binary contact raises success to 98.7%. Adding dense object, joint-DoF, or full-body targets then hurts generalization because the controller becomes overconstrained. The successful interface is the compact set torso + end effectors + contact.  CF-Track: The Learned Low-Level Executor  CF-Track is one unified policy for locomotion, carrying, pushing, sliding, relocating, kicking, and their transitions. Its input combines the target contact flow with a five-step observation history:  [x_t=[F_t,H_t],\\qquadH_t=[o_t,\\ldots,o_{t-4}].]  Each observation includes robot proprioception, base orientation, end-effector positions, the previous action, and the object’s relative 6D pose and bounding box. The policy outputs low-level motor actions at 50 Hz. The actor and critic use Transformer networks.  CF-Track is trained with reinforcement learning using three reward families:  [r_t=lambda_{\\text{track}}r_t^{\\text{track}}+\\lambda_{\\text{amp}}r_t^{\\text{amp}}+\\lambda_{\\text{reg}}r_t^{\\text{reg}}.]  The tracking term follows sparse body, object, and contact-flow targets. The AMP adversarial motion prior encourages human-like whole-body motion. The regularization term penalizes abrupt action changes. The selected balance places most weight on tracking while retaining enough motion prior to smooth the artifacts of rule-generated references.  Pure tracking achieves relatively accurate motion but reduces disturbance stability. Giving the motion prior too much weight produces stable-looking behavior with almost no task success. This is a genuine control trade-off: the policy must obey contact intent strongly enough to finish the task while preserving enough behavioral flexibility to absorb imperfect plans and perturbations.  CF-Gen: Rule-Based Contact-Flow Synthesis  CF-Gen is described as a lightweight, heuristic planner. It is not a learned trajectory generator. Each meta-skill has a hand-designed sequence of phases. Carrying, for example, is decomposed into approach, move to a hand-contact pose, lift, walk while maintaining contact, and release.  For each phase, CF-Gen performs four operations.  1. Select object-centric contact anchors  The planner uses object pose and dimensions to choose suitable contact faces and approach directions. It adapts wrist, torso, and ankle targets to object geometry, enabling the same carry or push template to handle different object sizes and initial poses.  2. Generate keyframes with selective IK  Locomotion phases mainly specify ankle goals and a nominal posture. Contact phases specify ankle and wrist targets, then solve constrained IK over pelvis height, pelvis pitch, and robot joints. The solver is limited to 20 iterations for real-time synthesis.  3. Interpolate a dense kinematic reference  Positions and joint values use linear interpolation, while pelvis orientation uses SLERP. This creates a continuous reference between phase keyframes.  4. Convert the reference into sparse contact flow  CF-Gen samples the interpolated trajectory at the non-uniform future offsets and transforms the targets into the current torso-yaw frame. CF-Track receives only the resulting sparse body targets and binary contact schedule.  This sequence is important: CF-Gen temporarily creates a dense reference internally, but the controller does not directly track the full joint trajectory. Contact flow discards unnecessary constraints before execution.  Closed-Loop Chaining and Recovery  CF-Gen maintains a finite stage state containing the active object, selected meta-skill, phase, and target. When a phase finishes, it generates the next contact-flow segment. Multi-stage tasks are built by switching objects and meta-skills at stage boundaries.  During execution, CF-Gen monitors the observed and predicted object state at 50 Hz:  [\\delta_t=d(x^{\\text{obj}}{t,\\text{obs}},x^{\\text{obj}}{t,\\text{pred}}).]  If (\\delta_t) crosses a threshold, the planner aborts the current reference and replans from the observed state. A dropped or displaced box can therefore trigger a new approach and contact sequence.  Online replanning raises Push Suitcase from 82.5% to 94.5%, Stack Boxes from 56.6% to 80.5%, and Push-Stack Boxes from 76.5% to 84.5%. The paper also notes an important nuance: most of this gain comes from refreshing future segments with updated object states. Full emergency recovery is triggered relatively rarely.  Under deliberately injected failures, recovery reaches 92.5% after dropping a carried box, 97.5% after offsetting its pose, and 89.5% after offsetting a pushed suitcase, with roughly 1.5-1.8 replans per trial.  OmniContact Dataset  The authors collect a synchronized MoCap human-object interaction dataset tailored to long-horizon loco-manipulation. It contains:      1,274 valid sequences and 22.29 hours of motion;    7.22 million object frames captured at 90 Hz;    synchronized BVH human motion and rigid-object 6D trajectories;    carry, push, relocate, slide, and kick primitives;    primitive-level contact modes and task-level language descriptions.    The average sequence lasts 62.98 seconds and moves the object 19.76 meters, emphasizing long-range transport. The dataset has fewer objects and sequences than OMOMO, but longer clips, higher temporal resolution, and explicit loco-manipulation structure. Scaling CF-Track training data from 2.2 to 22.3 hours improves both success and object accuracy.  The dataset does not provide measured contact forces. Contact supervision is constructed from synchronized body/object trajectories, binary contact modes, and dynamic/static phase annotations.  Main Results  In the randomized simulation benchmark, OmniContact reaches:      98.7% on Carry Box;    82.5% on Push Suitcase;..."
    },{
      "title": "FIGAROH: a Python toolbox for dynamic identification and geometric calibration of robots and humans",
      "url": "/posts/2026/07/figaroh-paper-and-code-notes/",
      "date": "2026-07-16",
      "tags": ["Robot Calibration","System Identification","Robot Dynamics","Optimal Experiment Design","Open Source"],
      "excerpt": "",
      "text": "FIGAROH is a Python toolbox for two closely related model-correction problems: geometric calibration, which estimates errors in robot kinematics, and dynamic identification, which estimates inertial, friction, actuator, and sensor parameters. Its most useful systems idea is the complete experimental loop: construct the identification model from URDF, determine which parameter combinations are observable, design informative postures or motions, estimate parameters, validate the result, and update the robot model.  This note reads the original paper together with the current figaroh-plus codebase. The distinction matters. The paper describes the unified scientific workflow demonstrated on robots and a human subject. The current v0.4.5 repository has grown into a more modular library with robust calibration, additional solvers, physical-consistency projection, URDF export, multi-backend interfaces, and reporting/verification tools.  Paper Info  The paper is “FIGAROH: a Python toolbox for dynamic identification and geometric calibration of robots and humans” by Dinh Vinh Thanh Nguyen, Vincent Bonnet, Maxime Sabbah, Maxime Gautier, Pierre Fernbach, and Florent Lamiraux. The provided PDF is the HAL version of the paper published at the 2023 IEEE-RAS International Conference on Humanoid Robots, DOI 10.1109/Humanoids57100.2023.10375232. Despite the local filename FIGAROH_IROS_2022-5.pdf, it is a Humanoids 2023 paper.  The current GitHub repository is a maintained fork of the original LAAS GitLab project. It publishes the figaroh Python package, while runnable robot examples live in a separate figaroh-examples repository.  Two Problems in One Toolbox  Geometric and dynamic identification use similar mathematics and experimental logic, although they correct different parts of the model.  Geometric calibration  A URDF gives each joint a nominal transform. Real assembly introduces small translation and rotation errors. FIGAROH parameterizes the local variation of joint (j) as  [\\Phi_{g,j}=[\\delta p_x,\\delta p_y,\\delta p_z,\\delta\\phi_x,\\delta\\phi_y,\\delta\\phi_z]^T.]  The forward-kinematics residual is locally linearized:  [\\Delta P=R_g(q,p)\\Phi_g,]  where (R_g) is a geometric regressor or kinematic Jacobian connecting parameter errors to measured pose errors. Measurements can be full poses, positions, orientations, camera observations, motion-capture markers, or geometric constraints such as contact with a plane.  The six-parameter representation does not mean that all six parameters of every joint can be independently recovered. The paper notes that a revolute joint has at most four independently identifiable geometric parameters and a prismatic joint at most two; collinear axes introduce additional dependencies. FIGAROH uses rank analysis to keep the observable parameter combinations.  Dynamic identification  Rigid-body inverse dynamics is linear in inertial and several extended mechanical parameters:  [D=R_d(q,\\dot q,\\ddot q)\\Phi_d.]  Here (D) is a stack of measured torques, motor currents, or external wrenches; (R_d) is the dynamic regressor; and (\\Phi_d) can include:      ten inertial parameters per body: mass, first moment, and six inertia-tensor terms;    joint viscous and Coulomb friction;    torque-sensor offsets;    actuator rotor inertia and drive-chain friction;    current-to-torque or sensor gains.    The identifiable set depends on available measurements. Joint torques can identify inertial and joint parameters. Combining joint torque with motor current exposes drive-chain terms. External wrench measurements identify a different subset. FIGAROH builds the regressor according to this measurement contract.  Rank Deficiency and Base Parameters  Robot regressors are generally rank deficient. Some physical parameters always appear together in the equations, so data cannot separate them regardless of sample count. FIGAROH numerically applies QR decomposition with column pivoting to rewrite the original regressor in terms of full-rank base parameters:  [Y=R\\Phi=R_b\\Phi_b,\\qquad\\operatorname{rank}(R_b)=\\dim(\\Phi_b).]  This step is central. A least-squares solver can produce numbers for an underdetermined parameterization, but those numbers are not uniquely supported by the measurements. Base-parameter reduction makes the estimation problem honest: the solver estimates only independent combinations that the experiment can observe.  The paper emphasizes the interpretability of numerical QR. It can expose which original columns combine into each base parameter, and it can produce different static and dynamic base sets. For example, static data removes velocity and acceleration terms, leaving mass and center-of-mass combinations.  Parameter Estimation  Once the base regressor is full rank, the simplest estimate is ordinary least squares:  [\\hat\\Phi_b=(R_b^TR_b)^{-1}R_b^TY.]  Weighted least squares compensates for measurement channels with different noise levels or units. The paper also covers total least squares for experiments with loaded and unloaded configurations, iterative least squares and Levenberg-Marquardt for nonlinear geometric calibration, and constrained optimization for physically plausible inertial parameters.  The current code retains the regressor/QR workflow and broadens solver support. BaseIdentification.solve() runs the standard identification path, while solve_with_custom_solver() can invoke least squares, QR/SVD-style methods, Ridge, Lasso, Elastic Net, robust regression, and constrained variants through figaroh.tools.LinearSolver.  Physical consistency deserves separate treatment. An unconstrained fit can yield negative mass or an impossible inertia tensor while still minimizing torque error. The paper uses quadratic constrained programming to recover individually meaningful parameters. The current code additionally provides optional pseudo-inertia SDP/LMI projection and full-parameter reconstruction from base parameters, with CAD-derived mass, center-of-mass, and symmetry constraints. These paths require an SDP solver and are disabled by default.  Optimal Experiment Design  FIGAROH treats data collection as part of identification. Random postures may leave the regressor poorly conditioned; random motions can waste robot time and become unsafe for humanoids. The toolbox therefore designs Optimal Exciting Postures (OEP) for calibration and static identification, and Optimal Exciting Motions (OEM) for dynamic identification.  The paper uses singular-value criteria derived from the regressor, including determinant-like information volume, condition number, and minimum singular value. Informally, a useful experiment makes the regressor columns distinguishable:  [\\text{good excitation}\\Longrightarrow\\sigma_{\\min}(R)\\text{ increases and }\\kappa(R)=\\frac{\\sigma_{\\max}(R)}{\\sigma_{\\min}(R)}\\text{ decreases}.]  Selecting calibration postures  Candidate postures receive information matrices (\\Sigma_i) and continuous weights (\\omega_i). FIGAROH solves  [\\max_{\\omega}\\Psi!\\left(\\sum_i\\omega_i\\Sigma_i\\right),\\qquad\\sum_i\\omega_i=1,]  then ranks candidates by weight. The excitation score eventually reaches a peak or plateau, which gives a data-driven stopping point for the number of postures. The paper illustrates a case where the criterion peaks near 40 postures.  Postures are optimized under joint limits, torque limits, workspace bounds, and self-collision constraints. The paper uses Pinocchio/hpp-fcl geometry and IPOPT-based nonlinear optimization.  Generating exciting motions  OEM generation connects consecutive OEP using cubic splines. The optimizer adjusts spline waypoints while respecting endpoint positions, zero endpoint velocity, joint/velocity/torque limits, workspace constraints, and collision avoidance. The objective improves the accumulated dynamic base regressor over the trajectory.  This design closes a loop that many system-identification scripts leave manual:  URDF + measurement setup        -> identifiable regressor        -> optimal postures/motions        -> robot data collection        -> parameter estimation        -> cross-validation and model update    What the Paper Demonstrates  The paper evaluates the workflow across human, humanoid, mobile-manipulator, and serial-arm cases.  For a 43-DoF human model, motion capture and force-plate data are used for geometry and dynamics. Compared with anthropometric-table parameters, identified parameters reduce average external-wrench RMSE from 19.47 N to 16.84 N for force and from 33.22 N·m to 16.77 N·m for moment.  For TALOS and TIAGo torso-arm geometric calibration, end-effector position RMSE drops from 14.1 mm to 0.3 mm on TALOS and from 16.9 mm to 1.5 mm on TIAGo. TALOS whole-body calibration with three-point planar contact and 31 automatically generated OEP reduces RMSE from 10.4 mm to 3.3 mm.  TIAGo dynamic identification includes drive-chain parameters and its differential wrist transmission; the identified model predicts measured motor curr..."
    },{
      "title": "Grasp Multiple Objects with One Hand",
      "url": "/posts/2026/07/multigrasp-paper-notes/",
      "date": "2026-07-15",
      "tags": ["Dexterous Manipulation","Multi-Object Grasping","Grasp Synthesis","Diffusion Models","Reinforcement Learning"],
      "excerpt": "",
      "text": "MultiGrasp enables a Shadow Hand to grasp and lift multiple independent objects concurrently. Its decisive choice is to make every stage object-aware: the pre-grasp objective asks for force closure on each object, the diffusion model keeps object identities separate, and the lifting reward is controlled by the lowest object. A grasp that secures only one of two targets cannot score well.  My read: the paper succeeds by dividing the problem at the boundary between static contact geometry and dynamic execution. An augmented differentiable force-closure optimizer and a diffusion model construct a multi-contact hand pose; motion planning and goal-conditioned reinforcement learning then reach that pose and preserve all contacts during lift. This split is especially important because moving one finger to help one object can destabilize the other.  Paper Info  “Grasp Multiple Objects with One Hand” is by Yuyang Li, Bo Liu, Yiran Geng, Puhao Li, Yaodong Yang, Yixin Zhu, Tengyu Liu, and Siyuan Huang. It appeared in IEEE Robotics and Automation Letters (RA-L), Vol. 9, No. 5, 2024.      Paper PDF    Project page    Paper on arXiv    Official code    Grasp’Em dataset branch    Why Two Objects Change the Problem  A single-object dexterous grasp can often envelop the target and close all fingers toward it. With two independent objects, improving contact on one target may release the other. The hand must use its full workspace—including finger lengths and side surfaces—and maintain a separate force-closure condition for every object.  MultiGrasp studies tabletop arrangements  [O={O_j}_{j=1}^{N_o},]  where each object is represented by a surface point cloud. The output is a sequence of hand actions that simultaneously grasps and lifts all targets. The objects must be close enough and small enough to fit in one hand; this is a geometric assumption of the task, not a capability learned around arbitrary placements.  The complete pipeline has two high-level stages:      propose and refine a pre-grasp pose that encloses all targets;    plan a collision-aware reach, then use a learned policy to lift every object.    1. Independent Force Closure for Every Object  The synthetic grasp generator extends Differentiable Force Closure (DFC). For hand configuration (H) and objects (O), it minimizes  [E(H,O)=\\sum_{j=1}^{N_o}\\min_{x_j\\subset S(H)} E_{\\mathrm{FC}}(x_j,O_j)+\\lambda_p E_p(H,O)+\\lambda_{sp}E_{sp}(H)+\\lambda_qE_q(H).]  The first term is the key multi-object design. For each object (O_j), the optimizer selects hand contacts (x_j) and independently evaluates force-closure error. Stability cannot come solely from treating the pair as one combined shape. The remaining terms penalize hand-object penetration, hand self-collision, and joint-limit violations. Gradient optimization with Metropolis-Adjusted Langevin Algorithm sampling explores multiple local solutions, followed by filtering.  This objective encourages different parts of the dexterous hand to support different objects. The allocation is produced by contact optimization instead of a fixed assignment such as “thumb and index finger for object A.”  2. A Diffusion Model That Preserves Object Identity  DFC synthesis is effective and slow—roughly 1000 seconds in the reported batched setting. MultiGrasp therefore learns a SceneDiffuser-style DDPM that maps the object point clouds to a pre-grasp pose:  [p(H^{(0)}\\mid O)=\\prod_{t=1}^{T}p(H^{(t-1)}\\mid H^{(t)},O).]  PointNet++ extracts features independently from each object. The denoising network uses the noisy hand representation as queries and object features as keys and values in cross-attention. A learned object embedding is attached to every point feature: points from the same object share an embedding, while different objects receive different embeddings. The network can therefore reason about two independent targets instead of seeing one merged point set.  The hand pose is represented by 31 Cartesian keypoints distributed over the palm and finger links. An optimization-based inverse-kinematics solver converts these points to joint angles. Cartesian keypoints expose local finger-link/object relations directly. In the ablation, this representation reaches 40.20% execution success, compared with 19.31% when the model directly generates joint angles.  The training set, Grasp’Em, contains about 90,000 synthetic pre-grasps:      16.4k single-object grasps;    73.7k dual-object grasps;    8 objects and 36 single/dual combinations from YCB and ContactDB;    randomized stable tabletop positions and orientations.    The objects are rescaled so several can fit in the hand. Training also normalizes the palm’s projected direction around the tabletop’s vertical axis. At inference, rotating the scene into this canonical orientation and rotating the predicted hand back allows the system to control palm direction and avoid particularly difficult approaches.  3. Contact Refinement Before Execution  Diffusion samples can contain shallow penetration or leave useful fingers floating near a surface. A coarse-to-fine refinement step minimizes penetration while attracting nearby potential hand contacts toward object surfaces. Its distance threshold decreases from 2 mm to 1 mm.  This small-looking stage has a large effect. Removing refinement increases maximum penetration from 1.67 mm to 7.68 mm and reduces success from 40.20% to 16.24%. The comparison shows that a visually plausible hand pose is insufficient; multi-object lifting needs contacts that survive dynamics.  4. Reach First, Then Learn to Lift  Execution is split into reaching and lifting. The reaching stage linearly interpolates from a flat hand to the pre-grasp, then optimizes the trajectory for smoothness and hand-object collision reduction. Small remaining collisions may displace an object, so the lifting policy is trained to recover from the resulting state.  The PPO lifting policy controls 18 actuated joint-angle targets and a 6D hand-base wrench. Its observations include proprioception, fingertip wrenches, current and goal hand states, object states, and point-cloud features. The policy is goal-conditioned on the pre-grasp configuration instead of discarding it after reaching.  The central reward is  [r=\\omega_{\\mathrm{lift}}r_{\\mathrm{lift}}+\\omega_{\\mathrm{succ}}\\mathbf{1}{\\mathrm{succ}}+\\omega_r r_r+\\omega_q r_q+\\omega{\\mathrm{obj}}r_{\\mathrm{obj}},\\qquadr_{\\mathrm{lift}}=\\min_j h_j.]  Here (h_j) is the height of object (j). Taking the minimum makes the lowest object the bottleneck: lifting one target high cannot compensate for leaving or dropping another. A success bonus is activated only when all objects exceed the training threshold. Additional terms maintain the goal hand orientation, joint configuration, and each object’s position relative to the hand.  This reward encodes simultaneous grasping in the dynamic stage. The ablation is direct: the complete training setting obtains 45.25%, while removing RL leaves only 1.37% success. Removing goal observations and rewards gives 16.79%.  5. Specialists, Vision Distillation, and Error Adaptation  Object shape, pair identity, and placement relative to the palm produce different lifting dynamics. The authors cluster training grasps by object combination and by the direction connecting object centers in the palm frame. A specialist policy is trained for each cluster. Their demonstrations are distilled with DAgger into a vision-based generalist that consumes a fused scene point cloud from three RGB-D cameras.  The final policy also needs to tolerate imperfect pre-grasps. MultiGrasp uses a three-phase curriculum:      train on high-quality synthetic pre-grasps;    mix synthetic samples with generated poses and states disturbed during reaching;    emphasize generated and displaced states.    Without adaptation to imperfect poses, success drops to 25.05%; removing the curriculum yields 24.88%. This training recipe closes the gap between a static target pose and the actual state encountered after physical reaching.  What the Experiments Establish  Success is defined as lifting all objects above 10 cm. Evaluation uses 512 unique poses with five trials per pose. For unseen object placements using synthetic pre-grasps, state-based specialists achieve 68.34%, while the distilled vision generalist achieves the paper’s headline 44.13%. With diffusion-generated pre-grasps on unseen placements, the corresponding results are 40.20% / 30.24%.  The number 44.13% therefore has a precise scope: it is the vision-based generalist’s simulated dual-object success with synthetic pre-grasps on unseen placements. It is not an unrestricted real-world success rate.  The pre-grasp ablations isolate the representation choices:                    Pre-grasp model        Q1 ↑        Penetration (mm) ↓        Success ↑                            Full model        0.29        1.67        40.20%                    Joint-angle representation        0.18        5.50        19.31%      ..."
    },{
      "title": "UniDexTok: A Unified Dexterous Hand Tokenizer from Real Data",
      "url": "/posts/2026/07/unidextok-paper-notes/",
      "date": "2026-07-14",
      "tags": ["Robot Learning","Dexterous Manipulation","Tokenization","Cross-Embodiment","Representation Learning"],
      "excerpt": "",
      "text": "UniDexTok studies a representation problem that sits before dexterous manipulation policy learning: how can data from different human and robot hands be put into one shared state space, then converted into discrete tokens that future VLA or world-action models can use?  My read: the paper’s core contribution is a retargeting-free state tokenizer. UniDexTok avoids converting every hand trajectory into a target robot hand through MANO-to-robot retargeting. It first standardizes human and robot hand states into a 22-DoF semantic interface called UDHM, then trains one conditional tokenizer with a shared encoder, shared codebook, and shared decoder across embodiments. The important shift is from “make every hand imitate one robot” to “make every hand speak the same joint-semantic language.”  Paper Info  The paper is “UniDexTok: A Unified Dexterous Hand Tokenizer from Real Data” by Dong Fang, Youjun Wu, Yuanxin Zhong, Rui Zhang, Yunlong Wang, Xiaosong Jia, and Yu-Gang Jiang from Fudan University, Rimbot, Hefei University of Technology, and Beijing University of Posts and Telecommunications. It is available as arXiv:2606.10683.  Why Dexterous Hand States Are Hard to Share  Parallel grippers are easy to represent: opening width, gripper pose, maybe a binary open-close state. Dexterous hands are messier. Different hands have different numbers of fingers, joint names, joint orders, coupled joints, control ranges, and units. Even datasets collected with the same family of hands can log joint states in different conventions, including radians, degrees, or discretized bins.  This makes cross-embodiment learning awkward. If every robot hand has a separate tokenizer, then each codebook learns a private latent space. The model may reconstruct each hand, but the discrete tokens do not have a stable meaning across hardware. If the pipeline relies on retargeting human motion into every robot hand, the representation inherits geometric mismatch from retargeting and loses the native state statistics of real robot data.  UniDexTok’s answer is to standardize the state space first, then tokenize.  UDHM: A 22-DoF Semantic Interface  The Unified Dexterous Hand Model, or UDHM, is the paper’s shared coordinate system. It assumes a human-hand-like kinematic structure and maps MANO-style human keypoints and robot hand joints into a 22-dimensional active joint interface.  For the index, middle, and ring fingers, UDHM uses four active coordinates per finger: MCP abduction/adduction, MCP flexion/extension, PIP flexion/extension, and DIP flexion/extension. The pinky adds one twist degree of freedom to cover larger lateral variation. The thumb uses five coordinates: CMC flexion, CMC spread, MCP flexion, MCP abduction, and IP flexion. Together these choices give 22 active coordinates.  The model fits a palm plane from the wrist and MCP keypoints, uses the palm normal and local lateral axes to define motion planes, and reconstructs each finger chain with analytic forward kinematics. For inverse mapping, it extracts palm offsets and bone lengths from the input frame, then solves a nonlinear least-squares problem so the reconstructed joints match the target keypoints.  The practical detail I like is the treatment of heterogeneous robot logs. Each available robot-hand DoF is inserted into the semantically corresponding UDHM coordinate, and missing coordinates are zero-padded. This “semantic insertion” is stronger than simply appending all available coordinates at the front of a vector, because the model can learn that a coordinate means “index MCP pitch” across embodiments without memorizing source-specific column positions.  All raw joint angles are converted to radians and normalized by a fixed scale of (\\pi). That fixed normalization keeps tokens from depending on train-test split statistics and keeps MPJAE interpretable after denormalization.  UniDexTok Architecture  Given a standardized hand state (x \\in \\mathbb{R}^{22}) and a hand-type or source label (h), the tokenizer learns  [\\hat{x} = \\pi D(Q(E(\\tilde{x}, h_{\\mathrm{embed}})), h_{\\mathrm{embed}}), \\quad \\tilde{x}=x/\\pi.]  The encoder projects the 22-dimensional normalized state into (N=8) latent tokens with width (C=512). Transformer blocks then model correlations among joint coordinates. Hand-type conditioning enters through adaptive layer normalization:  [\\mathrm{AdaLN}(z,h_{\\mathrm{embed}})=\\gamma(c_h)\\odot \\mathrm{LN}(z)+\\beta(c_h).]  This conditioning matters because the same semantic pose can require different numeric joint patterns on different hands. The shared encoder and decoder learn the cross-hand structure, while the conditioning keeps hardware-specific conventions available.  The decoder mirrors the encoder. It takes quantized tokens, adds positional embeddings and hand conditioning, and reconstructs the normalized 22-dimensional state. The reconstruction objective is based on normalized joint-angle reconstruction, and the full objective adds the VQ commitment loss:  [L = L_{\\mathrm{rec}} + L_{\\mathrm{vq}},]  with  [L_{\\mathrm{vq}}=\\beta\\lVert \\mathrm{sg}[q]-u\\rVert_2^2+\\lVert q-\\mathrm{sg}[u]\\rVert_2^2,]  where (\\beta=0.25) in the reported experiments.  Factorized Codebook  A simple 256-entry codebook would give only 256 choices per quantized vector. UniDexTok instead factorizes the vocabulary across channel groups. The encoder output is projected from 512 to 256 dimensions, divided into (K=8) groups, and each group uses a 32-entry sub-codebook. That means one token can express (32^8) code combinations while storing only (32 \\times 8 = 256) learned code vectors.  For each token position and group, the nearest normalized code vector is selected by cosine similarity:  [i_{n,k}=\\arg\\max_j\\left\\langle\\frac{u_{n,k}}{\\lVert u_{n,k}\\rVert_2},\\frac{e_{k,j}}{\\lVert e_{k,j}\\rVert_2}\\right\\rangle.]  This design is important for discrete representation quality. In the paper’s gesture benchmark, UniHM’s single-codebook quantized features drop from 96.15% to 84.62% linear probing accuracy, while UniDexTok keeps 100% accuracy after quantization. The interpretation is straightforward: a single codebook can collapse different gestures into the same discrete code; factorized VQ gives the token more compositional capacity.  Data and Evaluation  The training data combines two main sources. The first is coarse human hand-object interaction data from DexYCB, OakInk-v2, and EgoDex. UniDexTok does not retarget those motions to a robot hand. It reduces the MANO-style 45-DoF hand representation to the 22 active UDHM coordinates and trains on the standardized human hand as another embodiment.  The second source is real robot-hand data from LET, Dexora, and LinkerHand. These datasets differ in joint ordering, dimensionality, and units, so UniDexTok converts all values to radians, inserts available DoFs into the corresponding UDHM coordinates, and pads missing coordinates.  The paper evaluates reconstruction in joint-angle space and Cartesian space. MPJAE measures mean per-joint angle error in degrees. MPJPE measures mean per-joint position error in millimeters. FK error reports fingertip-position errors.  Main Results  Against UniHM on the standardized real datasets, UniDexTok reduces average MPJAE from 15.63 degrees to 0.16 degrees and average MPJPE from 18.51 mm to 0.18 mm. That is the paper’s headline result: reconstruction improves from centimeter-scale error to sub-millimeter error.  The improvement is consistent across LinkerHand L6, LinkerHand L10, LinkerHand L20, and Robotera XHand1. On the retargeted DexYCB evaluation, the gap is smaller because that protocol is closer to the retargeted data distribution used by UniHM, but UniDexTok still improves average MPJAE from 4.40 degrees to 2.83 degrees.  The zero-shot and few-shot results are also useful. On an unseen Inspire RH56E2 hand, zero-shot per-joint angle errors range from 4.14 to 7.85 degrees, and fingertip FK errors range from 7.73 to 16.05 mm. With only 4,528 frames, about 6.2% of the full Inspire dataset, fine-tuning for 2 epochs reduces joint errors to roughly 1.42 to 2.08 degrees and reduces fingertip errors by 58.5% to 78.8%.  This is the evidence for treating the tokenizer as a cross-embodiment representation, beyond a compression model for one hand. A new hand can enter the same token space and then adapt with a small amount of target data.  Ablations  The ablation on human-hand data is especially relevant for embodied datasets. With human-hand data included, the average MPJAE is 0.16 degrees and MPJPE is 0.18 mm. Without human-hand data, they rise to 0.37 degrees and 0.43 mm. The human hand is therefore useful as a training embodiment, not merely as a source for retargeting.  The semantic-insertion ablation supports the UDHM design. With semantic insertion, MPJAE, MPJPE, and FK error are 0.24 degrees, 0.25 mm, and 0.53 mm. Without semantic insertion, they degrade to 0.53 degrees, 0.57 mm, and 1.20 mm. The benefit comes from aligning coordinates b..."
    },{
      "title": "InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactions",
      "url": "/posts/2026/07/intermimic-paper-notes/",
      "date": "2026-07-14",
      "tags": ["Human-Object Interaction","Physics-Based Animation","Reinforcement Learning","Motion Imitation","Humanoid Control"],
      "excerpt": "",
      "text": "InterMimic learns one physics-based controller for diverse whole-body human-object interactions (HOIs) from imperfect motion-capture data. Its central recipe is “perfect first, then scale up”: small teacher policies first retarget and physically repair the captured interactions; a Transformer student then absorbs the teachers’ corrected references and actions, followed by reinforcement-learning fine-tuning.  My read: the most important idea is that the teachers are more than action-label generators. Their simulated rollouts become a cleaner training distribution for the student. This converts teacher-student learning into two coupled operations—reference distillation cleans what the student should track, while policy distillation teaches how to track it. PPO then resolves conflicts left by supervised aggregation.  Paper Info  The paper is “InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactions” by Sirui Xu, Hung Yu Ling, Yu-Xiong Wang, and Liang-Yan Gui, from the University of Illinois Urbana-Champaign and Electronic Arts. It was presented at CVPR 2025 as a Highlight paper.      Paper (arXiv:2502.20390)    Project page    Official code    The Problem: Large HOI Data Is Useful and Physically Messy  Physics-based motion imitation usually asks a simulated character to follow a reference trajectory. Human-object interaction raises the difficulty because errors couple across the body and the object. A floating hand contact can make the object fall; an inaccurate symmetric-object rotation can imply sliding; shape differences between captured subjects and the simulated body can destroy a valid interaction after retargeting. Many HOI datasets also omit detailed finger motion.  The target trajectory contains human and object states:  [q_t = {q_t^h,q_t^o},\\qquadq_t^h={\\theta_t^h,p_t^h},\\qquadq_t^o={\\theta_t^o,p_t^o}.]  The human model has 52 joints: 22 body joints and 30 hand joints. Fifty-one joints are actuated, and the action  [a_t\\in\\mathbb{R}^{51\\times 3}]  specifies exponential-map joint targets that a PD controller converts into torques. The objective is therefore richer than pose matching: the controller must preserve the interaction outcome while repairing reference errors and accommodating a new embodiment.  The Two-Stage Curriculum                    Stage        Model        Training signal        Main job                            1. Perfect        Subject-specific MLP teachers        PPO with tracking, contact, and energy rewards        Retarget and physically refine small data subsets                    2. Scale        One Transformer student        Refined references, teacher actions, DAgger, then PPO        Integrate many subjects, objects, and interaction skills              The decomposition is a space-time trade-off. Seventeen OMOMO teacher policies can be trained on subject-sized subsets in parallel. Their experience then bootstraps one large student, avoiding the sample cost of discovering every skill through PPO from scratch.  Policy Inputs: Geometry, Contact, and Future Goals  The policy state has a physical-state component (s_t^s) and a goal component (s_t^g):  [s_t={s_t^s,s_t^g}.]  The physical state contains human and object poses and velocities, plus two interaction descriptors:      (d_t): vectors from human joints to the nearest points on each object surface;    (c_t): body-part contact markers derived from applied forces.    The goal provides relative and absolute reference information at several future keyframes. Teachers use offsets (K={1,16}); the student uses (K={1,2,4,16}). The longer temporal context helps the student distinguish skills that look locally similar but require different future actions.  Most MoCap datasets have no reliable contact labels. InterMimic infers reference contact using object acceleration as evidence of human-applied force, then combines it with distance thresholds. Contacts fall into three zones: promote, neutral, or penalize. The neutral buffer is important because a noisy reference distance should not become a rigid physical constraint.  Stage 1: Imitation as Retargeting and Perfecting  Each teacher uses the same canonical human model while learning data from one captured subject. This embeds retargeting inside RL. The paper separates the reward into two kinds of evidence.  Embodiment-aware terms loosely preserve the captured kinematics. Joint-position tracking receives more weight near an object, where spatial precision controls contact; joint-rotation tracking dominates farther away. Joint-to-object distance tracking preserves the interaction layout.  Embodiment-agnostic terms preserve the interaction dynamics through object position and rotation tracking, promoted or penalized contacts, and energy penalties. The authors’ assumption is that body shape may change the exact kinematic trajectory while the human-object dynamics should remain consistent.  For a tracking cost (E), each corresponding reward takes the exponential form  [R_E=\\exp(-\\lambda E).]  The complete scalar reward combines human pose, interaction distance, object pose, contact, and energy terms. The contact-energy penalty discourages abrupt forces and jitter.  Recovering Missing Hand Interaction  OMOMO and BEHAVE often contain averaged or flattened hand poses. When a fingertip or palm lies close to an object, InterMimic activates a hand contact target and lets PPO discover a workable grasp under hand range-of-motion constraints. This recovers functional contact for the paper’s relatively low-dexterity tasks; it does not reconstruct precise captured finger articulation.  Physical State Initialization (PSI)  Reference State Initialization (RSI) starts an episode from a random MoCap frame. With faulty contacts, that frame may already be dynamically unrecoverable: the object drops immediately, or a floating hand cannot reach it in time.  PSI maintains an initialization buffer containing raw reference states and successful simulation states from earlier rollouts. After a rollout, high-return trajectory states enter the FIFO buffer. Future episodes can therefore begin from states that are close to the intended phase and physically achievable. PSI preserves access to later motion phases without repeatedly initializing the simulator inside broken reference configurations.  Interaction Early Termination (IET)  Standard motion-imitation termination checks character deviation and unwanted ground contacts. IET adds HOI-specific failure tests. An episode ends when:      mean object-point deviation exceeds 0.5 m;    the weighted joint-to-object distance differs from the reference by more than 0.5 m; or    a required body-object contact is absent for over 10 consecutive frames.    This keeps PPO from spending rollout budget after the interaction has already failed.  Stage 2: Two Forms of Distillation  After teacher training, the teachers are frozen and rolled out online. Their trajectories provide both states (s^{(T)}) and actions (a^{(T)}).  Reference Distillation  Teacher simulation states replace noisy MoCap states when constructing the student’s goal and reward. This transfers corrected contacts, feasible hand placement, a canonical embodiment, and physically consistent object motion. The student learns from references that its simulator can actually realize.  Policy Distillation and RL Fine-Tuning  The teacher actions supervise the student through behavior cloning and DAgger. Training gradually shifts toward PPO:  [\\mathcal{L}t=w_t\\mathcal{L}{\\mathrm{PPO}}+(1-w_t)\\lVert a^{(S)}-a^{(T)}\\rVert,\\qquadw_t\\uparrow 1.]  Early supervision provides efficient skill acquisition. Later PPO updates optimize the actual interaction reward and resolve ambiguous supervision, such as different teachers choosing different controls for similar observed states. The critic is trained throughout this transition.  The teachers are three-hidden-layer MLPs with widths 1024, 1024, and 512. The student uses a three-layer Transformer encoder with four heads, hidden size 256, and feed-forward size 512. All controllers run at 30 Hz in Isaac Gym.  Experiments  InterMimic draws on OMOMO, BEHAVE, HODome, IMHD, and HIMO. OMOMO is the main large-scale benchmark, with 15 objects and about 10 hours of motion. The authors train 17 subject-specific teachers, reserve subject 14 as the test subject, and discard a small amount of severely corrupted data that the teachers cannot repair.  The metrics measure success rate, imitation duration before IET, human joint-position error (E_h), and object point-position error (E_o).  Teacher-Level Correction  On one BEHAVE subject interacting with a yoga mat, the full teacher substantially improves over SkillMimic:                    Method        Duration ↑        (E_h) (cm) ↓        (E_o) (cm) ↓                            SkillMimic        12.2        7.2        13.4                    InterMimic without IET        40.3        6.7        9.9                    InterMimic wit..."
    },{
      "title": "WARP: Whole-Body Retargeting for Learning from Offline Human Demonstrations",
      "url": "/posts/2026/07/warp-retargeting-paper-notes/",
      "date": "2026-07-02",
      "tags": ["Robot Learning","Motion Retargeting","Whole-Body Manipulation","Imitation Learning","Humanoid Robots"],
      "excerpt": "",
      "text": "WARP is best understood as an analytic retargeting system that makes offline human motion usable as robot supervision. The retargeting part is not a learned neural model and is not a generic weighted IK optimizer. It is a geometry-first pipeline built around a closed-form Shoulder-Elbow-Wrist solver. Learning enters after retargeting: WARP-generated robot trajectories are used to train a behavior-cloning policy.  My read: the important contribution is the separation of two problems that often get mixed together. First, convert human whole-body motion into precise, consistent, robot-feasible actions. Second, train a policy on those actions. WARP argues that if the first step is noisy, multi-modal, or seed-dependent, then behavior cloning inherits that ambiguity as supervision error.  Paper Info  The paper is “WARP: Whole-Body Retargeting for Learning from Offline Human Demonstrations” by Zhenyang Chen, Chuizheng Kong, Chuye Zhang, Yuanshao Yang, Lawrence Y. Zhu, Shreyas Kousik, and Danfei Xu from Georgia Institute of Technology. It is available as arXiv:2606.29940, with project page warp-retarget.github.io.  WARP stands for Whole-body-Aware Retargeting from human Pose.  The Problem  Human demonstrations are attractive because they can be collected without robot hardware in the loop. A person can move naturally, use their torso, route their elbows around obstacles, shift their body to extend reach, and demonstrate contact-rich whole-body behavior. The hard part is that a human demonstration is not directly a robot demonstration.  Offline retargeting is stricter than online teleoperation. During teleoperation, a human operator can watch the robot and compensate when the wrist drifts, contact fails, or the IK solver chooses an awkward branch. Offline data has no such correction. The retargeted trajectory itself becomes the target action sequence for policy learning.  The paper frames two failure modes:      Imprecision: a retargeter may trade off end-effector accuracy against torso or elbow similarity, turning geometric error into action-label error.    Inconsistency: redundant humanoid kinematics can map similar human poses to different robot configurations depending on solver initialization, creating multi-modal labels for behavior cloning.    WARP’s goal is therefore precise and consistent retargeting. The robot should match the task-critical palm pose while preserving the demonstrator’s whole-body structural intent.  Is WARP Optimization-Based Or Learning-Based?  The clean answer is:  WARP retargeting is analytic / geometric. The downstream policy is learning-based.  The retargeter constructs a target robot skeleton and solves closed-form geometric subproblems. It avoids the usual weighted multi-task IK formulation:  [\\dot q^\\ast =\\arg\\min_{\\dot q}\\sum_i w_i\\lVert J_i\\dot q-\\dot x_i^\\ast\\rVert^2+w_{\\mathrm{posture}}\\lVert \\dot q-\\dot q_{\\mathrm{posture}}\\rVert^2.]  That optimization-based baseline, represented by MINK variants in the paper, makes palm tracking, torso orientation, elbow swivel, and posture regularization compete through task weights. WARP instead makes palm matching a hard geometric constraint and resolves the remaining arm configuration through SEW geometry.  Learning appears later. After retargeting, the generated robot trajectory provides proprioceptive states and action supervision for behavior cloning:  [q_t^r = \\mathrm{Ret}(H(t)).]  The policy maps observations and recent robot state history to future action chunks. So WARP is not an end-to-end learned retargeting map; it is an analytic data-conversion pipeline paired with a learned controller.  Core Representation: SEW  The core mechanism is the Shoulder-Elbow-Wrist (SEW) representation. For each arm, WARP represents the human skeleton as:  [H=(s,e,w,\\mathbf{H},t),]  where (s,e,w,t\\in\\mathbb{R}^3) are shoulder, elbow, wrist, and palm/tool positions, and (\\mathbf{H}\\in SO(3)) is the palm orientation. These quantities are expressed in an upper-body-centric frame built from the shoulder line and an anchor point.  The representation separates arm shape from embodiment scale. Human limb configuration is captured by scale-invariant directions:  [u=\\mathrm{unit}(e-s), \\qquad l=\\mathrm{unit}(w-e).]  For humanoid arms with a spherical shoulder, pin-joint elbow, and spherical wrist, SEW admits a closed-form IK solution with at most one valid joint configuration. This is the key to consistency: similar human poses should map to a single robot action, not a family of seed-dependent IK branches.  Vanilla SEW-Mimic aligns limb directions. WARP changes the target. For offline policy learning, the robot palm must land exactly where the human palm indicates task contact. WARP therefore uses SEW as the deterministic IK backbone while adding constrained skeleton alignment for exact palm matching.  WARP Retargeting Pipeline  WARP solves retargeting in two main stages.  First, adaptive offset accounts for link-length differences. If the robot copies the human arm directions (u^{hm}, l^{hm}), its predicted palm (\\hat t^{rb}) may drift from the human palm (t). WARP computes a two-arm centroid offset:  [p_{\\mathrm{offset}}=\\frac{1}{2}(t_L+t_R)-\\frac{1}{2}(\\hat t_L^{rb}+\\hat t_R^{rb}).]  This shifts the robot upper-body origin so the left/right palm centroid aligns with the human palm centroid.  Second, per-arm palm alignment solves each arm independently. The robot hand orientation transfers from the human hand:  [\\mathbf{H}^{rb}=\\mathbf{H}^{hm}.]  The robot wrist is recovered from the desired palm pose and the fixed wrist-to-palm offset (p_{WT}). The hard constraint is:  [w^{rb}+\\mathbf{H}^{rb}p_{WT}=(T^{hm\\leftarrow rb})^{-1}t.]  After shoulder and wrist are fixed, the remaining question is where the robot elbow should lie. WARP transfers the human elbow configuration through the SEW elbow angle (\\psi), computes the elbow half-plane normal (\\hat n), and uses a geometric subproblem, SP3, to find the robot elbow:  [e^{rb}=s^{rb}+R(\\hat n,\\theta_{\\mathrm{SEW}})\\ell_{SE}\\hat e_{SW}.]  The corrected robot skeleton ((s^{rb},e^{rb},w^{rb},\\mathbf{H}^{rb})) is then passed to the SEW solver to recover joint angles in closed form. The paper is explicit that no iterative solver is invoked in this retargeting path. The torso pose is also solved with a closed-form IK-Geo solver.  Lazy Mobile-Base Control  Whole-body human motion implies a base pose, but directly tracking every small upper-body shift with the mobile base would make the wheels chase jitter and introduce lag. WARP’s design is to let the torso absorb fine adjustments while the base moves for real relocation.  It keeps a filtered base target (q_b=(p_b,\\theta_b)) and compares it with the desired base pose (q_d=(p_d,\\theta_d)). Small errors inside a deadband are ignored:  [\\tilde e_{xy}=\\max(0,\\lVert p_d-p_b\\rVert-\\delta_{xy})\\hat e_{xy},\\qquad\\tilde e_\\theta=\\mathrm{sign}(e_\\theta)\\max(0,\\lvert e_\\theta\\rvert-\\delta_\\theta).]  Then a damped second-order filter updates the base:  [\\ddot p_b=\\omega_n^2\\tilde e_{xy}-2\\zeta\\omega_n\\dot p_b,\\qquad\\ddot\\theta_b=\\omega_n^2\\tilde e_\\theta-2\\zeta\\omega_n\\dot\\theta_b.]  This is a small systems detail with a large effect: smooth base motion makes open-loop replay and downstream policy learning less brittle.  Policy Learning After Retargeting  The policy is trained after WARP converts human motion into robot actions. The authors instantiate the policy with HPT, a transformer-based action-chunk model trained with flow matching. The action chunk is decomposed into base, torso, arm, and hand components:  [a_{1:H}=[a_b,a_\\tau,a_r,a_h]_{1:H}.]  The model uses a proximal-to-distal hierarchy:  [b \\preceq \\tau \\preceq r \\preceq h.]  A block-causal attention mask lets proximal body blocks inform distal blocks, while temporal tokens over the action horizon remain jointly visible. The final policy is lightweight, with about 11M trainable parameters. This policy detail matters because WARP’s retargeting is designed to produce unimodal, consistent action labels that a standard behavior-cloning model can learn.  Experiments  The experiments test three claims: WARP improves retargeting quality, WARP-generated data improves downstream policy learning, and whole-body retargeting is necessary for tasks where torso, elbow, or base configuration matters.  For retargeting quality, the authors sample 514 manipulation demonstrations from BONES-SEED-SOMA and compare WARP with SEW-Mimic and MINK variants. MINK-EF tracks end-effectors only. MINK-TE adds soft torso and elbow objectives. WARP achieves very low palm error without joint-limit constraints: 0.0046 mm mean palm position error and 0.046 mm P95 error, compared with 0.701 mm / 1.853 mm for MINK-EF and 18.557 mm / 73.980 mm for MINK-TE. It also has lower joint-limit and self-collision fractions than the MINK variants. The paper reports roughly 30x faster solving than iterative optimization-based retargeters: converting SEED takes about one CPU hour for WAR..."
    },{
      "title": "Orca: The World is in Your Mind",
      "url": "/posts/2026/07/orca-world-foundation-model-paper-notes/",
      "date": "2026-07-02",
      "tags": ["World Models","Multimodal Learning","Embodied AI","Vision-Language-Action","Foundation Models"],
      "excerpt": "",
      "text": "Orca is an early attempt to define a general world foundation model around one central modeling target: learn a unified latent state of the world, then expose that state through downstream readouts for language, images, and robot actions.  My read: the paper’s main move is the shift from isolated next-token, next-frame, or next-action objectives to Next-State-Prediction. Orca treats text, image, and action as interfaces to a shared world latent. The backbone is trained to model state transitions, then frozen; lightweight decoders test whether the latent can be read out into reasoning, future images, and embodied control.  Paper Info  The paper is “Orca: The World is in Your Mind” by the Orca Team, Beijing Academy of Artificial Intelligence. The arXiv page is arXiv:2606.30534, with v1 submitted on June 29, 2026 and v2 revised on June 30, 2026. The project page is orca-wm.github.io.  The core contributors listed in the paper are Yihao Wang, Yuheng Ji, Mingyu Cao, Yanqing Shen, and Runze Xiao, with Yuheng Ji marked as project lead and Zhongyuan Wang and Pengwei Wang as corresponding authors.  The Big Idea  Many current foundation-model families are organized by output format. Language models predict the next token. Image and video models predict or generate frames. VLA models predict actions. Orca proposes a different center of gravity: the model should learn an internal world state and its transitions, and downstream tasks should read from that latent state.  The paper writes this as latent world-state modeling. Given multimodal world signals (X={X^m}_{m\\in M}), the encoder maps them into a latent state:  [S = f_\\theta(X).]  The state then evolves forward or backward under implicit dynamics and explicit conditions:  [S_{t+\\Delta}\\sim p_\\Theta(S_{t+\\Delta}\\mid S_t,z_t,c_t),\\quad \\Delta\\in\\mathbb{Z}_{\\ne 0}.]  Here (z_t) represents unobserved dynamics such as physical laws, object properties, scene dynamics, and environmental forces. (c_t) represents explicit conditions such as human instructions or language-described events. Positive (\\Delta) predicts future states; negative (\\Delta) backtracks to past states.  In this first Orca version, the world signals are vision and language. The authors frame them as two complementary channels: visual signals capture how the world evolves, while language signals describe meaning, intention, causal premises, and event structure.  Two Learning Modes  Orca uses a VLM backbone, based on Qwen3.5, and learns through two complementary paradigms.  Unconscious learning captures dense natural transitions from continuous video. Given a video frame (v_t), Orca predicts the latent of the adjacent next frame (v^l_{t+1}). The target is obtained from a frozen vision encoder, so supervision happens in latent space instead of pixel space. This mode is meant to absorb motion, occlusion, object interaction, and local physical regularities from observation alone.  Conscious learning captures sparse meaningful transitions under language conditions. Videos are segmented into events, and each event has a caption. Given a frame (v_t) and an instruction or event description (e_{t+\\Delta}), Orca predicts the latent of a frame sampled from the specified adjacent event. This lets language describe target transitions such as a future event, past event, task intention, or causal premise.  The same conscious branch also includes VQA response generation. Given a video (V) and question (l_q), the LM head generates answer (l_a), preserving a natural-language interface to the latent.  Pre-Training Objective  The pre-training objective combines three terms:  [\\mathcal{L}=\\lambda_{\\mathrm{obs}}\\mathcal{L}{\\mathrm{obs}}+\\lambda{\\mathrm{evt}}\\mathcal{L}{\\mathrm{evt}}+\\lambda{\\mathrm{vqa}}\\mathcal{L}_{\\mathrm{vqa}}.]  The appendix gives the concrete weights:  [\\mathcal{L}{\\mathrm{pre}}=0.1\\mathcal{L}{\\mathrm{obs}}+0.5\\mathcal{L}{\\mathrm{evt}}+0.4\\mathcal{L}{\\mathrm{vqa}}.]  The latent matching loss is a blend of MSE and cosine distance:  [\\ell_{\\mathrm{lat}}(\\hat v^l,v^l)=0.1\\lVert \\hat v^l-v^l\\rVert_2^2+0.9\\left(1-\\frac{\\langle \\hat v^l,v^l\\rangle}{\\lVert \\hat v^l\\rVert_2\\lVert v^l\\rVert_2}\\right).]  (\\mathcal{L}{\\mathrm{obs}}) predicts the adjacent next-frame latent. (\\mathcal{L}{\\mathrm{evt}}) predicts event-conditioned previous or next event latents. (\\mathcal{L}_{\\mathrm{vqa}}) is the standard next-token loss for answers. The paper mixes state-transition samples and VQA samples at roughly 5:1.  The query mechanism is simple. Observation-only transition uses <Query 1> to read out a predicted visual latent. Event-conditioned transition adds an instruction and <Query 2> to read out the instruction-conditioned target latent. All learnable queries are trained from scratch.  Data and Training Scale  The paper constructs a world-learning inventory with:                    Data type        Scale        Role                            General video        125K hours        Dense visual transitions                    Event annotations        160M        Language-described state transitions                    General VQA        11.5M        Language grounding and world understanding              The video data covers ego-centric interaction, exo-centric manipulation, action-free robot execution, and natural dynamics. Event data is derived from video through coarse and fine event segmentation plus captions. VQA data teaches the model to describe and interpret observed world states.  The current release uses only about one-tenth of the video inventory. The appendix reports 12.5K hours of video used for both Orca-4B and Orca-0.8B training, over 10,844 steps on 32 nodes / 256 GPUs. The base VLMs are Qwen3.5-4B and Qwen3.5-0.8B. The vision encoder is frozen; the LLM part and visual transition head are trainable.  The infrastructure section is practical. The authors use FlagScale with FSDP2, chunked cross-entropy loss, activation recomputation, and communication prefetching. Throughput rises from 0.66 samples/sec/GPU in the StarVLA pipeline to 2.91 samples/sec/GPU in full Orca training, a reported 4.4x improvement.  Readout Design  After pre-training, Orca freezes the backbone and trains only modality-specific readout modules.  Language readout reuses the LM head. It exposes the latent as answers, explanations, and event-level reasoning.  Vision readout maps Orca’s predicted visual latent into image space through a frozen Stable Diffusion 3.5 MMDiT. The trainable parts are an MLP adaptor and LoRA modules. The target image size is 768 x 768, and vision readout training runs for 200K steps.  Action readout maps the latent to robot action chunks. A DiT-based Action Expert with flow-matching loss receives Orca latent, noisy action with time embedding, and proprioception. Only the MLP adaptor and Action Expert are trained. For real-robot tasks, the Action Expert sees 200 trajectories per task across five tasks.  This readout protocol is important because it isolates the question the paper wants to test: does pre-training produce a latent that downstream decoders can use?  Results  The first scaling result is straightforward: total pre-training loss decreases as video data grows, and the 4B model reaches lower loss than the 0.8B model. More importantly, frozen-backbone readouts improve as pre-training scales. The paper reports gains in text generation, image prediction, and action generation as the world latent strengthens.  Text generation. Orca is evaluated on MVBench, TemporalBench, 3DSRBench, and SWITCH. Orca-4B reaches an average score of 51.8, compared with 46.7 for Qwen3.5-4B in the same table. The paper also groups questions into capability dimensions: state transition, commonsense reasoning, spatial relations, and dynamic motion. Orca-4B improves most clearly on state transition and dynamic motion, which matches the training objective.  Image prediction. The paper introduces PRICE-V0.1, a real-world interaction-conditioned image prediction benchmark. Given an initial image and instruction, the model predicts the target state image after the action. Orca-4B+2B reaches 59.8±10.9 average score across Gemini 3.1 Pro, GPT 5.4, Doubao-Seed-2.0, and Gemma 4-31B judges. The closest baseline in the main table, Flux.2 [klein], reaches 56.1±18.1. The qualitative claim is that Orca better preserves scene consistency, robot morphology, contact relationships, and instruction-conditioned state change.  Action generation. The robot evaluation uses five tasks on a dual-arm wheeled humanoid: Take Book, Stacked Bowls, Pull Out Tissue, Stamp, and Scoop Sugar. The benchmark tests environment OOD and object OOD. Orca is compared with V-JEPA 2.1, Qwen3.5, and (\\pi0.5). For overall rule-based score, Orca reaches 32.4, compared with 29.4 for (\\pi0.5), 17.0 for V-JEPA 2.1, and 10.5 for Qwen3.5. Orca is especially strong in environment OOD, with 36.6 rule-based score versus 27.6 for (\\pi0.5). Object OOD is closer: Orca scores 28.2, while (\\pi0.5) scor..."
    },{
      "title": "Inverting the Bellman Equation: From Q-Values to World Models",
      "url": "/posts/2026/07/inverting-bellman-q-values-world-models-paper-notes/",
      "date": "2026-07-02",
      "tags": ["Reinforcement Learning","Goal-Conditioned RL","World Models","Model-Free RL","Theory"],
      "excerpt": "",
      "text": "Inverting the Bellman Equation asks a deceptively simple question: if a model-free, value-based agent has learned (Q(s,a,g)) for many goals, how much of the environment dynamics is already encoded inside those values?  My read: the paper turns the usual RL direction around. Classical value learning fixes the transition kernel (P) and solves for (Q). This work fixes (Q), the policy (\\pi), and the reward family (r), then solves for a compatible transition kernel (P). The resulting method, (P)-learning, is both an extraction algorithm and a conceptual bridge between model-free RL, model-based RL, and goal-conditioned RL.  Paper Info  The paper is “Inverting the Bellman Equation: From (Q)-Values to World Models” by Alistair Letcher, Mattie Fellows, Alexander D. Goldie, Jonathan Richens, Jakob N. Foerster, and Oliver Richardson from FLAIR, University of Oxford, Google DeepMind, and Mila. It appears on arXiv as arXiv:2606.21173, submitted on June 19, 2026. The paper reports code at github.com/aletcher/inverting-bellman.  The high-level claim is that goal-conditioned value functions can carry more than task-specific preference. Under suitable goal coverage, they can identify the environment transition kernel. Even when the theory’s sufficient conditions are stronger than the experiments satisfy, the extracted world models are accurate in Reacher, MountainCar, and stochastic FourRooms variants.  Why This Is Interesting                    Model-free RL is usually described as reward-specific: the agent learns values or policies useful for the training reward, while a model-based agent learns dynamics (P(s’        s,a)). The value-equivalence problem explains why this distinction is not trivial. Many different dynamics can induce the same value function for a single reward, so a value function alone may underdetermine the world.              Goal-conditioned RL changes the information content. A collection of values ({Q_g}_{g\\in G}) gives many Bellman constraints on the same unknown transition kernel. Each goal contributes a different “probe” of the next-state distribution. If these probes are rich enough, the transition kernel becomes identifiable.  The paper’s central question is:  [\\text{When do value-based agents implicitly encode an accurate model of their environment?}]  P-Learning: Bellman Inversion  For a fixed goal-conditioned agent, assume we know:      a (Q)-function (Q(s,a,g)),                                        the policy (\\pi(a’            s’,g)), usually induced by the (Q)-function,                                the reward family (r(s,g)),    the discount factor (\\gamma).    Instead of learning (Q) from environment transitions, (P)-learning learns a candidate world model (P_\\phi) by minimizing the Bellman residual:  [L(\\phi)=\\left|T^\\pi_{P_\\phi}(Q)-Q\\right|_d^2.]  Here the Bellman operator depends on the candidate model:  [T^\\pi_{P_\\phi}(Q)(s,a,g)=\\mathbb{E}{s’\\sim P\\phi(s,a),\\,a’\\sim \\pi(s’,g)}\\left[r(s’,g)+\\gamma Q(s’,a’,g)\\right].]  So (P)-learning searches for dynamics under which the already-learned (Q)-values satisfy the Bellman equation. This is the inverse analogue of (Q)-learning:                    Usual direction        Inverted direction                            Fix (P), learn (Q) such that (T_P(Q)=Q)        Fix (Q), learn (P) such that (T_P(Q)=Q)                    Environment samples provide next states        The candidate model samples next states                    Target moves as (Q) changes        Objective is stationary for fixed (Q,\\pi,r)              For finite state spaces, the paper rewrites the Bellman equation as a linear system. Define  [M(s’,g)=r(s’,g)+\\gamma V(s’,g),]  where (V(s’,g)=\\mathbb{E}_{a’\\sim\\pi(s’,g)}[Q(s’,a’,g)]). For each state-action pair:  [M P(s,a)=Q(s,a).]  This makes the extraction problem especially clear. The rows of (M) are Bellman probes indexed by goals; the unknown is the next-state distribution (P(s,a)). If (M) has enough rank, (P(s,a)) is determined.  The tabular (P)-learning update is:  [P_{n+1}(s,a)=P_n(s,a)-\\alpha M^\\top(MP_n(s,a)-Q(s,a)).]  Theorem 1 shows that this converges to:  [P_\\infty(s,a)=M^+Q(s,a)+(I-M^+M)P_0(s,a),]  where (M^+) is the Moore-Penrose pseudo-inverse. The second term is the remaining value-equivalence ambiguity. If (M) has full column rank, that ambiguity vanishes and the solution becomes unique.  When Are Dynamics Identifiable?  The theoretical section gives sufficient conditions for recovering (P) from (Q,\\pi,r). A useful way to remember the results is:                    Setting        What goals need to do                                                                Deterministic finite MDP        A single generic goal can be enough                                                        Stochastic finite MDP        Goals need to span the state space; (        G        \\ge        S        ) is a clean sufficient condition                    Local stochastic finite MDP        Fewer goals can suffice if transition support is local or known                                                        Deterministic continuous MDP        Coverage or structured Gaussian goals can separate successor states                                                        Stochastic continuous MDP        The paper proves coverage-based results mainly for unconditional policies                                                  The intuition is test functions. For each goal (g), the Bellman equation gives:  [Q(s,a,g)=\\mathbb{E}_{s’\\sim P(s,a)}[M_g(s’)].]  So each (M_g) is a known test function whose expectation under the unknown next-state distribution must match (Q(s,a,g)). One goal gives one expectation constraint. Many sufficiently different goals can determine the whole distribution, much like enough moments can determine a finite distribution.  This also clarifies the deterministic case. If dynamics are deterministic, the unknown next state is a point, so the columns (M(\\cdot,g)) only need to distinguish states. A single generic reward can make those columns distinct. For stochastic dynamics, the unknown is a distribution over states, so spanning or rank conditions are stronger.  Experiments  The experiments test whether this extraction works when the theoretical conditions are only partly met. Agents are trained with PQN plus Hindsight Experience Replay, then (P)-learning extracts a world model from the converged (Q)-values.  Reacher. The agent is trained on only four position goals on the unit circle. The continuous state includes joint angles, angular velocities, and fingertip position, but the training rewards depend only on fingertip position. Despite imperfect learned (Q)-values, the extracted world model is very accurate: the paper reports world-model NMSE around (1.2\\times 10^{-4}). Policies trained entirely inside the extracted model are quasi-optimal on unseen goals, including velocity goals that were outside the position-only reward family.  The architecture sweep is also important. Across 42 Reacher architectures and 10 seeds each, better goal-conditioned agents tend to have more accurate extracted world models. The paper reports a Spearman correlation of (-0.98) between agent return and world-model error, and (+0.95) between agent return and the return of policies trained inside the extracted model on unseen goals.  MountainCar. The position-trained agent uses four sparse position goals. The extracted dynamics match the true transition kernel well, with NMSE around (6.7\\times 10^{-3}). Planning inside the extracted world model works on out-of-distribution goals such as velocity targets and constrained hill-climbing. A second agent trained on velocity goals yields a world model extremely close to the position-trained agent’s world model, suggesting that different reward views of the same environment can lead to similar implicit dynamics.                    FourRooms. The finite-state experiments test deterministic, windy, and teleporting gridworlds. In deterministic FourRooms, one generic training goal recovers the world model exactly, matching the finite deterministic theory. In windy FourRooms, four goals bring the world-model-derived policy within about 1% of optimal return. In teleporting FourRooms, 20 goals work well even though the worst-case stochastic finite-state guarantee would suggest (        G        =        S        =68).              Strengths and Limitations  The strongest part of the paper is the clean inversion view. It gives a concrete procedure for asking what dynamics are already latent in a trained value function. That makes the result relevant to transfer, interpretability, auditing, and hybrid RL: a model-free agent may already contain a usable model, even if it never explicitly represented one during training.  The second strength is the theory-experiment contrast. The theorems explain when recovery is guaranteed, while the experiments show that useful recovery can happen with far fewer goals than the worst-case ..."
    },{
      "title": "HRDexDB: A Paired Human-Robot Dataset for Cross-Embodiment Dexterous Grasping",
      "url": "/posts/2026/07/hrdexdb-paper-notes/",
      "date": "2026-07-02",
      "tags": ["Dexterous Manipulation","Cross-Embodiment Learning","Robot Learning","Human-Robot Transfer","Dataset"],
      "excerpt": "",
      "text": "HRDexDB is a dataset paper, but its real contribution is a data contract for studying cross-embodiment dexterity. It captures human grasps and multiple dexterous robot grasps on the same objects, in the same calibrated workspace, with reconstructed 3D hand motion, robot states, object 6D trajectories, egocentric views, and tactile signals where the robot hardware supports them.  My read: the paper is useful because it moves the question from “can we retarget a human hand pose to a robot hand?” to “can we learn how the contact strategy itself changes across embodiments?” That distinction matters for dexterous grasping. Different hands can share the same grasp intent while using different fingers, contact patches, timing, and feasible force closure patterns.  Paper Info  The paper is “HRDexDB: A Paired Human-Robot Dataset for Cross-Embodiment Dexterous Grasping” by Jongbin Lim, Taeyun Ha, Mingi Choi, Jisoo Kim, Byungjun Kim, Subin Jeon, and Hanbyul Joo from Seoul National University and RLWRLD. It is available as arXiv:2604.14944, with project page snuvclab.github.io/HRDexDB.  The Problem  Human manipulation is an attractive source of demonstrations because most everyday objects are designed around the human hand. The hard part is that human hands and robot hands differ in morphology, kinematics, actuation, sensing, and feasible contacts. Even two robot hands can require different grasp strategies for the same object.  Existing datasets usually cover one side of the problem. Human-object interaction datasets provide rich human hand motion and object annotations. Robot datasets provide robot trajectories, often with grippers or a single dexterous embodiment. Paired human-robot datasets exist, but the paper argues that they lack the combination HRDexDB targets: markerless RGB observations, multiple dexterous robot hands, shared objects, 3D hand and object annotations, and tactile signals in a unified setup.  The central missing ingredient is paired behavior. For cross-embodiment learning, the dataset should show a human grasp and a robot grasp that are semantically comparable on the same object, while still allowing embodiment-specific differences in motion and contact.  Dataset Contract  HRDexDB contains 2.1K grasping sequences, 24M frames, 100 objects, and five embodiments: one human hand plus four robotic hands, namely Allegro Hand V4, Allegro Hand V5 Plus, Inspire Hand RH56DFTP, and Inspire Hand RH56F1. The capture system uses 21 synchronized exocentric RGB cameras and 2 egocentric views, giving 23 synchronized RGB streams in total. The dataset also includes scanned object meshes, object 6D pose trajectories, success/failure labels, and tactile force signals for tactile-enabled robot fingertips.  The robot trial representation is:  [T^{robot}=\\left{{I_t^{c_i}}{i=1}^{21}, I_t^{ego}, q_t^{robot}, T_t^{object}, F_t^{tactile}, y\\right}{t=1}^{T_r}.]  The human trial representation is:  [T^{human}=\\left{{I_t^{c_i}}{i=1}^{21}, I_t^{ego}, \\theta_t^{human}, T_t^{object}, y\\right}{t=1}^{T_h}.]  Here (q_t^{robot}) is the robot state, (\\theta_t^{human}\\in\\mathbb{R}^{51}) is the MANO hand pose, (T_t^{object}\\in SE(3)) is the object pose, (F_t^{tactile}) is tactile force data, and (y\\in{0,1}) marks grasp success.  This representation is compact but important. It aligns visual, kinematic, object, tactile, and outcome signals in one coordinate system, making the dataset usable for both manipulation learning and perception evaluation.  Capture And Reconstruction  The acquisition protocol has two stages. A human subject first performs a natural grasp on the target object. Then a teleoperator observes that demonstration and performs a semantically corresponding grasp with the target robot hand. The teleoperation setup uses an Xsens inertial motion-capture suit and MANUS gloves to map wrist and finger motion to the robot arm and hand.  This protocol avoids forcing an exact joint-level correspondence. The robot execution preserves the grasp intent while adapting to the robot’s own morphology. That is the point of the dataset: the paired examples expose how successful contact strategies change when the hand changes.  The reconstruction pipeline combines several vision components:      Human hand motion: HaMeR detects 2D hand keypoints in each calibrated view, triangulation produces 3D joints, and MANO fitting recovers pose parameters. Subject-specific shape is calibrated with SAM3 masks and silhouette alignment.    Object pose: FoundationStereo estimates dense depth from a calibrated stereo pair, SAM3 segments the object, and FoundationPose estimates and tracks the CAD model pose through time.    Multi-view consistency: the object mesh is rendered into all calibrated views, and silhouette misalignment is minimized to reduce drift during long grasps.    The practical challenge is occlusion. Dexterous hands and objects hide each other constantly, so the paper’s multi-camera system is not decoration; it is what makes markerless 3D supervision plausible.  Benchmark 1: Contact Map Transfer  The first downstream task asks: given a human contact pattern on an object, can a model predict the robot-specific contact pattern that will actually work for a target hand?  Each grasp is represented on an object point cloud (O\\in\\mathbb{R}^{N\\times3}) with a contact map (C\\in[0,1]^N) and a part map (P). The human part map uses six parts, while the robot part map uses (B=6) for Inspire and (B=5) for Allegro. Conditioned on the human representation ([C^h,P^h]) and PointNet++ object features, the model predicts ([C^r,P^r]). Training uses a contact-weighted L1 loss for (C^r) and cross-entropy over contacted points for (P^r).  The predicted robot contact map is then used as the objective for a CEDex-style physics-aware grasp optimizer with contact, penetration, and self-collision terms. The comparison is clean: the optimizer stays fixed, and only the contact objective changes.  The results show why paired data matters. Directly using the human contact map reaches 66.7% real-world success for Inspire and 63.3% for Allegro. Using the transferred robot-specific contact map improves this to 73.3% and 80.0%. In simulation, the same trend appears, with Allegro improving from 60.2% to 65.8%.  Benchmark 2: Latent-Space Grasp Retrieval  The second task learns a shared embedding space across human, Inspire, and Allegro grasps. Given a human hand-object grasp as query, the model retrieves robot grasp candidates from HRDexDB whose geometry and function match the query.  The architecture is CLIP-like: separate point-cloud encoders process the human hand, Inspire hand, Allegro hand, and object, then project query and candidate branches into a shared latent space with a symmetric contrastive loss.  Retrieval is evaluated in two ways. First, the paper measures whether paired robot grasps are ranked highly among 33 candidates. Human-to-Inspire retrieval reaches 36.36% R@1, 81.82% R@3, and 100.00% R@5. Human-to-Allegro is harder, at 24.24% R@1, 63.64% R@3, and 72.73% R@5.  Second, retrieved grasps are used to initialize BODex refinement. This is a useful test because a retrieved prior only matters if it helps downstream optimization. Retrieval-top5 gives the best episode-level success: 75.76% for Inspire-F1 and 93.94% for Allegro-V5, outperforming vanilla BODex and kinematic retargeting. Retrieval-top1 gives higher seed-level precision, while top5 gives better coverage across episodes.  Benchmark 3: 3D Hand Pose Estimation  HRDexDB also functions as a perception benchmark. The paper evaluates WiLoR, HaMeR, Hamba, MeshGraphormer, and FrankMocap on captured human hand-object sequences.  The dataset is harder than FreiHAND for most methods. For example, WiLoR’s PA-MPVPE increases from 5.27 mm on FreiHAND to 6.09 mm on HRDexDB, and MeshGraphormer’s PA-MPJPE increases from 6.64 mm to 8.31 mm. The reason is expected: hand-object grasping causes persistent occlusion and contact-rich hand shapes, which are underrepresented in cleaner single-hand benchmarks.  The paper also tests whether HRDexDB adds useful training signal. Adding 6K HRDexDB samples to a 2.7M-sample finetuning pool slightly improves HaMeR and WiLoR on FreiHAND. The improvement is modest, but it supports the claim that the data is complementary instead of redundant.  Benchmark 4: Object 6D Pose Under Grasping  The object pose benchmark compares FoundPose, GigaPose, PicoPose, and MegaPose-refined variants on paired human-grasp and robot-grasp frames. All methods receive the same RGB image, mask, camera intrinsics, object identity, and CAD model.  Robot grasping is consistently harder than human grasping. FoundPose with MegaPose refinement reaches 3.35 cm ADD and 70.00% ARMSSD on human-grasp frames, but drops to 4.40 cm ADD and 64.10% ARMSSD on robot-grasp frames. The paper’s explanation is plausible: rigid robot links and fingertips introduce visual structures that overlap object boundaries and confuse pose localization.  The au..."
    },{
      "title": "DexWorldModel: Causal Latent World Modeling towards Automated Learning of Embodied Tasks",
      "url": "/posts/2026/07/dexworldmodel-clwm-paper-notes/",
      "date": "2026-07-02",
      "tags": ["World Models","Embodied AI","Robotic Manipulation","Sim-to-Real","Vision-Language-Action"],
      "excerpt": "",
      "text": "DexWorldModel proposes a Causal Latent World Model (CLWM) for manipulation. Its main design choice is to move world-action generation away from pixel reconstruction and into a semantic latent space: predict future DINOv3 features, then decode action chunks conditioned on those predicted future semantics.  My read: the paper is about making world-action models deployable, not just imaginative. CLWM targets three bottlenecks at once: pixel-level prediction wastes capacity on visual texture, autoregressive history makes memory grow with horizon, and diffusion-style generation slows closed-loop control. The proposed system answers these with DINOv3 latent targets, Dual-State TTT Memory, and Speculative Asynchronous Inference.  Paper Info  The paper is “DexWorldModel: Causal Latent World Modeling towards Automated Learning of Embodied Tasks” by Yueci Deng, Guiliang Liu, and Kui Jia from DexForce AI. It appears on arXiv as arXiv:2604.16484, submitted on April 13, 2026.  The paper also introduces EmbodiChain as the training-data engine behind CLWM’s post-training stage. The referenced platform is github.com/DexForce/EmbodiChain.  Core Problem  World Action Models (WAMs) extend VLA policies by generating future world states before generating actions. A common formulation is:  [\\hat{o}{t+1}\\sim p\\theta(\\cdot\\mid o_{\\le t},a_{<t},l),]  [a_t\\sim g_\\psi(\\cdot\\mid o_{\\le t},a_{<t},\\hat{o}_{t+1},l).]  This gives the policy an explicit future-state interface. The cost is heavy: if (\\hat{o}_{t+1}) lives in pixel space or low-level VAE latent space, the model spends capacity on lighting, texture, and background details. If the model keeps all history through a Transformer KV cache, memory scales as (O(T)). If each action waits for sensing, full denoising, and execution in sequence, the robot pays a large latency penalty at every control chunk.  DexWorldModel’s thesis is that a manipulation world model should predict the part of the future that matters for action: interaction semantics.  CLWM Architecture  CLWM has two coupled generative modules under a Mixture of Transformers (MoT) design:                    Module        Target        Role                            Latent Video Model        future DINOv3 feature (f_{t+1})        predicts semantic world evolution                    Action Model        action chunk (a_t)        decodes control from predicted future semantics              The visual feature is extracted by a frozen DINOv3 model:  [f_t=\\Phi_{\\mathrm{DINO}}(o_t)\\in\\mathbb{R}^{C\\times H’\\times W’}.]  The paper uses patch size (P=16). The two generators share a core Transformer initialized from Wan2.2-5B, while keeping separate input/output projections and flow-time embeddings:  [\\phi_{\\mathrm{vid}}=\\phi^{out}{\\mathrm{vid}}\\circ\\phi{\\mathrm{share}}\\circ\\phi^{in}{\\mathrm{vid}},\\quad\\phi{\\mathrm{act}}=\\phi^{out}{\\mathrm{act}}\\circ\\phi{\\mathrm{share}}\\circ\\phi^{in}_{\\mathrm{act}}.]  Generation is autoregressive and causal. First, the Latent Video Model predicts the next semantic feature via conditional flow matching:  [\\mathcal{L}{\\mathrm{video}}=\\mathbb{E}\\left[\\left\\lVertv{\\phi_{\\mathrm{vid}}}(f^{(s)}{t+1},s\\mid h{\\le t},l)-\\dot{f}^{(s)}_{t+1}\\right\\rVert^2\\right].]  Then the Action Model predicts an action chunk:  [a_t={a_{t,1},a_{t,2},\\ldots,a_{t,\\tau}},]  with (\\tau=16) in the experiments. The action vector is compact for dual-arm control: each arm contributes a 7-DoF end-effector pose, 7 joint positions, and 1 gripper state, giving ((7+7+1)\\times 2=30) continuous dimensions.  The action loss is also a flow-matching objective, conditioned on history, language, and the predicted future feature:  [\\mathcal{L}{\\mathrm{action}}=\\mathbb{E}\\left[\\left\\lVertv{\\phi_{\\mathrm{act}}}(a^{(s)}t,s\\mid \\tilde{h}{\\le t},l,\\tilde{f}_{t+1})-\\dot{a}^{(s)}_t\\right\\rVert^2\\right].]  A small but important detail is history augmentation. During training, with probability (p=0.5), the model injects Gaussian noise into historical latent features:  [\\tilde{f}{\\le t}=(1-s{\\mathrm{aug}})\\epsilon+s_{\\mathrm{aug}}f_{\\le t},\\quad s_{\\mathrm{aug}}\\in[0.5,1].]  This teaches the action model to tolerate imperfect predicted histories. That matters later because SAI deliberately starts inference from speculative future features before the real observation arrives.  Dual-State TTT Memory  Standard autoregressive world models preserve history through KV cache. For long-horizon robot interaction, that creates a growing memory and latency burden:  [\\text{KV cache memory}\\sim O(T).]  CLWM replaces the cache with a Test-Time Training (TTT) Memory layer. Instead of storing all historical tokens, the model compresses history into dynamically updated layer weights. The TTT-MLP is trained through a self-supervised reconstruction objective:  [\\ell_{\\mathrm{self}}(W;z_t)=\\lVert f(\\theta_K z_t;W)-\\theta_V z_t\\rVert^2.]  After the inner-loop weights update to (W_t), the query projection extracts the hidden state:  [l_t=f_{\\mathrm{TTTmlp}}(\\theta_Q z_t;W_t).]  The paper wraps this in a Dual-State design:                    Memory        Updated by        Purpose                            Long-Term TTT Memory        real observations and executed actions        anchors true physical history                    Working TTT Memory        forked copy plus predicted future features        conditions short-term generation              The long-term memory update is:  [W^{long}t=W^{long}{t-1}-\\eta\\nabla_W\\ell_{\\mathrm{self}}(W^{long}_{t-1};h_t).]  During generation, CLWM forks:  [W^{work}_t\\leftarrow W^{long}_t.]  The working memory stays frozen during continuous ODE integration, then updates once the predicted future feature (\\hat{f}_{t+1}) is produced:  [W^{work\\prime}t\\leftarrowW^{work}_t-\\eta\\nabla_W\\ell{\\mathrm{self}}(W^{work}t;\\hat{f}{t+1}).]  This separation is the paper’s cleanest systems idea. Real observations update the durable memory; imagined future features update a temporary fork. The model can use predicted context for action generation while keeping physical history anchored to measured data.  Speculative Asynchronous Inference  Even with constant memory, a normal robot loop is sequential:      execute the current action;    wait for the next observation;    run denoising to predict the next future state and action;    execute again.    CLWM uses Speculative Asynchronous Inference (SAI) to overlap neural computation with robot motion. While the robot executes (a_{t-1}), the true next observation (o_t) is unavailable. CLWM already predicted (\\hat{f}_t), so it treats that feature as a surrogate observation and starts partial denoising in the background:  [s=0\\rightarrow s_{\\mathrm{mid}}.]  When the real observation arrives, DINOv3 extracts (f_t), the Long-Term TTT Memory is calibrated with ground truth, and the ODE solver only finishes the remaining denoising interval:  [s_{\\mathrm{mid}}\\rightarrow 1.]  The paper reports that this cuts blocking latency by about 50% compared with a sequential autoregressive pipeline. The history augmentation from training is what makes the speculative phase plausible: the model has already learned to generate stable vector fields under noisy or partially predicted histories.  EmbodiChain and Online Data Streaming  The other half of the paper is the data engine. CLWM pretraining uses aggregated open-source robot manipulation datasets, mainly RoboMind, Agibot World Beta, and InternData-A1. For post-training, the paper says it avoids manually collected real-world or downstream demonstrations and relies on EmbodiChain to generate physics-grounded simulation data.  EmbodiChain has three roles:                    Component        What it adds                            Generative simulation        creates assets, scenes, layouts, physical metadata, and simulation-ready USD assets                    Domain expansion        expands trajectories through reachability-aware sampling, recovery data, visual augmentation, and physics-grounded variation                    Online Data Streaming        streams fresh synthetic trajectories into training through a lock-free shared-memory pipeline              The training principle is called the Efficiency Law of Embodied Intelligence. The paper frames useful scaling through experience throughput (E), the volume of unique state-action pairs consumed per training iteration. For fixed compute (C) and parameters (P), performance improves when fresh experience throughput exceeds a critical threshold:  [E>\\tau(C,P).]  In practice, this means the optimizer should see continuously refreshed, physically valid interactions. A static dataset of the same nominal size can still overfit because each trajectory is replayed too many times.  Training Setup  The reported implementation is substantial:                    Detail        Value                            Visual encoder        DINOv3 base, frozen                    Generative backbone        MoT initialized from Wan2.2-5B                    Patch size ..."
    },{
      "title": "CHORD: Contact Wrench Guidance From Human Demonstration",
      "url": "/posts/2026/07/chord-contact-wrench-guidance-paper-notes/",
      "date": "2026-07-02",
      "tags": ["Robot Learning","Dexterous Manipulation","Human Demonstration","Reinforcement Learning","Sim-to-Real"],
      "excerpt": "",
      "text": "CHORD studies a central problem in dexterous robot learning: human demonstrations contain rich manipulation knowledge, yet robot hands cannot directly replay human hand motion. Different morphology, kinematics, fingertip geometry, and contact timing can turn the same-looking motion into a different object effect.  My read: the paper’s useful abstraction is object-centric contact wrench guidance. CHORD asks the robot to reproduce the object-level mechanical effect of a human contact, measured by the force-torque directions that contacts can induce on the object. This gives reinforcement learning a contact-rich reward that is less tied to a specific hand shape than raw contact positions.  Paper Info  The paper is “Learning Dexterous Manipulation Using Contact Wrench Guidance From Human Demonstration” by Xinghao Zhu, Zixi Liu, Shalin Jain, Chenran Li, Milad Noori, Huihua Zhao, John Welsh, Michael Andres Lin, Wei Liu, Tingwu Wang, Xingye Da, Zhengyi Luo, Vishal Kulkarni, Naema Bhatti, Yuke Zhu, Linxi Fan, Bowen Wen, Danfei Xu, Soha Pouya, and Yan Chang from NVIDIA. The project page is nvidia-isaac.github.io/video_to_data/chord, and the technical report PDF is linked there.  CHORD stands for Contact Wrench Guidance from Human Demonstration in Robotic Dexterous Manipulation. The reported benchmark contains 4,739 bimanual dexterous manipulation tasks, and the main evaluation covers 1,831 tasks. The paper reports 82.12% average success on this large-scale simulation evaluation, 90.77% success on whole-body manipulation tasks, and successful transfer to a real Dexmate robot with two Sharpa dexterous hands.  Why Human Motion Is Not Enough  A human demonstration gives at least three signals: hand motion, object motion, and hand-object contact. Purely tracking object motion is often too sparse for reinforcement learning because the robot receives useful feedback only after making the right contact. Purely tracking hand motion is brittle because the robot hand may need a different pose to generate the same effect. Tracking contact locations helps, yet it can still be mechanically wrong: touching the same object patch with a different surface normal or force direction may push, lever, or slide the object in the wrong way.  CHORD’s answer is to move the comparison into wrench space. A contact is judged by the force and torque directions it can apply to an object part. This makes the target more functional: the robot can choose embodiment-specific contacts as long as those contacts support object motion aligned with the human reference.  The Core Formulation  Each reference trajectory is written as  [\\tau^{\\mathrm{ref}}={x_t^{\\mathrm{human}}, x_t^{\\mathrm{object}}}_{t=1}^{H},]  where the human term contains 3D hand keypoints and the object term contains the poses of (K) rigid bodies or articulated parts. The human hand motion is first retargeted by inverse kinematics to a robot reference (x_t^{\\mathrm{robot}}). The policy then receives robot observations, object observations, and reference motion, and learns actions that make the rollout object poses track the reference.  The reward has three parts:  [r = r_{\\mathrm{task}} + r_{\\mathrm{imit}} + r_{\\mathrm{contact}}.]  The task term tracks object-part poses. For multi-object interactions such as insertion, pouring, scooping, and tool use, CHORD also adds a relative object-pose reward that activates during demonstrated object-object interaction phases. The imitation term keeps the robot near the retargeted motion. The contact term is the main contribution.  For object part (k), CHORD extracts human contact positions (p_{h,k}^{i}) and normals (n_{h,k}^{i}) in the object frame. It approximates each Coulomb friction cone with (d) edge forces. Each edge force creates a primitive wrench:  [w_{h,k}^{i,j}= \\begin{bmatrix}f_{h,k}^{i,j} p_{h,k}^{i} \\times f_{h,k}^{i,j}\\end{bmatrix}\\in \\mathbb{R}^{6}.]  Collecting all primitive wrenches gives the human wrench matrix:  [\\mathcal{W}{h,k}\\in \\mathbb{R}^{6\\times(c{h,k}d)}.]  Directly comparing human and robot wrench matrices is awkward because the two hands may have different contact counts and different ordering of contact primitives. CHORD therefore compares the geometry of the induced wrench set through a support function. With pre-sampled unit basis directions (\\mathcal{B}\\in\\mathbb{R}^{6\\times b}),  [\\sigma_{h,k} = \\max_{\\mathrm{col}}(\\mathcal{B}^{\\top}\\mathcal{W}_{h,k})\\in\\mathbb{R}^{b},]  and (\\sigma_{r,k}) is computed the same way for the robot. The contact wrench-space reward then keeps the robot support close to the human support under a relative tolerance (\\beta):  [r_{\\mathrm{cws}}^{k}= \\exp\\left(-\\frac{\\lVert\\max(0,(1-\\beta)\\sigma_{h,k}-\\sigma_{r,k})\\rVert_2^2}{v_{\\mathrm{cws}}}-\\frac{\\lVert\\max(0,\\sigma_{r,k}-(1+\\beta)\\sigma_{h,k})\\rVert_2^2}{v_{\\mathrm{cws}}}\\right).]  The lower-bound term encourages the robot to supply enough support in the same wrench directions as the human contact. The upper-bound term prevents excessive or mechanically unrelated support. The paper also adds penalties for unintended contacts and missed contacts because the exponential kernel alone would still give a positive value for mismatched contact states.  Training Recipe  CHORD still uses practical scaffolding around the reward. It uses a virtual object controller (VOC) during training, resets the simulator to arbitrary frames along the reference trajectory, keeps VOC active for a short stabilization window, and anneals assistance through a curriculum. The policy uses residual actions around the retargeted robot motion, so RL learns corrections instead of rediscovering the whole movement from scratch.  The paper adds two robustness choices. First, object parts are perturbed by wrenches sampled from the human contact wrench matrix, which creates disturbances aligned with the demonstrated contact mechanics. Second, when contact estimates are noisy, CHORD can switch from exact wrench matching to a reduced force-closure-style objective:  [r_{\\mathrm{fc}}^{k}=\\frac{1}{B}\\sum_{b=1}^{B}\\mathbf{1}[\\sigma_{r,k,b}>\\epsilon].]  That fallback is less behavior-specific, but it avoids trusting corrupted contact normals and positions from noisy video reconstruction.  Benchmark and Results  The benchmark is a large part of the contribution. CHORD processes human hand-object interaction data from ARCTIC, TACO, HOT3D, OakInk2, DexYCB, GRAB, and H2O, then imports the retargeted tasks into Isaac Lab. The resulting library includes rigid objects, articulated objects, multi-object interactions, long horizons, and dense contact events.  On 1,831 sampled tasks, CHORD reports an average success rate of 82.12% with one shared training recipe across tasks. Compared with prior methods under their own evaluation protocols, CHORD matches or exceeds DexMachina, ManipTrans, and SPIDER across the tested suites. The contact reward ablations are especially clear:                    Setting        Main finding                            CHORD contact wrench support        Best performance on box grabbing, mixer use, and whole-body ablations                    Contact position reward        Weaker when the same object patch does not imply the same force/torque effect                    No contact reward        Weakest on contact-rich manipulation because kinematic tracking alone gives poor exploration guidance              The paper also reports that normalized contact wrench reward correlates strongly with task success across 1,831 runs, with Pearson correlation around 0.80. That is an important empirical check: the proposed reward is not just elegant mechanics; it tracks downstream manipulation performance.  For whole-body manipulation, CHORD handles both hand-only references and third-person whole-body references. Hand-only references can be expanded with a motion inpainting module that predicts whole-body motion from end-effector trajectories. For noisy third-person reconstructions, the reduced force-closure objective is used. The whole-body evaluation reports 90.77% success, and a cross-embodiment ablation shows a large gap between wrench guidance and contact position guidance when transferring five-finger human contacts to the three-fingered Dex3 hand on a Unitree G1.  Real-World Deployment  The real-world experiments use a Dexmate robot with two Sharpa dexterous hands. The paper tests both open-loop action-chunk execution and closed-loop inference. Object and robot poses are tracked with a six-camera Vicon setup, and the deployed policy runs as an ONNX model at 20 Hz while lower-level bridges hold commands at higher frequency.  This is a state-based real-world transfer result, not a vision policy. That matters for interpreting the result: the paper shows that the learned contact behavior can leave simulation, while visual perception remains future work.  Strengths and Limitations  The strongest part of CHORD is the abstraction boundary. It does not ask morphology-specific robot hands to co..."
    },{
      "title": "Shape Your Body: Value Gradients for Multi-Embodiment Robot Design",
      "url": "/posts/2026/06/shape-your-body-paper-notes/",
      "date": "2026-06-29",
      "tags": ["Robot Learning","Robot Co-Design","Multi-Embodiment Learning","Reinforcement Learning","Value Functions"],
      "excerpt": "",
      "text": "Shape Your Body turns a multi-embodiment value function into a reusable robot-design model. The key idea is simple and useful: train one embodiment-aware policy and critic across many randomized robot bodies, freeze them, then optimize a new robot’s continuous design parameters by differentiating through the critic.  My read: the paper is best understood as amortized co-design. Traditional robot co-design repeatedly proposes a body and retrains or adapts a controller. Here, most of the expensive learning is paid once during multi-embodiment RL training. After that, robot design becomes a fast gradient-based search over morphology, actuator, and control parameters.  Paper Info  The paper is “Shape Your Body: Value Gradients for Multi-Embodiment Robot Design” by Nico Bohlinger and Jan Peters from Technical University of Darmstadt, Robotics Institute Germany, DFKI, and hessian.AI. It is available as arXiv:2606.00702, with project page nico-bohlinger.github.io/shape-your-body.  The Problem  Robot embodiment and robot control are coupled. A controller can only exploit the physical capabilities that the body gives it, while a good body should be evaluated under a competent controller. Many co-design methods therefore use a bi-level loop: an outer optimizer proposes robot designs, and an inner loop trains or adapts a controller for each candidate. That is expensive, especially when the design vector includes hundreds of continuous parameters such as masses, inertias, joint limits, actuator force limits, velocity limits, geometry, PD gains, and action scales.  This paper changes the unit of reuse. Instead of solving a new co-design problem from scratch for every target robot, it trains a generalist policy and value function over a distribution of robot embodiments. The policy learns to control many bodies. The critic learns to predict how promising a given state and embodiment are under that shared policy. Once trained, the critic becomes a differentiable surrogate objective for downstream design.  Core Formulation  Each robot design starts as a normalized vector (f \\in [-1,1]^{d_{\\mathrm{design}}}). A differentiable map (\\Phi(f)) turns this normalized design into physical embodiment parameters (e): joint origins, ranges, torque and velocity limits, damping, stiffness, friction, armature, body mass, inertia, center of mass, link geometry, foot properties, PD gains, and related quantities.  For an embodiment-aware policy (\\pi_\\theta(a_t \\mid s_t,e)), the expected return is:  [J(\\theta,e)=\\mathbb{E}{\\tau\\sim\\pi\\theta(\\cdot\\mid\\cdot,e)}\\left[\\sum_{t=0}^{T}\\gamma^t r_e(s_t,a_t)\\right].]  During training, each parallel environment samples a design from the current curriculum range, maps it through (\\Phi), and keeps that design fixed for the episode. The training objective is the expected return over the full design distribution:  [\\max_\\theta\\;\\mathbb{E}{f\\sim U([-1,1]^{d{\\mathrm{design}}})}\\left[J(\\theta,\\Phi(f))\\right].]  After this stage, both policy and critic are frozen. Downstream robot design no longer runs a full RL loop.  Value-Gradient Design Search  The search method is called Value-Gradient Design Search (VGDS). It first collects a state bank (S={s_1,\\ldots,s_M}) from rollouts of the final policy. Then it optimizes the design vector directly against the frozen critic. With an ensemble of (K) value heads, the mean value prediction is:  [\\bar{V}(s,\\Phi(f))=\\frac{1}{K}\\sum_{k=1}^{K}V_k(s,\\Phi(f)).]  Naively maximizing the critic is dangerous because the design may move toward regions where the learned value function extrapolates badly. The paper therefore anchors search around a reference design (f_{\\mathrm{ref}}), usually a nominal URDF or other valid robot, through a soft trust-region objective:  [\\hat{J}\\lambda(f)=\\frac{1}{M}\\sum{m=1}^{M}\\bar{V}(s_m,\\Phi(f))-\\lambda\\frac{\\lVert f-f_{\\mathrm{ref}}\\rVert_2^2}{d_{\\mathrm{design}}}.]  The gradient has two parts: the value gradient from backpropagating through the critic and design map, and the analytic trust-region gradient that pulls the design toward the reference. VGDS applies Adam ascent on minibatches from the state bank, clips each parameter update, and clips the final vector back into the valid design range.  This is the conceptual center of the paper. A value function trained for control becomes a design critic. The design loop is cheap because each step is only batched neural-network inference and backpropagation, not another RL run.  Critic Architecture Detail  The paper builds on URMA, a multi-embodiment architecture that handles variable robot topology by encoding per-joint observations and per-joint description vectors. In the original URMA critic, the description vector mainly affects attention keys. For design optimization, that is too indirect: the gradient with respect to embodiment parameters needs to be strong and useful.  The authors therefore introduce a direct-design URMA critic. The value encoder receives both the per-joint observation (o_j) and the per-joint description (d_j), written as (g_\\psi(o_j,d_j)). This makes masses, gains, limits, geometry, and other design parameters enter the value path more directly. They also use multiple value heads and optimize the mean prediction, reducing sensitivity to single-head approximation errors.  That architecture change matters because VGDS depends on gradients through the critic. If the critic can predict returns but gives weak or noisy design gradients, the downstream optimizer has little to work with.  Experiments  The experiments use a velocity-tracking locomotion task in MuJoCo XLA through RL-X. The largest training set contains 50 robots: 15 quadrupeds, 31 bipeds and humanoids, and 4 hexapods. The design spaces are high-dimensional, reaching over 1100 continuous embodiment parameters.  The paper evaluates three progressively harder settings.  First, in single-robot design, the policy and critic are trained on randomized versions of one robot, then used to improve perturbed designs of that same robot. The main robots are Unitree Go2 with 358 design parameters, MIT Humanoid with 514, and Golem with 688. VGDS significantly improves the sampled initial designs and often approaches or exceeds the nominal reference design.  Second, in cross-training-distribution generalization, the target robot may be held out from its morphology-class training set, or included in a larger all-robot training set. VGDS remains the strongest search method overall. Training on all 50 robots helps Go2 and MIT Humanoid, while Golem benefits more from the hexapod-specific training set, likely because the all-robot set is dominated by quadrupeds and bipeds.  Third, in comparison to RL-based co-design, VGDS matches or slightly exceeds adapted RL co-design baselines while having a much lower marginal cost. The initial multi-embodiment training run takes around 7-9 hours, but each additional downstream design takes only about 1-2 minutes of search. In contrast, RL co-design baselines scale roughly linearly with the number of initial designs because each one needs a separate training run.  What the Gradients Reveal  One especially practical contribution is design analysis. Because VGDS optimizes a design vector, the same gradients can indicate which parameter groups limit performance.  For MIT Humanoid, the largest changes involve nominal joint positions and gains, along with reduced foot size. Copying only the optimized gains into the initial design already improves return from 5.8 to 12.5 in the paper’s analysis. For Golem, VGDS reduces the action scale, lowers the P gain, and increases the D gain. For Unitree Go2, the changes are more physical and local, including rear-leg joint-axis changes, foot geometry changes, and actuator velocity-limit changes on the front hip and calf.  This makes the method interesting beyond automatic optimization. It can serve as an engineering diagnostic: the critic does not just propose a better robot, it points to which body parts, actuator limits, gains, or geometry parameters are worth inspecting.  Strengths and Limitations  The strength of Shape Your Body is that it separates expensive learning from cheap design search. Once a broad multi-embodiment policy and critic exist, the same value function can optimize many new designs through gradients. The method scales to realistic legged-robot models and hundreds to thousands of continuous parameters, where pure black-box search is difficult.  The limitation is that the design topology is fixed. VGDS tunes continuous parameters while leaving joints, body parts, and structural graph edges unchanged. It also depends on critic coverage: if the search leaves the distribution where the critic learned reliable predictions, the value gradients can become misleading. The trust region helps, though it also assumes that a reasonable reference design is available. Finally, all results are in relatively simple MuJoCo simulations, so manufacturability, hardware transfer, ..."
    },{
      "title": "Mana: Dexterous Manipulation of Articulated Tools",
      "url": "/posts/2026/06/mana-articulated-tools-paper-notes/",
      "date": "2026-06-29",
      "tags": ["Robot Learning","Dexterous Manipulation","Articulated Tools","Sim-to-Real","Diffusion Policy"],
      "excerpt": "",
      "text": "Mana studies a hard corner of dexterous manipulation: small articulated tools such as tongs, pliers, clothespins, and syringes. These objects require the hand to grasp thin moving parts, stabilize the tool body, and apply enough force to actuate an internal joint.  My read: the main contribution is a data-generation view of tool use. Mana treats manipulation like animation. A few human affordance clicks define functional regions; procedural keyframes sketch the important poses; motion planning fills geometric reaching; short-horizon RL fills contact-rich inbetweening; a point-cloud diffusion policy then learns from the generated simulation trajectories and transfers zero-shot to the real robot.  Paper Info  The paper is “Mana: Dexterous Manipulation of Articulated Tools” by Zhao-Heng Yin, Guanya Shi, Pieter Abbeel, and C. Karen Liu from UC Berkeley, CMU, Stanford University, and Amazon FAR. It is available as arXiv:2606.13677, with project page zhaohengyin.github.io/mana.  Why Articulated Tools Are Hard  Rigid object manipulation already has difficult contact dynamics, but articulated tools add another layer: the robot must control the tool’s internal degree of freedom while keeping the whole tool stable. A tong must open and close; a plier must squeeze; a syringe requires plunger motion; a clothespin needs forceful actuation around a spring-loaded joint.  The paper emphasizes that successful force directions are often misaligned with stable surface normals. Pressing along the normal may reduce slip, yet it can rotate or eject the tool. Pressing in the desired actuation direction may generate the correct moment, yet it may also leave the friction cone. Small contact-location errors change moment arms and can destabilize the entire coupled hand-tool system.  This also explains why common data sources are weak here. Position-based teleoperation retargets fingertip poses, so it cannot reliably command the precise contact forces needed for centimeter-scale handles. End-to-end RL from scratch faces a sparse exploration problem: it must discover functional contacts, maintain them as the tool moves, and generate large task-specific forces in a high-dimensional hand action space.  Mana as Manipulation Animation  Mana reframes articulated tool use as a coarse-to-fine motion synthesis problem. The input is a tool mesh, an articulated joint model, and a small set of human-provided functional affordance annotations. The user marks regions such as handles, arms, plungers, or barrels in a 3D interface; the paper reports that this takes less than one minute per tool instance.  The output is a dataset of successful simulated manipulation trajectories. Each trajectory contains wrist motion, hand joint commands, object state, and semantic phase labels such as grasping, opening, and closing. These trajectories become training data for the real-world visuomotor policy.  The pipeline has three main layers.  Grasp generator. Mana builds on Lightning Grasp and adds a collision-aware IK procedure, called Lightning Grasp+, for thin tabletop objects. It uses annotated functional regions to construct contact domains, optimizes fingertip contact locations and finger joints, filters grasps in IsaacLab, and samples dense keyframes across relevant tool configurations.  Trajectory generator. Mana decomposes the episode into pre-grasping, grasping, and in-hand actuation. Pre-grasping uses GPU-accelerated RRT-Connect because it is mostly geometric. Grasping can be procedural when contact normals and squeezing motions are reliable, and it can use RL for thin or unstable cases. In-hand actuation uses RL because it needs coupled position-force control.  Visuomotor policy. After simulation data is generated, Mana trains a point-cloud-conditioned diffusion policy. The policy observes the segmented tool point cloud in the wrist frame plus proprioception, then outputs delta 6D wrist poses and delta hand joint position targets.  The RL Inbetweening Objective  The RL part is used for the phases where geometry alone is insufficient. Each episode starts from a generated stable grasp or pre-grasp keyframe and targets a meaningful tool configuration, such as moving from open to closed.  The dense reward has the form:  [r = r_{\\mathrm{tool}} + w_1 r_{\\mathrm{hand}} + w_2 r_{\\mathrm{contact}}.]  The tool term tracks the target tool joint, position, and orientation. The hand term regularizes the hand gesture, and in grasping also tracks wrist pose. The contact term counts active contacts between finger links and tool links:  [r_{\\mathrm{contact}}=\\sum_{i\\in\\mathrm{Finger},j\\in\\mathrm{Tool}} \\mathbf{1}[f_{ij}>\\epsilon].]  The appendix makes the physical story clearer. The RL teachers are trained with strong force-related randomization: action noise, robot PD gain randomization, tool PD gain randomization, tool mass randomization, friction randomization, and object force perturbations. The goal is to prevent policies from depending on a single brittle force balance.  This matters because tool use is force sensitive. A policy that looks good under one simulated stiffness, friction, or mass can fail immediately when the real tool pushes back differently.  Real Robot Policy and Hardware  The real platform is a 7-DoF xArm7 with a 16-DoF Allegro hand. Mana also uses custom flattened compliant fingertips because standard hemispherical rigid fingertips create unstable point contacts on thin handles. The soft silicone layer enlarges the contact patch and helps tolerate small pose errors during high-force actuation.  For perception, the system uses one RealSense D435 RGB-D camera. During deployment, SAM 3 segments the image, Fast Foundation Stereo produces an object point cloud, and the point cloud is represented in the wrist frame. The system runs at about 10 Hz on a workstation with two RTX 4090 GPUs.  The student policy uses 512 point-cloud points. A Perceiver-style transformer compresses the point cloud into four 128-dimensional tokens. Proprioception, including hand joint positions and targets from the past two frames, is encoded into another token. A lightweight transformer diffusion head then predicts the action. The policy is trained with a standard denoising objective:  [L=\\mathbb{E}_{(o_i,a_i)\\sim D,\\;t\\sim U[0,T]}\\left[\\lVert a_i-\\pi(\\tilde{a}_i,o_i,t)\\rVert^2\\right].]  Point-cloud randomization, including noise and random part masking, is used to bridge imperfect segmentation and depth noise in real deployment.  Results  Mana evaluates four articulated object categories: tongs, pliers, clothespins, and syringes, with two instances per category. The tools are thin, about 0.8-1.5 cm thick, and require around 3-7 N of actuation force.  The main table reports roughly 70% success for both grasping and in-hand manipulation across categories, with each cell evaluated over 10 trials per instance. Teleoperation performs poorly, often near zero on clothespins and syringes, and only around 30% on some tong phases. Open-loop execution of Mana-generated trajectories is better than teleoperation but still sensitive to mesh and pose errors. The closed-loop learned policy is consistently strongest.  The ablations support the data-generation thesis. Real-world success improves with more generated trajectories, more grasp keyframes, and stronger force randomization. That is an important result: for articulated tool use, robustness scales with dense state coverage around functional contacts and with physical diversity in simulation.  The paper also composes learned skills into functional tasks, using manual wrist teleoperation only for fine alignment to task sites. It reports 7/10 success for tong picking, 5/10 for plier cutting, 6/10 for clothespin use, and 5/10 for syringe injection. These numbers are lower than phase-level results because tool-object interaction introduces unseen perturbations and additional slip modes.  Strengths and Limitations  The strength of Mana is its decomposition. It avoids asking one method to solve everything. Human clicks provide functional intent. Procedural grasp generation gives the system contact candidates that RL would struggle to discover. Motion planning handles collision-free approach. RL focuses on short contact-rich segments. Diffusion policy learning turns the generated trajectories into a perception-conditioned controller.  The limitations are also concrete. The Allegro hand cannot handle stiff tools requiring more than about 10 N because of motor torque limits. The work focuses on precision grasps and does not explore human-like power grasps, partly because the Allegro hand is around twice the size of a human hand. Perception under occlusion and slip detection remain difficult, especially for very small tools. Finally, the demonstrated tool-use workflows still use wrist teleoperation for fine alignment, so full autonomous skill chaining needs another policy layer.  Takeaway  Mana is a useful reminder that dexterous manipulation data does not ..."
    },{
      "title": "FTP-1: A Generalist Foundation Tactile Policy Across Tactile Sensors for Contact-Rich Manipulation",
      "url": "/posts/2026/06/ftp-1-foundation-tactile-policy-paper-notes/",
      "date": "2026-06-29",
      "tags": ["Robot Learning","Tactile Sensing","Dexterous Manipulation","Generalist Policy","Vision Language Action"],
      "excerpt": "",
      "text": "FTP-1 is a foundation-policy attempt for tactile manipulation. The paper asks whether tactile feedback can be pretrained at scale across many sensor types and then reused for downstream contact-rich manipulation, including sensor setups that were never seen during pretraining.  My read: the important idea is not simply “add touch to a VLA.” FTP-1 is about making heterogeneous tactile sensors look like a shared robot-policy interface. It defines a morphology-aware tactile token space, uses sensor-specific encoders to map raw tactile signals into that space, and trains a separate tactile Transformer expert so tactile knowledge can be reused across sensors without disturbing the vision-language backbone.  Paper Info  The paper is “FTP-1: A Generalist Foundation Tactile Policy Across Tactile Sensors for Contact-Rich Manipulation” by Chengbo Yuan, Zicheng Zhang, Mingjie Zhou, Wendi Chen, Yi Wang, Zhuoyang Liu, Dantong Niu, Shuo Wang, Hui Zhang, Wenkang Zhang, Yingdong Hu, Yuanqing Gong, Wanli Xing, Chuan Wen, Cewu Lu, Kaifeng Zhang, and Yang Gao. It is available as arXiv:2606.13102, with project page ftp1-policy.github.io.  The Problem  Vision-based generalist robot policies can aggregate large heterogeneous datasets because camera observations are relatively easy to standardize. Tactile sensing is different. Across hardware, tactile data may be images, arrays, force-torque states, glove readings, fingertip signals, wrist forces, or hand-specific contact fields. Sensors vary in resolution, placement, morphology, physical response, and data format.  This makes tactile policy learning fragmented. A tactile policy trained for one sensor or hand often has little direct path to another sensor or embodiment. FTP-1 tries to build the tactile analogue of a foundation policy: one pretrained model that absorbs many tactile experiences and becomes a useful initialization for new contact-rich tasks.  Policy Interface  FTP-1 predicts action chunks from language, vision, proprioception, and tactile observations:  [\\hat{A}{t:t+H-1}=\\pi\\theta(\\ell, I_t, s_t, X_t).]  Here (\\ell) is the language instruction, (I_t) is multi-view RGB, (s_t) is proprioception, and (X_t) is tactile input. The action lives in a predefined Unified Action Space (UAS). Following UniDex-style alignment, UAS represents different robots with fixed sparse action slots for left arm, right arm, head pose, and supplementary controls. Dexterous hand joints are mapped through function-aligned slots, so different hands can share action semantics where their functions match.  That action-space design matters because tactile pretraining is only useful if both input and output heterogeneity are handled. FTP-1 pairs a unified tactile input interface with a unified robot-action interface.  Morphology-Aware Tactile Token Space  The core design is Morphology-Aware Tactile Token Space (MTTS). MTTS defines 24 functional-area slots. Slots 0-14 represent in-hand functional regions, slots 15-20 represent wrist and finger force/torque signals, and slots 21-23 are reserved. For a parallel gripper, the two side sensors are mapped to the thumb-tip and index-fingertip slots because those are their functional equivalents.  Each tactile input is grouped by functional area and converted into one token per area. FTP-1 adds a learnable functional-area embedding to each token, shared across sensors, with separate left/right hand embeddings where needed. This is the paper’s key abstraction: a token carries both the sensor reading and the functional contact region of the hand or robot.  This is also what makes cross-sensor transfer plausible. A GelSight fingertip image, a tactile array, or a force state can all contribute to a similar functional-area slot if they correspond to a similar contact role.  Heterogeneous Tactile Encoders  MTTS gives the common target space, but raw tactile observations still have different modalities. FTP-1 uses different encoders depending on the input type.  For image-type tactile inputs, such as GelSight-style signals, the model uses a lightweight sensor-specific ViT followed by a shared pretrained T3 Transformer tactile encoder. The final CLS token becomes the tactile token.  For array-type inputs, the model applies Fourier encoding on the signal dimension, then uses a CNN plus MLP to compress the functional area into one token.  For state-type inputs, such as force-torque vectors, the model applies Fourier encoding and uses an MLP.  All tactile tokens are LayerNorm-normalized, augmented with functional-area embeddings, and projected into the tactile expert dimension. The result is a set of unified tactile tokens with sensor-specific front ends and shared downstream semantics.  Separate Tactile Expert  FTP-1 is built on the (\\pi0.5) VLA codebase. It keeps a pretrained vision-language expert and a flow-matching action expert, then adds a separate 300M-parameter tactile Transformer expert. The action expert attends to tactile-expert outputs, while the tactile expert does not attend back into the action expert.  This modularity is important. Prior tactile-augmented VLA methods often inject tactile tokens into the vision-language expert through adapters. FTP-1 argues that this can interfere with pretrained vision-language knowledge and may not provide a reusable tactile module. A separate tactile expert lets the model reuse tactile knowledge across sensors while preserving the vision-language backbone.  The paper also injects proprioception through adaptive RMSNorm instead of a plain proprioceptive token, which improved generalization in their preliminary experiments.  Pretraining Dataset  The FTP-1 dataset aggregates 26 data sources covering 21 tactile sensors and roughly 3,000 hours of tactile manipulation data. The sensors include 7 image-type, 5 array-type, and 9 state-type tactile or force sensors.  The data mix spans human-hand demonstrations, dexterous-hand robot data, gripper robot data, and UMI-style data. After resampling, the mixture is approximately 20% human hand, 30% dexterous hand, and 50% gripper data. The authors also collect Sharpa North-FTP-1, with 4,000 long-horizon dexterous demonstrations.  During pretraining, the vision encoder, tokenizer, vision-language expert, and action expert are initialized from (\\pi0.5). The tactile encoder, tactile expert, adaptive RMSNorm proprioception injector, and action projector are trained from scratch. Pretraining uses 48 NVIDIA H20 GPUs for 50k steps with global batch size 768.  Evaluation  FTP-1 is evaluated by distributing checkpoints to independent institutions for downstream finetuning across five hardware setups. The seen-sensor setups include UniVTAC in simulation with GelSight-Mini, Sharpa North with Sharpa DTC, and Sharpa&Dexmate with Sharpa DTC. The unseen-sensor setups are FlexivXense with Xense image tactile sensors and TactileUMI with Contactile array sensors.  The baselines separate three questions.      (\\pi0.5): strong vision-language-action policy without tactile input.    Tactile-VLA: tactile tokens injected into the VLM expert without a separate tactile expert.    FTP-(\\pi0.5): FTP-1 architecture initialized from (\\pi0.5), without large-scale FTP-1 tactile pretraining.    On the UniVTAC simulation benchmark, FTP-1 reaches 66.66% average success, and 59.5% when the two lift tasks are excluded. This is about +17.5 points over the next strongest result under both averages.  On seen real tactile sensors, FTP-1 averages 62.5% across Sharpa North and Sharpa&Dexmate tasks, compared with 45.3% for (\\pi0.5), 35.8% for Tactile-VLA, and 41.6% for FTP-(\\pi0.5). This is a useful caution: adding tactile input through a weak fusion design can hurt compared with a strong vision-only VLA.  On unseen tactile sensors, FTP-1 is strongest again. It reaches 46.6% average success across Insert Hanoi, Insert USB, and Wipe Board, compared with 15.0% for (\\pi0.5), 8.3% for Tactile-VLA, and 15.0% for FTP-(\\pi0.5). The paper reports this as a +31.6 point gain over the architecture without FTP-1 tactile pretraining.  Does the Gain Come from Tactile Pretraining?  The paper includes a useful ablation called NTP-1. It pretrains on the same data distribution but removes tactile inputs and tactile-specific architecture during pretraining. During finetuning, the tactile architecture is added back. This tests whether the gains come from just seeing similar robot-task data, or from tactile-branch pretraining itself.  On UniVTAC, NTP-1 improves over FTP-(\\pi0.5), suggesting that data distribution does help. But FTP-1 remains clearly better. On FlexivXense, FTP-1 outperforms NTP-1 by +37.5 points, supporting the stronger claim that pretrained tactile knowledge transfers to unseen tactile-sensor setups.  Strengths and Limitations  The strength of FTP-1 is its interface design. MTTS turns sensor-specific touch into morphology-aware functional tokens. Heterogeneous encoders respect sensor modality differences. The tactile expert gives to..."
    },{
      "title": "ENPIRE: Agentic Robot Policy Self-Improvement in the Real World",
      "url": "/posts/2026/06/enpire-paper-notes/",
      "date": "2026-06-29",
      "tags": ["Robot Learning","Coding Agents","Real-World RL","Autonomous Research","Dexterous Manipulation","Paper Notes"],
      "excerpt": "",
      "text": "ENPIRE is a harness for letting coding agents improve real robot policies through a closed physical feedback loop. The loop is deliberately simple: reset the scene, run the policy, verify the outcome, inspect logs, change the code or training recipe, then try again. The paper’s main claim is that this loop can be made concrete enough for frontier coding agents to operate on real manipulation hardware, reaching high success on tasks such as Push-T, pin insertion, GPU insertion, and zip-tie cutting.  My read: the important move is the separation between building the environment interface and running autonomous policy improvement. The authors are not asking a language model to directly “do robotics” from raw pixels and motors. They first construct a reusable physical API around safety, reset, rollout, and reward. Once that interface is stable, the coding agent can work like an experimental scientist with a robot budget: propose a hypothesis, edit code, launch rollouts, read results, and keep the variants that improve success.  Paper Info  The paper is “ENPIRE: Agentic Robot Policy Self-Improvement in the Real World” by Wenli Xiao, Jia Xie, Tonghe Zhang, Haotian Lin, Letian “Max” Fu, Haoru Xue, Jalen Lu, Yi Yang, Cunxi Dai, Zi Wang, Jimmy Wu, Guanzhi Wang, S. Shankar Sastry, Ken Goldberg, Linxi “Jim” Fan, Yuke Zhu, and Guanya Shi. The affiliations are NVIDIA, CMU, and UC Berkeley.  The paper is available as arXiv:2606.19980. The project page is research.nvidia.com/labs/gear/enpire.  The Problem: Physical Autoresearch  Coding agents have become useful in digital environments because each trial is cheap: run code, see the error, patch, repeat. Real-world robot learning has a different bottleneck. Every trial consumes robot time, hardware can be damaged, the scene must be reset, and success must be measured reliably. Human supervision often remains hidden inside the “learning” pipeline through data collection, resetting, evaluation, reward engineering, and algorithm tuning.  ENPIRE frames the target as physical autoresearch: an agent should be able to run the research loop on hardware under an explicit robot-time budget. The missing abstraction extends beyond a better policy class to a repeatable experimental interface that turns robot interaction into structured feedback.  The paper decomposes that interface into four modules:                    Module        Role                            EN: Environment        hard safety constraints, automatic reset, automatic verification                    PI: Policy Improvement        agent-driven code edits, training recipes, algorithm variants                    R: Rollout        execution on one or more physical robots, logs, videos, rewards                    E: Evolution        multi-agent hypothesis selection, Git-based sharing, branch merging              This naming is useful because it makes the dependency clear. Policy improvement only becomes autonomous after the Environment and Rollout modules provide reliable feedback.  Stage 1: Build the Environment From Human Feedback  The first stage is human-guided. A person provides the task objective and feedback while the coding agent constructs the environment APIs. This stage is a one-time setup cost for a task; after it is validated, the API becomes an immutable interface reused during autonomous improvement.  There are three pieces.  Hard safety constraints restrict the robot’s configuration space and kinematic behavior. Violating the safe region immediately terminates the episode and triggers reset. This is both a physical safety layer and a clean episode boundary for learning.  Automated verification converts sensor streams into reward or success signals. Given a few minutes of success and failure demonstrations, the agent synthesizes procedural reward code using videos and proprioception, then optimizes for prediction accuracy and low latency. In pin insertion, the reward combines visual alignment, end-effector height, and force estimates. In zip-tie insertion, the reward uses cropped segmentation from two camera views to detect whether the strap passes through the head, with latency reduced below 150 ms.  Automated reset restores the scene after success or failure. For contact-rich tasks, ENPIRE uses modular manipulation tools inspired by CaP-X to reset directly to the critical phase, such as hovering above a pin hole, seating a GPU near a slot, or positioning scissors before cutting. This matters because policy learning should spend robot time on the hard part of the task, not on repeatedly reproducing easy setup motions.  The appendix gives the more concrete tool stack: SAM3 for open-vocabulary segmentation, BundleSDF for 6-DoF pose tracking, cuRobo for collision-free planning, RGB-D and proprioceptive state, and gripper torque as a tactile-like signal for slip and force control.  Stage 2: Let the Agent Improve the Policy  After the environment is stable, the second stage becomes autonomous. The agent receives the task objective, a writable training codebase, access to rollout logs and videos, and the goal of maximizing real-world success rate. It can review literature, formulate hypotheses, edit behavior cloning or RL code, tune infrastructure, and launch real rollouts through the environment API.  The pin insertion task is the paper’s clearest example. The robot must insert a pin into a hole with a tight 4 mm clearance, and agents are asked to reach 50 consecutive successes. During the run, agents try behavior cloning, iterative BC with online data aggregation, online RL, offline RL, offline-to-online RL, and BC-regularized RL. They also tune practical variables such as batch size, actor-critic update rate, and the BC regularization weight.  The mechanism is closer to an automated lab notebook than to a single learned controller. ENPIRE gives the coding agent enough observability to ask: did the last idea improve success, what failures remain, which branch found a better recipe, and which code changes should be merged?  Fleet Scaling: Evolution Through Git  ENPIRE scales by assigning one agent to one robot station. The paper uses a fleet of eight bimanual YAM robot stations. Each station has its own arms, cameras, compute, and coding agent. Hardware-control requests go through a local FastAPI server, while coordination across stations happens through Git: agents push branches, pull peer branches, cherry-pick useful commits, and merge promising training recipes.  This is the Evolution part of ENPIRE. The fleet does not require a central optimizer that collects all state. It uses a decentralized version-control protocol where ideas compete through measured success rates. In the pin-insertion idea tree, a few changes account for most of the improvement, including BC regularization at +10.8 percentage points, online RL mixed with demonstrations at +3.8 pp, batch-size tuning at +0.9 pp, and controller compensation at +1.3 pp.  This framing is nice because it treats code changes as experimental hypotheses. A branch is not just software state; it is a record of a policy-improvement idea tested on hardware.  Experiments and Results  The real-world tasks cover several forms of dexterity:                    Task        Core challenge                            Push-T        non-prehensile pushing to align a T-shaped block                    Pin insertion        plug pins into 4 mm holes                    GPU insertion        seat GPU chips into thin motherboard sockets                    Zip-tie cutting        grasp scissors and cut the zip-tie tail              Success is measured as completing the task in one rollout with up to eight retries, where later retries can react to earlier failures. This is stricter in a useful way: it measures precision together with recovery under real-world uncertainty.  Several results stand out.  First, heuristic learning works in simulation but is much harder on real hardware. In Gym-PushT, Claude Code and Codex reach 95% success in about two hours, while Kimi Code takes roughly twice as long. In the real Push-T setup, two of the three agents fail, showing that real contact, friction, dynamics, and scene variation are the real test.  Second, ENPIRE can support gradient-based real-world policy improvement. For pin insertion, agents use BC, online data aggregation, and RL variants to hill-climb toward the 50-success objective. The paper reports policy convergence to 100% in pin insertion, faster than a frontier human-in-the-loop method.  Third, fleet size reduces wall-clock time. In Push-T, scaling from one to eight agent-robot pairs reduces the time to reach a 1.0 normalized score from roughly five hours to two hours. In pin insertion, scaling from one to eight agents reduces time to near-perfect success from more than 1.5 hours to about 40 minutes.  Fourth, ENPIRE can discover hybrid strategies that combine code-based policies and VLAs. In RoboCasa365, the agent improves over GR00T and zero-shot CaP..."
    },{
      "title": "End-to-End Dexterous Arm-Hand VLA Policies via Shared Autonomy",
      "url": "/posts/2026/06/shared-autonomy-dexgrasp-vla-paper-notes/",
      "date": "2026-06-26",
      "tags": ["Vision-Language-Action","Dexterous Manipulation","Tactile Sensing","Teleoperation"],
      "excerpt": "",
      "text": "This paper is about a very practical bottleneck in dexterous VLA work: high-quality arm-hand demonstrations are hard to collect because full-DoF teleoperation overloads the operator. The proposed answer is Shared Autonomy. A human controls the arm’s coarse end-effector motion through VR, while a tactile-aware DexGrasp-VLA policy controls the dexterous hand as a Copilot. The collected trajectories then fine-tune an end-to-end arm-hand VLA policy.  My read: the interesting part is the division of labor. The human supplies scene understanding, target choice, approach strategy, and pre-grasp positioning. The autonomous hand policy supplies contact-rich grasp closure, tactile adaptation, and force modulation. This converts full arm-hand teleoperation into a semi-autonomous data engine, then uses Arm-Hand Feature Enhancement and Corrective Teleoperation to turn that data into a stronger policy.  Paper Info  The paper is “End-to-End Dexterous Arm-Hand VLA Policies via Shared Autonomy: VR Teleoperation Augmented by Autonomous Hand VLA Policy for Efficient Data Collection” by Yu Cui, Yujian Zhang, Lina Tao, Yang Li, Xinyu Yi, and Zhibin Li from ByteDance Seed. The arXiv version is 2511.00139, and the project page is dexvla-seed.github.io/dex-vla.  Problem  Dexterous arm-hand manipulation has two coupled difficulties. The arm handles macro motion: reaching, avoiding obstacles, positioning the hand, and aligning pre-grasp poses. The hand handles micro motion: compliant multi-finger closure, contact adaptation, force control, and slip recovery. A monolithic teleoperation setup asks the operator to control both at once. The paper reports that even 20-30 minutes of continuous full-DoF teleoperation can be tiring, especially for untrained operators.  Automated planning has the opposite failure mode. It can create many state-action pairs, but its data distribution reflects its solvers and constraints. The resulting motions may be stiff, physically valid, and diverse while still missing the small human strategies that make dexterous manipulation work.  The paper’s formulation is a compromise:      human operator: VR-guided 6-DoF arm end-effector motion;    DexGrasp-VLA Copilot: autonomous 12-DoF dexterous hand control;    final policy: end-to-end arm-hand VLA trained from synchronized shared-autonomy data.    Both the hand-only policy and the final arm-hand policy are fine-tuned from (\\pi_0) using LeRobot.  DexGrasp-VLA As Hand Copilot  The paper uses VLA policies in two places. The first is DexGrasp-VLA, a hand-only tactile-aware VLA policy. Its job is to control the fingers during data collection, so the operator can focus on moving the arm.  DexGrasp-VLA is bootstrapped in two stages. First, the authors train a vision-free LSTM grasping policy from proprioception and tactile feedback. The dataset combines 68 parameterized force-adaptive control demonstrations and 150 human teleoperation demonstrations. The force-adaptive controller follows:  [q_c(i)=q_m(i)+q(0)e^{-k f_z(i)}.]  When contact is weak, the command closes the hand quickly. As fingertip normal force increases, the closure slows and tightens into a stable grasp. The LSTM sees a temporal window of proprioception and tactile readings:  [x_t=[s^{hand}_t,f^{hand}_t]\\in\\mathbb{R}^{39},\\quads_t=[q^{hand}_t,\\tau^{hand}_t]\\in\\mathbb{R}^{24},\\quadf^{hand}_t\\in\\mathbb{R}^{15}.]  It predicts a 12-DoF hand action with behavior cloning and MSE loss. This creates a compact reactive expert for tactile force adaptation.  Second, the LSTM expert collects data for a multimodal hand VLA policy. DexGrasp-VLA consumes:  [o^{hand}_t=[I^{hand}_t,l_t,q^{hand}_t,z^{tac-f}_t,z^{tac-s}_t].]  Here (I^{hand}t) is the eye-in-hand image, (l_t) is the language instruction, (q^{hand}_t) is the hand state, (z^{tac-f}_t) is per-fingertip resultant force, and (z^{tac-s}_t) is a spatial tactile embedding. The raw tactile tensor is (F{raw}\\in\\mathbb{R}^{10\\times12\\times3}) per fingertip. The paper compresses it two ways:      resultant force vectors (f^{tac-f}_t\\in\\mathbb{R}^{5\\times3}), giving explicit net force magnitude and direction;    convolutional autoencoder tactile latents (f^{tac-s}_t\\in\\mathbb{R}^{5\\times128}), preserving spatial contact patterns.    This combination is important because force magnitude alone cannot describe where contact sits on the fingertip. Spatial tactile latents help the policy detect slippage, off-center contact, and shifts in contact distribution.  Shared Autonomy Data Collection  Shared autonomy combines VR arm teleoperation with autonomous hand control. The VR system uses relative motion mapping: once the user engages the controller, the initial VR controller pose (T_{VR,0}) and robot end-effector pose (T_{robot,0}) define the mapping:  [T_{robot,t}=T_{robot,0}(T_{VR,0}^{-1}T_{VR,t}).]  The target pose is converted to joint commands through a velocity-level QP IK solver using PlaCo and Pinocchio. The paper reports OpenXR support at 90 fps, less than 100 ms latency, and synchronized data saving at 30 Hz.  During collection, the human drives the arm while DexGrasp-VLA produces hand actions:  [D^{uni}={(o^{uni}t,a^{arm}_t,a^{hand}_t)}{t=1}^{T},\\quada^{arm}t\\sim p{teleop},\\quada^{hand}t\\sim\\pi{hand}(\\cdot|o^{hand}_t).]  The combined observation contains multi-view RGB, language, arm joint states, and hand joint states. The result is a synchronized arm-hand dataset where the arm motion has human spatial intent and the hand motion has tactile-adaptive execution.  In the appendix, shared autonomy collects 110 trajectories/hour/person for the main dataset, compared with 90 trajectories/hour/person for full teleoperation. Corrective collection is 100/hour/person versus 80/hour/person. The authors frame this as roughly a 25% collection-rate gain, with lower operator fatigue and a faster policy iteration loop.  End-to-End Arm-Hand VLA  The final policy (\\pi_{uni}) controls the full arm-hand system end to end:  [\\pi_{uni}(A^{uni}_t|o^{uni}_t).]  The key architectural addition is Arm-Hand Feature Enhancement. The base (\\pi_0) model produces a shared task representation (z^{share}_t) from visual, language, and proprioceptive inputs. The enhancement module passes this shared representation through two dedicated MLPs:  [z^{arm}t=E{arm}(z^{share}t),\\quadz^{hand}_t=E{hand}(z^{share}_t).]  Auxiliary heads predict arm and hand sub-actions from the limb-specific features. The main action head uses the fused representation:  [z^{fused}_t=[z^{share}_t,z^{arm}_t,z^{hand}_t].]  The loss combines the main flow-matching action loss with auxiliary arm and hand losses:  [L_{total}=L_{main}+\\lambda(L_{hand}+L_{arm}).]  The motivation is straightforward. Arm motion is smooth, spatial, and longer horizon. Hand motion is contact-rich, local, and force-sensitive. A single undifferentiated latent representation can mix these demands poorly. The extra branches encourage the model to preserve shared task context while learning limb-specific dynamics.  Corrective Teleoperation  The paper also builds a corrective human-in-the-loop system. During deployment, successful autonomous rollouts are recorded. When the policy fails, the operator intervenes through the same shared autonomy interface, recovers the task, and creates a corrective trajectory. At iteration (k):  [D^{(k)}=D^{(k)}{success}\\cup D^{(k)}{corrective}.]  The next policy is fine-tuned from (\\pi_0) using the original shared-autonomy dataset plus the curated deployment data:  [\\pi^{(k+1)}_{uni}=SFT(\\pi_0;D^{uni}\\cup D^{(k)}).]  The important design choice is targeted correction. Human effort is concentrated on failure states: bad orientations, corner positions, unseen object shapes, and adversarial configurations. This creates a small but high-value data flywheel.  Experiments  The hardware is a UR3e arm with a 12-DoF five-finger Xhand, each fingertip carrying 120 tri-axial force channels, plus two fixed RealSense D435i cameras and one wrist-mounted RealSense D405. The paper also validates the Arm-Hand Feature Enhancement module on an RY-H2 hand. Policies are evaluated at 30 Hz.  Datasets are deliberately small:      LSTM pretraining: 150 human teleoperation demonstrations plus 68 force-adaptive autonomous trajectories.    DexGrasp-VLA hand policy: 180 successful grasping trajectories in clutter, covering 60 objects.    End-to-end arm-hand policy: 100 shared-autonomy demonstrations over 20 common household objects.    Corrective datasets: 50 orientation-recovery trajectories and 50 corner-case trajectories.    The hand-only DexGrasp-VLA policy is evaluated in cluttered tabletop scenes with more than 50 objects. It reaches 95.5% overall success in a handheld setup, where the human moves the mounted hand and the policy decides finger-level grasping.  The final end-to-end arm-hand VLA is tested on a pick-and-place task across 50 objects. It reaches 88.7% average success, with 91.7% on seen objects and 85.6% on unseen objects.  Ablations  The tactile ablation i..."
    },{
      "title": "LUCID: Learning Embodiment-Agnostic Intent Models from Unstructured Human Videos for Scalable Dexterous Robot Skill Acquisition",
      "url": "/posts/2026/06/lucid-paper-notes/",
      "date": "2026-06-26",
      "tags": ["Robot Learning","Dexterous Manipulation","Human Video","Sim-to-Real"],
      "excerpt": "",
      "text": "LUCID is a clean split between two scalable sources of supervision. Internet-scale human video gives task intent: what should change in the scene. Massively parallel simulation gives robot control: how a specific embodiment should realize that intent. The phrase that prompted this note, “Next, control: a generalist sensorimotor policy,” refers to the second half of the system, but the full paper is about the interface between the two halves.  My read: LUCID is valuable because it avoids treating human video as hidden robot action. Human video is used to predict short-horizon intent, represented as object flow plus a rough palm pose. A separate embodiment-specific sensorimotor policy, trained in simulation, converts that intent into low-level robot actions. This division lets the same intent model drive a dexterous LEAP hand and a parallel-jaw gripper, while the controllers remain specialized to their robot bodies.  Paper Info  The paper is “LUCID: Learning Embodiment-Agnostic Intent Models from Unstructured Human Videos for Scalable Dexterous Robot Skill Acquisition” by Harsh Gupta, Guanya Shi, and Wenzhen Yuan from the University of Illinois Urbana-Champaign and Carnegie Mellon University. It is available as arXiv:2606.11628, with project page lucid-robot.github.io.  The Problem  Most robot learning pipelines still need robot demonstrations, teleoperation, or structured human data. These sources are expensive and tied to a particular robot, camera setup, hand, or interface. Unstructured human videos are much easier to scale, yet they lack robot actions. Simulation can generate action-labeled robot data at scale, while task-level rewards are hard to write for open-ended dexterous manipulation.  LUCID pairs these two imperfect sources. Human video supervises an intent model (f_\\theta). Simulation trains an embodiment-specific sensorimotor policy (\\pi). The shared interface is a short-horizon reference (R), made of object flow and palm pose:  [f_\\theta\\left(I_t,{x^{trk}{n,0}}{n=1}^{N},(p^{palm}0,R^{palm}_0)\\right)= R= \\left{{x^{trk}{n,\\tau}}{n=1,\\tau=1}^{N,T},{(p^{palm}\\tau,R^{palm}\\tau)}{\\tau=1}^{T}\\right}.]  In words: from recent RGB-D observations, current object query points, and current palm pose, the intent model predicts where object points should move and where the palm should roughly go over the next short horizon.  Intent Model  The intent model adapts CoTracker3 into a forward predictor. Standard trackers estimate where points went in already observed frames. LUCID changes that direction: it predicts future object point motion from the current observation.  The model input is:      a stack of recent RGB-D frames (I_t);    (N=16) current query points on the target object;    the current palm pose.    The output is:      (T=8) future steps over a 1-second horizon;    3D object flow for the query points;    a palm-pose trajectory.    The architecture uses frozen DINOv3 ViT-B/16 patch tokens for RGB, a residual depth adapter, and a CoTracker-style EfficientUpdateFormer over object tokens plus a palm token. The loss is a sum of mean-squared errors over object flow, palm position, and palm rotation:  [L(\\theta)=L_{trk}+L_{palm,p}+L_{palm,r}.]  The supervision pipeline is heavily engineered. For each task, the authors mine 20k clips from datasets such as Panda-70M, Action100M, Something-Something-V2, EPIC-Kitchens, and LVP metadata, mostly in-the-wild YouTube footage. For underrepresented tasks, they record about 1 hour of smartphone demonstrations. Each clip is processed with ViPE for camera and depth, SAM 3.1 for object and human masks, DenseTrack3Dv2 for 3D object tracks, and WiLoR plus a rigid fit for MANO palm pose. Human-region augmentation is strong, so the model learns object motion without depending on human hand appearance.  Generalist Sensorimotor Policy  The sensorimotor policy is the “control” part. It is trained in Isaac Lab with goal-conditioned RL to realize any reference (R) in the same format as the intent model output. The policy is embodiment-specific, so the LEAP hand and the parallel-jaw gripper get separate controllers, but both consume the same intent interface.  Training references are procedural, not generated by the intent model. Each simulated episode samples a procedurally generated object from a pool of blob-like, tool-like, and plate-like shapes, then samples a reference trajectory with four segments:      approach;    in-hand motion;    goal;    disengage.    This is why the policy is generalist. It sees many object shapes, many object-flow trajectories, and many palm-pose references. At deployment, when the learned intent model emits a new (R), the policy treats it like another goal to track.  The action space is structured:  [a_t = [a^{arm}_t; a^{eig}_t; a^{hnd}_t].]  Here (a^{arm}_t\\in\\mathbb{R}^6) controls arm joint-position deltas, (a^{eig}_t\\in\\mathbb{R}^5) controls an eigen-grasp basis, and (a^{hnd}_t\\in\\mathbb{R}^{16}) adds per-joint hand residuals. The eigen-grasp basis is the top-5 PCA basis from retargeted human grasps. It biases exploration toward coordinated grasp modes, while residual hand joints keep fine control available.  The policy is trained in two stages. A privileged teacher (\\pi^T) learns with PPO using full simulator state, full-surface object query points, object pose, contact information, and proprioception. A student (\\pi^S) is distilled from the teacher with a hybrid PPO plus behavior-cloning loss:  [L_{student}=L_{PPO}(\\pi^S)+\\lambda_D \\kappa \\mathbb{E}\\left[|\\mu_S(o^S)-\\mu_T(o^T)|_2^2\\right].]  The student sees deployable observations: camera-visible object query points, palm-pose reference, wrist-mounted depth, and proprioception. Domain randomization covers object scale, mass, friction, robot actuator parameters, reset jitter, observation noise, gravity curriculum, wrench perturbations, and success tolerances. The authors also do real-to-sim actuator calibration with CMA-ES against joint step responses.  Closed-Loop Deployment  Deployment runs a slow intent cycle around a fast control loop. The sensorimotor policy runs at 50 Hz. DenseTrack3Dv2 updates sliding-window point tracking at 30 Hz. SAM 3.1 refreshes masks at 1 Hz, and the intent model also runs at 1 Hz. The full stack runs on a single RTX 5090 GPU.  At each intent cycle, the system refreshes the object mask, samples and back-projects query points, estimates current palm pose from forward kinematics, and asks (f_\\theta) for a fresh short-horizon reference (R). Between intent cycles, the point tracker keeps current points aligned with the live scene, and the student policy consumes the advancing lookahead window from (R).  This closed-loop property is the real difference from video-plan methods. If a grasp misses or the object shifts, LUCID can re-query the live scene and produce a new reference. The control policy then follows the updated reference with the same sensorimotor machinery.  Experiments  The paper evaluates five real-world tasks:      Stirring: pick up a spoon and complete three stirring circles.    Wiping: pick up a cloth and clear marks or substances from a surface.    Binning: pick up each object and place it into a target container.    Push-T: push a T-shaped block to a target pose.    Cable routing: thread a cable through fixtures.    The first three tasks are supervised only by web-scraped video, with 20k clips per task. Push-T and cable routing use 1 hour of self-collected smartphone video each.  On the web-supervised tasks, closed-loop LUCID reaches 73% average success, compared with 28% for an open-loop video-generation baseline that uses Veo 3.1 to produce a single video plan from the initial scene. The difference is mostly about recovery. Open-loop plans become stale when execution diverges. LUCID can keep asking the intent model what should happen next.  The per-task breakdown is also useful:      Stirring: LUCID succeeds 19/30, open-loop succeeds 7/30.    Wiping: LUCID succeeds 26/30, open-loop succeeds 13/30.    Binning: LUCID succeeds 21/30, open-loop succeeds 5/30.    For embodiment transfer, the same intent model is used with two controllers: the LEAP hand policy and a parallel-jaw gripper policy. On Push-T and cable routing, both embodiments reach 19/30 aggregate success. Cable routing favors the gripper because its two opposing jaws are well-suited to the cable; the hand has a harder time with precise cable grasping. The key result is that the intent model transfers while the controller changes.  Scaling and Ablations  The binning data-scaling sweep uses 1k, 2k, 5k, 10k, and 20k clips. Held-out intent loss decreases steadily, and real-world binning success rises with more clips. At 1k-2k clips, the model often reaches the object but fails to localize the bin. Around 5k-10k clips, container localization appears, but release alignment remains weak. This gives a practical signal: internet video quantity helps, but task-level precision still needs enough coverage.  The senso..."
    },{
      "title": "HY-Embodied-0.5: Embodied Foundation Models for Real-World Agents",
      "url": "/posts/2026/06/hy-embodied-0-5-paper-notes/",
      "date": "2026-06-26",
      "tags": ["Embodied Intelligence","Vision-Language Models","Vision-Language-Action","Robot Learning","Spatial Reasoning","Paper Notes"],
      "excerpt": "",
      "text": "TL;DR  HY-Embodied-0.5 is Tencent Robotics X and HY Vision Team’s attempt to build an embodied VLM foundation model before attaching a robot action head. The paper’s main claim is that real-world agents need more than general image-language understanding: they need fine-grained spatial perception, temporal/trajectory reasoning, affordance grounding, and planning-oriented reasoning. The system therefore combines a modality-adaptive Mixture-of-Transformers (MoT) architecture, visual latent tokens, embodied/spatial data construction, iterative RL + rejection-sampling post-training, and large-to-small on-policy distillation.  My read: this paper is best treated as a backbone paper for embodied agents, not a complete robot policy paper. The strongest technical story is the path from embodied VLM to robot-ready VLA: first build a compact MoT-2B model that can reason about spatial and embodied tasks, then reuse it as the perception/reasoning backbone for downstream real-robot control.  Paper Info  The paper is “HY-Embodied-0.5: Embodied Foundation Models for Real-World Agents” by Tencent Robotics X and HY Vision Team, including Xumin Yu, Zuyan Liu, Ziyi Wang, He Zhang, Yongming Rao, Fangfu Liu, Yani Zhang, Ruowen Zhao, Oran Wang, Yves Liang, Haitao Lin, Minghui Wang, Yubo Dong, Kevin Cheng, Bolin Ni, Rui Huang, Han Hu, Zhengyou Zhang, Linus, and Shunyu Yao. It appears on arXiv as arXiv:2604.07430, submitted April 8, 2026. Code and models are released at Tencent-Hunyuan/HY-Embodied, with the MoT-2B weights linked from the project repository.  The model family has two main variants:      HY-Embodied-0.5 MoT-2B: an edge-oriented model with about 2B activated / 4B total parameters.    HY-Embodied-0.5 MoE-A32B: a larger reasoning model with about 32B activated / 407B total parameters.    Core Problem  General VLMs are good at describing images, answering broad visual questions, and using internet-scale semantics. Embodied agents need a different mix of abilities. A robot policy or planning system has to know where objects are, how far they are, which part is graspable, what trajectory is plausible, what state will come next, and how an instruction maps into a physical sequence.  HY-Embodied-0.5 frames this as a foundation-model gap. Action outputs are only the final interface. The backbone itself needs stronger fine-grained visual perception, 3D/spatial grounding, and embodied reasoning for prediction, interaction, and planning. This is why the paper spends most of its effort on VLM architecture, embodied/spatial data, and post-training, before showing a downstream VLA experiment.  Architecture  The compact model keeps the usual VLM shape: a visual encoder produces visual tokens, and an LLM processes visual and text tokens together. The difference is that HY-Embodied-0.5 changes how visual tokens are handled after the encoder.  The visual encoder is HY-ViT 2.0, a native-resolution ViT with about 400M parameters. It supports arbitrary-resolution inputs and is distilled from a stronger internal ViT. The paper also trains a larger ViT that turns each 8 x 8 image patch into a discrete code from a 2K codebook; those codes are later used as supervision for the visual branch.  The central architectural choice is Mixture-of-Transformers (MoT). Instead of forcing visual and text tokens through fully shared transformer parameters, the model duplicates the QKV and FFN parameters for a vision branch while keeping the language branch separate. Visual tokens use the vision-specific parameters; text tokens use the original language parameters.  This design targets a practical problem: heavy visual training can improve perception while damaging language capability, especially in small models. MoT gives the model more visual capacity without turning the compact model into a slow dense model. The paper reports that MoT converges faster than a dense transformer baseline and adds negligible inference overhead because decoding dominates runtime.  The attention pattern is also modality-specific. Visual tokens use more bidirectional/full attention because image patches do not have the left-to-right causal structure of language. Text tokens retain causal attention. This makes the vision branch more like a visual modeling module while preserving the LLM-style generation path.  Visual Latent Tokens  HY-Embodied-0.5 appends a learnable visual latent token after each image or video frame. The paper treats this token as a bridge between visual full attention and language causal attention. During pre-training, the latent token is supervised to match a global feature from the teacher ViT, so it is encouraged to carry image-level semantics instead of staying at local patch information.  The training objective combines three losses during large-scale pre-training:  \\[L_{\\mathrm{total}} = L_{\\mathrm{llm}} + L_{\\mathrm{vision}} + L_{\\mathrm{global}}\\]  The visual branch predicts the teacher ViT’s next discrete visual code:  \\[L_{\\mathrm{vision}} =-\\frac{1}{N_v}\\sum_{i=1}^{N_v}\\log p_i(z_i)\\]  The latent token is aligned to the teacher ViT global feature using negative cosine similarity:  \\[L_{\\mathrm{global}} =-\\frac{f_{\\mathrm{latent}}^\\top f_{\\mathrm{teacher}}}{|f_{\\mathrm{latent}}||f_{\\mathrm{teacher}}|}\\]  After pre-training and mid-training, the extra visual/global objectives are removed; later stages use standard autoregressive language loss. The design is useful because the model gets visual supervision when it is learning representations, then returns to a clean generation objective for reasoning and instruction following.  Data Recipe  The data construction is the other major contribution. The paper does not rely on generic image-text pairs alone. It mixes visual perception, embodied-centric, spatial-centric, and general understanding data.  The large-scale pre-training stage uses more than 600B tokens: 389B general understanding tokens and 236B embodied/perception tokens. Inside the embodied/perception portion, spatial and robotics data account for 43%, with the rest coming from visual perception data. The mid-training stage then uses about 30M higher-quality instances mixed at a 12:5:3 ratio across general understanding, embodied, and spatial data.  The visual perception data includes:      62M Omni-Detection samples for 2D/3D detection and object grounding.    36M depth-estimation samples for absolute and relative depth.    5M segmentation samples based on filtered high-quality masks.    11M pointing/counting samples to reduce enumeration errors and spatial hallucinations.    The embodied-centric data is organized into grounding, affordance, trajectory, understanding, planning, and reasoning. This structure matters. It turns “embodied data” from a vague label into a curriculum: locate the relevant thing, infer what can be done to it, predict or evaluate a path, understand task state, then plan the next actions.  The spatial-centric data targets correspondence, geometry, configuration, measurement, and dynamics. This is the part that makes the model more than a robot benchmark specialist. It trains capabilities like 2D-3D correspondence, metric object size estimation, relative direction, room area estimation, camera motion, and object movement.  Post-Training  The post-training pipeline is built around long-chain embodied reasoning. It starts with about 100K cold-start CoT instances, constructed through a human-model collaborative pipeline and filtered for reasoning quality, correctness, and repetition.  For RL, the paper avoids a fixed training set. Each RL round samples from a large candidate pool using the current model. Examples that are always solved are too easy; examples that always fail are too hard. The retained samples are those with partial success, which sit near the current capability frontier. Each RL stage uses 50K newly selected samples, balanced across perception, prediction, interaction, and planning.  The reward design is task-aware. Grounding uses IoU, Hungarian matching, point distance, or Chamfer distance. Trajectory tasks use DTW or Frechet-style path similarity. Counting and multiple-choice tasks use exact or partial matching. Continuous estimates get regression-style rewards. Free-form reasoning falls back to an LLM judge.  The RL objective is GRPO-style. For each input, the model samples a group of G = 16 responses, normalizes rewards inside the group, and updates with a clipped policy-ratio objective. The group-relative form matters because embodied tasks have different reward scales; a point-localization score and a planning score should not be treated as directly comparable raw numbers.  The paper then alternates RL with rejection-sampling fine-tuning. RL expands the frontier; rejection sampling selects successful high-quality reasoning traces and turns them into supervised data. In practice, around 1M candidate examples are filtered into about 300K high-quality traces for the next SFT stage.  On-Policy Distillation  The large MoE..."
    },{
      "title": "GRAIL: Generating Humanoid Loco-Manipulation from 3D Assets and Video Priors",
      "url": "/posts/2026/06/grail-paper-notes/",
      "date": "2026-06-26",
      "tags": ["Humanoid Robotics","Loco-Manipulation","Synthetic Data","Sim-to-Real","Video Foundation Models","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  GRAIL is NVIDIA’s fully digital data-generation pipeline for humanoid loco-manipulation. It starts from 3D assets and simulator-ready scenes, uses video foundation models as interaction priors, reconstructs metric 4D human-object interaction trajectories, retargets them to a Unitree G1, and trains task-general policies for object pick-up, whole-body manipulation, sitting, and terrain traversal. The main claim is practical: humanoid robots can get useful loco-manipulation supervision before any physical robot teleoperation or physical scene rebuild.  My read: the paper is about moving the data bottleneck upstream. Instead of collecting robot demonstrations for every object and terrain, GRAIL builds known 3D configurations first, asks a video model to propose plausible human-object interactions inside that configuration, then recovers and retargets those interactions into robot-compatible references. The important distinction is that GRAIL does not try to reconstruct arbitrary internet videos. It gives the reconstruction system known geometry, metric scale, camera parameters, object assets, environment depth, and a character already proportioned for the target humanoid.  Paper and Resources  The paper is “GRAIL: Generating Humanoid Loco-Manipulation from 3D Assets and Video Priors” by Tianyi Xie, Haotian Zhang, Jinhyung Park, Zi Wang, Bowen Wen, Jiefeng Li, Xueting Li, Qingwei Ben, Haoyang Weng, Yufei Ye, David Minor, Tingwu Wang, Chenfanfu Jiang, Sanja Fidler, Jan Kautz, Linxi Fan, Yuke Zhu, Zhengyi Luo, Umar Iqbal, and Ye Yuan from NVIDIA and UCLA. It is available as arXiv:2606.05160, with project materials at research.nvidia.com/labs/dair/grail, code at NVlabs/GRAIL, and a released Hugging Face dataset at nvidia/PhysicalAI-Robotics-Locomanipulation-GRAIL.  The generated data covers more than 20,000 sequences across object pick-up, whole-body manipulation, sitting, and terrain traversal. Using only GRAIL-generated data, the authors train egocentric visual policies and deploy them on a real Unitree G1, reporting 84% real-world pick-up success and 90% stair-climbing success.  The Problem: Humanoid Data Does Not Scale Cleanly  Humanoid loco-manipulation couples locomotion, balance, object contact, hand actions, and scene geometry. A pick-up trajectory is more than an arm reaching motion: the robot must approach, place feet, bend or squat, establish contact, grasp, lift, and recover balance. Terrain traversal has a similar whole-body structure, where stairs, slopes, curbs, and chairs impose geometry-specific constraints.  Teleoperation and motion capture provide high-quality demonstrations, but their scaling properties are poor. Each new object, scene, or terrain layout may need physical setup, human operation, instrumentation, and robot runtime. In-the-wild videos are abundant, but converting them into robot-ready trajectories is underconstrained: camera, metric scale, object geometry, human morphology, contact, and world-space motion all need to be inferred after the fact.  GRAIL changes the order of operations. It first specifies the 3D world, then asks a video model for behavior inside that world. This gives the system privileged information during reconstruction and turns several hard inference problems into known inputs.  Pipeline Overview  Given a 3D object asset, GRAIL produces three forms of supervision:      humanoid kinematic motion;    object kinematic motion;    robot actions generated by task-general tracking policies.    The pipeline has four stages:      Robot-centric human video generation: build a known 3D scene, render the first frame, and use a video foundation model to synthesize a human-object interaction video.    Interaction-aware 4D HOI reconstruction: estimate human motion and object pose, then jointly optimize them using keypoints, projection, depth, contact, and temporal regularization.    Task-general loco-manipulation tracking: retarget the reconstructed motion to Unitree G1 and train policies that can track whole families of generated references.    Sim-to-real visual policy training: distill the tracking policies into egocentric RGB policies and deploy on real hardware.    The architecture is built around a specific tradeoff: use generative video models for behavioral diversity, but keep metric geometry and robot compatibility anchored in simulation assets.  Stage 1: Asset-Conditioned Video Generation  GRAIL does not directly generate robot videos. The authors argue that current video foundation models have stronger priors over human motion and manipulation than over robot bodies, and the human motion reconstruction stack is more mature. The pipeline therefore uses a human character asset prefitted to the Unitree G1 morphology, making later retargeting less brittle.  The system constructs candidate scene configurations with Infinigen: an indoor floor-only environment and a furnished room with a table. A VLM decides whether an object should be placed on the floor or table according to affordance. The object is settled into a stable initial configuration, and Blender renders the initial frame with known camera intrinsics and extrinsics. A VLM writes the interaction prompt, and a video foundation model such as Kling generates a static-camera interaction video.  This “known before generation” setup matters. The camera parameters, metric scale, object geometry, object texture, environment depth, and character morphology are all available when reconstruction starts. Compared with unconstrained video mining, GRAIL avoids estimating the entire world from ambiguous pixels.  Stage 2: Interaction-Aware 4D HOI Reconstruction  The reconstruction stack first estimates human and object motion independently. GENMO provides per-frame SMPL-X body pose parameters from the generated video, with body shape fixed to the prefitted character. WiLoR refines hand poses, filling missing detections with interpolation and smoothing. FoundationPose tracks the 6-DoF object pose from the known first-frame object pose and known asset geometry; the paper fine-tunes it for RGB-only tracking by zeroing depth channels.  Independent estimates are not enough. They can produce floating contacts, penetrations, and depth-scale drift. GRAIL therefore performs joint optimization over residual human and object trajectory updates:  \\[L =\\lambda_{\\mathrm{kp}}L_{\\mathrm{kp}}      \\lambda_{\\mathrm{proj}}L_{\\mathrm{proj}}    \\lambda_{\\mathrm{depth}}L_{\\mathrm{depth}}    \\lambda_{\\mathrm{cont}}L_{\\mathrm{cont}}    \\lambda_{\\mathrm{reg}}L_{\\mathrm{reg}}.\\]    Each term has a specific role. Keypoint loss keeps projected human motion aligned with detected 2D body and hand keypoints. Object projection loss preserves image-space alignment with the tracked object pose. Depth loss uses MoGe-2 and SAM2 to generate metric human/object point clouds and align visible mesh vertices through Chamfer distance. Contact loss uses VLM-predicted contact labels to pull relevant hand/body regions and object regions together in depth. Regularization suppresses foot skating, velocity drift, and temporal jitter.  This objective is the mechanism that makes the generated video usable as robot data. The video model provides a plausible interaction prior; optimization turns it into a metric, temporally coherent, contact-aware 4D trajectory.  The paper also filters failures. Generated videos can have texture inconsistency, blurry motion, or geometry mismatch. GRAIL compares SAM2 object masks against rendered silhouettes from FoundationPose predictions and discards sequences whose mask tracking error exceeds a threshold. This is an important engineering detail: the pipeline is scalable because it can generate and filter, not because every VFM sample is reliable.  Stage 3: Retargeting and Task-General Tracking  The optimized SMPL-X motion is retargeted to Unitree G1 with GMR, while the object trajectory provides a reference object pose. GRAIL then trains tracking policies on top of SONIC, a pretrained whole-body controller. The point is to convert retargeted references into robot-action data without fitting a new controller per sequence.  The paper uses two complementary trackers:                    Tracker        Used for        Adaptation                            Object-aware adaptor        pick-up and whole-body manipulation        freezes SONIC and learns a latent residual plus hand open/close primitives                    Scene-aware tracker        stairs, curbs, slopes, sitting        fine-tunes SONIC with a height-map encoder for terrain-conditioned control              The object-aware adaptor observes proprioception and object references, including object pose in the robot body frame, hand-to-object transforms, finger contact forces, a BPS shape encoding, and future reference deltas. It outputs a latent residual \\(\\Delta z_t\\) and left/right hand primitives:  \\[(\\De..."
    },{
      "title": "GentleHumanoid: Learning Upper-body Compliance for Contact-rich Human and Object Interaction",
      "url": "/posts/2026/06/gentlehumanoid-compliance-paper-notes/",
      "date": "2026-06-26",
      "tags": ["Humanoid Robots","Compliance Control","Human-Robot Interaction","Reinforcement Learning","Whole-Body Control","Paper Notes"],
      "excerpt": "",
      "text": "TL;DR  GentleHumanoid is a whole-body humanoid control framework for safe upper-body contact. The paper’s main move is to make compliance part of the motion-tracking target itself: shoulder, elbow, and hand links are governed by a spring-damper reference dynamics model, then an RL policy learns to reproduce that compliant reference while still tracking human-like motion.  My read: this is best understood as impedance reference tracking for upper-body humanoid interaction. The policy still outputs joint position targets for low-level PD control, but the reward asks the robot to behave like a multi-link impedance system under simulated contact. That changes the training problem from rigid motion imitation to motion imitation with bounded, coordinated yielding.  Paper Info  The paper is “GentleHumanoid: Learning Upper-body Compliance for Contact-rich Human and Object Interaction” by Qingzhou Lu, Yao Feng, Baiyu Shi, Michael Piseno, Zhenan Bao, and C. Karen Liu from Stanford University. It appears on arXiv as arXiv:2511.04679, dated November 6, 2025. The project page is gentle-humanoid.axell.top, and the deployment code is available at Axellwppr/gentle-humanoid.  The showcased tasks are deliberately physical: handshaking with a 5 N force limit, sit-to-stand support, shape-aware hugging, and balloon handling. These are the right examples for the paper because they expose the weakness of stiff tracking controllers: they can follow a pose, but they do not know when to give way.  Core Problem  Recent humanoid RL has become strong at whole-body motion tracking. A Unitree G1 can imitate expressive upper-body motions, walk, recover, and execute long motion clips. But rigid tracking is uncomfortable and unsafe when a human is part of the contact loop. A hug should yield around the partner’s body, a handshake should move with the human hand, and sit-to-stand assistance should provide firm support without spiking contact force.  The key difficulty is that upper-body contact is not localized to one end effector. In hugging or support, the hand, elbow, shoulder, torso, and arm chain can all participate. A controller that only adapts the wrist or base misses the coupled nature of the interaction. GentleHumanoid therefore models compliance over multiple upper-body links instead of treating external forces as disturbances to suppress.  Reference Dynamics  GentleHumanoid starts from a simple second-order model for each upper-body link:  \\[M\\ddot{x}i = f{\\mathrm{drive},i} + f_{\\mathrm{interact},i}\\]  Here \\(x_i\\) is the 3D Cartesian position of a link such as a shoulder, elbow, or hand in the robot root frame. \\(M\\) is a scalar virtual mass, set to 0.1 kg in the reference model. The total behavior comes from two forces.  The driving force pulls the link toward the target motion:  \\[f_{\\mathrm{drive}} =K_p(x_{\\mathrm{tar}} - x_{\\mathrm{cur}})      K_d(v_{\\mathrm{tar}} - v_{\\mathrm{cur}})\\]    This is classical impedance control: the target motion acts like a virtual spring-damper. The damping is set near critical damping, \\(K_d = 2\\sqrt{M K_p}\\), so the reference does not oscillate unnecessarily.  The interaction force represents contact with humans or objects:  \\[f_{\\mathrm{interact}} =K_{\\mathrm{spring}}(x_{\\mathrm{anchor}} - x_{\\mathrm{cur}})\\]  The clever part is how the anchor is chosen. GentleHumanoid uses one unified spring formulation for two contact cases:      Resistive contact: when the robot presses against a surface, the anchor is fixed at the initial contact point. The spring produces a restoring force.    Guiding contact: when an external agent pushes or pulls the robot, the anchor is sampled from plausible upper-body postures in human motion data. The spring guides the robot toward a new coordinated pose.    This matters because the sampled anchor is not an independent random offset for each link. It comes from complete human upper-body postures, so the shoulder, elbow, and hand forces remain kinematically consistent. That is the paper’s main answer to multi-link compliance: generate simulated contact that already respects the body chain.  Force Thresholding  Unbounded impedance tracking can still be unsafe. If the target motion and current link position differ a lot, the virtual spring can demand a large force. GentleHumanoid caps the driving force with a task-adjustable threshold:  \\[f_{\\mathrm{drive}}^{\\mathrm{limited}} =\\min\\left(1.0, \\frac{\\tau_{\\mathrm{safe}}}{|f_{\\mathrm{drive}}|}\\right)f_{\\mathrm{drive}}\\]  The policy observes \\(\\tau_{\\mathrm{safe}}\\), so compliance becomes tunable at deployment. Lower thresholds produce softer interactions for hugging and handshaking; higher thresholds allow firmer support for sit-to-stand assistance.  The paper trains with force limits sampled between 5 N and 15 N. It explicitly relates this range to ISO/TS 15066 pain-onset limits and comfort studies. For realistic hugging contact areas, the resulting pressure range is reported as roughly 3-9 kPa, in the same band as soft human hugs and below common comfort thresholds.  Policy Training  The RL policy is trained to track the impedance reference dynamics, not merely the original motion clip. The reference positions and velocities are integrated with semi-implicit Euler:  \\[\\dot{x}{t+1}^{\\mathrm{ref}} =\\dot{x}{t}^{\\mathrm{ref}} +\\Delta t \\frac{f_{\\mathrm{drive}} + f_{\\mathrm{interact}}}{M}\\]  \\[x_{t+1}^{\\mathrm{ref}} =x_t^{\\mathrm{ref}} + \\Delta t \\dot{x}_{t+1}^{\\mathrm{ref}}\\]  The policy uses a teacher-student PPO setup for sim-to-real. The student receives deployable observations: the force-safety threshold, target motion information, root angular velocity, projected gravity, joint-position history, and recent action history. The teacher additionally sees privileged simulation/reference signals, including reference link states, predicted and actual interaction forces, link heights, previous torques, and cumulative tracking error.  Both policies output 29-dimensional joint position targets at 50 Hz, tracked by low-level joint PD controllers. Training motions come from GMR-retargeted AMASS, InterX, and LAFAN, filtered down to about 25 hours of interaction-relevant motion at 50 Hz.  The compliance reward has three parts:      Reference dynamics tracking: match simulated link positions and velocities to the impedance reference.    Reference force tracking: align simulated contact forces with the force predicted by the reference dynamics.    Unsafe force penalty: penalize interaction force above \\(\\tau_{\\mathrm{safe}} + \\delta_{\\mathrm{tol}}\\), with \\(\\delta_{\\mathrm{tol}} = 10\\,\\mathrm{N}\\).    This reward is the mechanism that turns the analytic impedance model into a deployable neural whole-body policy.  Experiments  The simulation comparison uses a hugging motion with external pulling force. GentleHumanoid keeps forces lower and more stable across the hand, elbow, and shoulder. At the hand, it stabilizes around 10 N, while Vanilla-RL exceeds 20 N and Extreme-RL exceeds 13 N. At elbow and shoulder links, the baselines saturate around 15-20 N, while GentleHumanoid stays closer to 7-10 N.  On the Unitree G1, the first real-world test applies external wrist force in a static pose. The baselines resist stiffly and move the torso instead of yielding at the arm. Extreme-RL requires a peak force of 51.14 N to reposition the arm, and Vanilla-RL requires 24.59 N. GentleHumanoid yields more smoothly and keeps the response near the specified force threshold across different postures.  The hugging evaluation uses customized waist-mounted pressure pads with 40 calibrated capacitive taxels. The robot hugs a mannequin under correct alignment and intentional misalignment. GentleHumanoid maintains moderate, bounded contact pressure, while baselines create localized pressure peaks and less predictable force profiles.  The object test uses balloon handling with the force threshold set to 5 N. GentleHumanoid holds the balloon without damaging it. The baselines apply excessive pressure, deform the balloon, and eventually lose balance and drop it.  The paper also shows a shape-aware hugging pipeline. A head-mounted RGB camera estimates a human body mesh, the system extracts waist targets, and an optimization step adapts the G1 upper-body hugging motion to the person’s body shape. This is more than a demo trick: it connects compliance control with perception, where the robot must choose both a comfortable target pose and a safe contact response.  Limitations  The main limitation is contact realism. The training force model uses simulated spring forces, which provide structure and coverage, but real human contact includes friction, tissue deformation, clothing, delayed human response, and complex distributed pressure. The paper reports occasional 1-3 N force overshoots on hardware from sim-to-real mismatch, suggesting that tactile feedback would be useful for tighter force regulation.  The method also depends on the diversity of human motion data. Because guidin..."
    },{
      "title": "DynamicVLA: A Vision-Language-Action Model for Dynamic Object Manipulation",
      "url": "/posts/2026/06/dynamicvla-paper-notes/",
      "date": "2026-06-26",
      "tags": ["Vision-Language-Action","Dynamic Manipulation","Robotics","Imitation Learning"],
      "excerpt": "",
      "text": "DynamicVLA studies a failure mode that is easy to hide in static manipulation: by the time a VLA finishes thinking, the object may have already moved. Static pick-and-place tolerates slow action chunking because the scene changes little during inference. Dynamic object manipulation changes the constraint. A rolling can, a moving ball, or a deflected object requires fast perception, temporal anticipation, and action execution that stays synchronized with the latest observation.  My read: the paper’s center of gravity is temporal control, not model scaling. DynamicVLA uses a compact 0.4B model, overlaps inference and execution with Continuous Inference, and uses Latent-aware Action Streaming to discard stale actions. The benchmark contribution, Dynamic Object Manipulation (DOM), is equally central because it creates the data and evaluation surface that static robot datasets lack.  Paper Info  The paper is “DynamicVLA: A Vision-Language-Action Model for Dynamic Object Manipulation” by Haozhe Xie, Beichen Wen, Jiarui Zheng, Zhaoxi Chen, Fangzhou Hong, Haiwen Diao, and Ziwei Liu from S-Lab, Nanyang Technological University. It is available as arXiv:2601.22153. The project page is haozhexie.com/project/dynamic-vla, and the official implementation is released at hzxie/DynamicVLA.  The Problem  Most VLA systems use action chunking: the model observes the scene, predicts a short action sequence, then the robot executes that sequence. This amortizes inference cost, but it creates two timing problems under moving objects.  First, there is a perception-execution gap. The model starts inference from observation (O_t), but the resulting action chunk becomes available only after latency (m). The first actions in the chunk were meant for the old state, while the world has already advanced to (O_{t+m}). Second, there is inter-chunk waiting. Many systems trigger the next inference only after finishing the current action chunk, which leaves pauses or stale behavior between chunks.  DynamicVLA frames dynamic manipulation as a synchronization problem:  [A_t = M(O_t, L_t, P_t), \\quad A_t = {a_t,\\ldots,a_{t+n}}]  Here (O_t) is a temporal window of visual observations, (L_t) is the language instruction, and (P_t) is proprioception. The latent object state (s_t) keeps evolving while the model reasons. The control system therefore needs a model that is fast enough and an execution policy that knows which actions are already outdated.  Model Design  DynamicVLA has a compact VLA backbone and a diffusion-style action expert. The language backbone is SmolLM2-360M, truncated to the first 16 transformer layers. For vision, the paper uses FastViT, a convolutional vision encoder that compresses spatial structure efficiently and avoids the token growth of transformer-based visual encoders under multi-frame inputs.  The model input includes:      visual observations from a sparse temporal window, defaulting to (O_t={o_{t-2},o_t});    language instruction tokens;    robot proprioception, projected as a state token.    The output is an action chunk:      horizon (n=20);    each action is a 32D vector representing end-effector pose and gripper state, with padding for unused dimensions.    The action expert is a conditional Flow Matching Transformer. Given VLM features (f_t), noisy action chunk (A_t^\\tau), and diffusion timestep (\\tau), it learns the denoising vector field:  [\\ell_\\tau(\\theta)=\\mathbb{E}{p(A_t|f_t),q(A_t^\\tau|A_t)}\\left[\\left|E\\theta(A_t^\\tau,O_t)-u(A_t^\\tau|A_t)\\right|\\right]]  This architecture is designed around latency. The model is small enough for high-frequency inference, while the action expert still produces temporally structured action chunks.  Continuous Inference  Continuous Inference changes when the next inference starts. In a serial chunking pipeline, the system predicts (A_t), executes the full chunk, then predicts the next chunk. That creates waiting at chunk boundaries.  DynamicVLA instead starts a new inference as soon as the previous inference finishes. While the robot executes actions from (A_t), the model already predicts (A_{t+m}). If the action horizon (n) is longer than inference latency (m), a fresh chunk arrives before the current chunk is exhausted. Execution no longer blocks on inference completion.  This is a simple systems idea, but it matters because dynamic manipulation is dominated by timing. A policy that waits cleanly in static scenes can lose the object in dynamic scenes.  Latent-Aware Action Streaming  Continuous Inference creates overlapping action chunks. At a given execution timestep, actions from the older chunk (A_t) and the newer chunk (A_{t+m}) may both exist. DynamicVLA resolves this with Latent-aware Action Streaming (LAAS).  LAAS applies two rules:      discard actions in (A_t) whose intended timesteps are earlier than (t+m), since they correspond to already outdated observations;    when old and new chunks overlap, prioritize the newer chunk (A_{t+m}).    This turns action chunking from a fixed commitment into a streaming control interface. The model still benefits from chunk prediction, but execution remains tied to the most recent observation available after inference delay.  DOM Benchmark  The paper also introduces Dynamic Object Manipulation (DOM), a benchmark and data pipeline for moving-object manipulation. It contains 200K synthetic episodes, 2.8K scenes, 206 objects, and 2K real-world episodes collected without teleoperation.  The data collection pipeline is important. In simulation, Isaac Sim provides object 6D pose and velocity. A state-machine controller produces four stages: approach object, grasp and lift, approach target and place, then reset. In the real world, teleoperation is too slow for fast moving objects, so the authors build a real-world “simulator”: synchronized third-person RGB-D cameras estimate object pose and velocity, then feed the same state-machine controller.  DOM evaluates three groups of abilities:      Interaction: closed-loop reactivity, dynamic adaptation, long-horizon sequencing.    Perception: visual understanding, spatial reasoning, motion perception.    Generalization: visual generalization, motion generalization, disturbance robustness.    This benchmark design is useful because it separates dynamic manipulation into the axes that usually get collapsed into a single success rate.  Results  In simulation, DynamicVLA is evaluated against Diffusion Policy, OpenVLA-OFT, (\\pi_0), (\\pi_{0.5}), SmolVLA, GR00T-N1.5, VLA-Adapter-Pro, and VLASH. Each method is fine-tuned on DOM. DynamicVLA reaches 47.06% average success across the nine DOM dimensions, while the strongest baseline in the table is 13.61%. It also completes tasks faster, with 8.53 s average time versus around 10 s for most baselines.  The interaction gains are especially large. In simulation, DynamicVLA reports 60.5% closed-loop reactivity, 38.5% dynamic adaptation, and 40.5% long-horizon sequencing. Real-world interaction tasks show the same pattern: on tasks such as placing a moving coffee can, conical bottle, pickleball, or collecting repeated balls, DynamicVLA is far more reliable than (\\pi_{0.5}), SmolVLA, and VLASH.  Perception remains harder. DynamicVLA improves visual understanding, spatial reasoning, and motion perception, but motion-conditioned targets are still difficult. The real-world perception section reports 51.9% average success for DynamicVLA, while the best baselines remain near 11.7%.  For generalization, DynamicVLA improves transfer to unseen appearances and unseen motion patterns. Disturbance robustness remains the weakest axis, which makes sense because random impacts and surface variation create state changes that are difficult to model from rigid-body pose streams alone.  Ablations  The ablations make the paper’s engineering choices clearer:      Full DynamicVLA reaches 47.06% success on DOM.    Removing both Continuous Inference and LAAS drops success to 30.27%.    Keeping Continuous Inference but removing LAAS gives 39.72%, showing that overlapping inference is useful but stale action handling still matters.    Replacing FastViT with a transformer-style visual encoder drops success to 28.89%.    A 135M backbone is too small at 26.67%, while a 1.7B backbone is too slow at 24.33%. The 360M setting gives the best latency-capacity balance.    The appendix adds a useful detail about temporal context. A single current frame reaches 38.22%, while the sparse pair ({o_{t-2},o_t}) reaches 47.06%. Adding more frames gives little extra benefit. The takeaway is precise: dynamic manipulation needs motion cues, but extra visual history can quickly become redundant if it raises compute without improving timing.  Strengths and Limitations  DynamicVLA is strong because it treats latency as part of the model design. The architecture, inference schedule, action streaming rule, and benchmark all target the same failure mode. The paper also avoids relying on human teleoperation for dynamic object da..."
    },{
      "title": "Do as I Do: Dexterous Manipulation Data from Everyday Human Videos",
      "url": "/posts/2026/06/do-as-i-do-paper-notes/",
      "date": "2026-06-26",
      "tags": ["Robot Learning","Dexterous Manipulation","Human Videos","Retargeting","Sim-to-Real"],
      "excerpt": "",
      "text": "Do as I Do is a complete video-to-robot-data pipeline for dexterous manipulation. It starts from ordinary monocular RGB videos of people manipulating objects, reconstructs 4D hand-object interaction, and retargets the recovered motion into executable trajectories for multi-fingered robot hands.  My read: the paper is strongest as a systems bridge between two worlds that usually stay separate. Vision foundation models recover a usable hand-object reference from messy human video. A dynamics-aware optimizer then turns that noisy reference into robot-complete manipulation data. The important shift is from treating human video as a vague prior to treating it as a source of physically replayable trajectories.  Paper Info  The paper is “Do as I Do: Dexterous Manipulation Data from Everyday Human Videos” by Bhawna Paliwal, Haritheja Etukuru, William Liang, Pieter Abbeel, Nur Muhammad “Mahi” Shafiullah, and Jitendra Malik from UC Berkeley. It is available as arXiv:2606.19333, with project page do-as-i-do.com and code at malik-group/do-as-i-do.  The Problem  Dexterous robot learning needs data, but the most scalable data is observational. Humans leave behind enormous amounts of manipulation video; robots need action-labeled, embodiment-specific trajectories. Teleoperation is slow and hardware-bound. Simulation can scale rollouts, but open-ended dexterous tasks are hard to specify with rewards and environments.  The paper frames this as the old “Do as I do” problem under modern constraints: reconstruct what the human did, then retarget it to a different body. The hard setting is everyday monocular RGB video. There is no depth, no motion capture, no known object mesh, no clean hand-object contact signal, and no robot action.  Method Overview  The pipeline has two stages.  First, reconstruction estimates the human hand, object shape, object pose, and camera-aligned hand-object trajectory from RGB video. The hand side uses HaWoR. The object side uses SAM 3D for single-frame mesh generation and a guided-diffusion tracker that keeps object shape fixed while updating pose over time.  Second, retargeting takes the reconstructed hand-object trajectory and solves for a robot trajectory in simulation. The target embodiment in the paper is a 22-DoF Sharpa Wave hand, deployed with UR3e arms for real-world bimanual rollouts. The retargeting step is dynamics-aware: it optimizes in physics simulation, so the result must track the reference while respecting contact, gravity, object motion, and hand-object interaction.  This two-stage split matters because the reconstruction is inevitably noisy. The retargeter is designed around that reality and handles references far messier than clean MoCap.  Object Tracking via Guided Diffusion  The most interesting reconstruction component is the SAM 3D-based object tracker. Applying SAM 3D independently to every frame gives inconsistent meshes and poses. Do as I Do instead anchors the object shape at one frame and uses flow-matching inference to sample pose updates conditioned on the current image and previous pose.  The paper writes the guided update as:  [x^s_t=(1-\\alpha_s)(x^s_{t-\\Delta}+\\Delta v^s_\\theta)+\\alpha_s z^s_{\\mathrm{ref}}(t),\\qquadx^p_t=(1-\\alpha_p)(x^p_{t-\\Delta}+\\Delta v^p_\\theta)+\\alpha_p z^p_{\\mathrm{ref}}(t).]  Here (x^s) is the shape block, (x^p) is the pose block, and the reference interpolants pull the diffusion sample toward the fixed canonical shape and previous-frame pose. Shape guidance can stay high because objects are rigid. Pose guidance is adaptive: the method estimates object rotational velocity from 2D point tracks, then lowers or raises pose guidance depending on how much motion the object appears to have.  Per frame, the tracker samples multiple candidate poses and clusters them under a weighted SE(3) distance. Candidate selection by clustering performs about as well as log-likelihood scoring while avoiding the expensive trace computation needed for exact flow-model likelihood.  After tracking, the system aligns hand and object scales. HaWoR hand reconstruction and SAM 3D object reconstruction live in different scale conventions, so the method uses MoGe pointmaps and hand/object centroids to slide the object along its camera ray until the visible object position is consistent with the hand.  Dynamics-Aware Retargeting  The retargeting stage begins with a kinematic hand retargeting reference, then runs MPPI-style sampling-based optimization in MuJoCo Warp. It plans every 0.5 seconds over a 3-second horizon, evaluates 1024 samples per planning step, and runs 32 optimization iterations. Rewards track object position/orientation, hand position/orientation, and finger joints, with penalties for excessive penetration.  The paper adds three pieces that make the optimizer robust to reconstructed references.  Warmup steps. A noisy first frame can put the robot hand and object in an unrecoverable state. The method prepends a warmup horizon where the object is temporarily held in place while the robot hand moves into a better configuration. Then the weld is released and normal simulation begins.  Random force perturbation. Some rollouts look good briefly but are dynamically fragile, such as balancing an object on fingertips. Random forces during sampled rollouts push the optimizer toward grasps that survive small disturbances.  Transition reward. Rest-to-in-hand and in-hand-to-rest transitions are discrete interaction events. Tracking loss alone is too soft when references are noisy, so the method adds a penalty when the object should be resting but lacks floor contact, or should be in-hand but lacks hand-object contact.  These additions are practical. They do not require hand-written grasp samplers or task-specific object heuristics; they use the same simulation optimizer and make it less brittle.  Results  On reconstruction, Do as I Do improves over existing hand-object reconstruction and object-tracking baselines. On DexYCB it reports 0.71 F-5, 0.93 F-10, and 0.66 Chamfer distance. On HOI4D it reports 0.72 F-5, 0.91 F-10, and 0.49 Chamfer distance. On a 150-video in-the-wild benchmark, human raters prefer its object tracking over FoundationPose 67% of the time, with 79% win rate among non-tie judgments.  On retargeting, the gap is large. On reconstructed in-the-wild references, the annealed-sampling baseline succeeds 25% of the time. Adding warmup raises success to 66%; adding perturbation reaches 67%; adding transition reward reaches 71%. On clean OakInk2 MoCap references, the same sequence improves from 72% to 81%. This is a good sign: the new components help noisy internet-video references, yet they also improve cleaner human-object trajectories.  For real-world deployment, the pipeline produces 500 high-quality, human-verified dexterous manipulation trajectories from internet videos, egocentric videos, and generated videos. The paper demonstrates real robot rollouts on 10 tasks including whisking, pouring, dusting, squeezing, tamping, erasing, stirring, hammering, spreading, and picking.  Data Filtering Playbook  One especially useful section is the data-quality analysis. From 2,000 ten-second 100DOH clips, only 187 contain meaningful hand-object interaction. After removing boundary failures, shot-boundary issues, camera-motion failures, SAM 3D failures, and other problems, only 83 clips remain suitable for reconstruction, about 4% of the sampled data.  The practical message is blunt: internet video is huge, but usable robot-learning video is sparse after quality filtering. Scaling from human video needs serious preprocessing, not just a bigger crawler. The paper estimates roughly a 20x penalty if one skips careful filtering.  Strengths and Limitations  The strength of Do as I Do is end-to-end completeness. It goes from an everyday RGB video to robot rollouts with dexterous hands, and the key modules are chosen for open-world coverage: SAM 3D for objects, HaWoR for hands, MoGe for metric geometry, and sampling-based simulation optimization for retargeting.  The limitation is also clear. The method assumes rigid objects and reasonably accurate monocular depth. Monocular video has contact ambiguity: visual occlusion can look like physical touch. The reconstruction covers the hand and one object, so it misses scene constraints such as tables, obstacles, articulated supports, and container geometry. The final rollouts also inherit simulator mismatch, which bounds real-world reliability.  Takeaway  Do as I Do is best read as a recipe for making human videos operational. The paper does not claim that human video directly gives robot actions. It builds a layered compiler: RGB video to 4D hand-object reference, reference to physically plausible robot trajectory, trajectory to real-world rollout.  For dexterous manipulation research, the key lesson is that video-scale data becomes useful only after representation and retargeting are designed together. Better object tracking alone is insuffi..."
    },{
      "title": "DexHoldem: Playing Texas Hold'em with Dexterous Embodied System",
      "url": "/posts/2026/06/dexholdem-paper-notes/",
      "date": "2026-06-26",
      "tags": ["Dexterous Manipulation","Embodied Agents","Robot Learning","Benchmark","ShadowHand","Vision-Language-Action","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  DexHoldem is a real-world benchmark for dexterous embodied systems built around Texas Hold’em tabletop interaction. The platform uses a ShadowHand mounted on a UR10e arm, three RealSense RGB-D cameras for policy-side observation, and a separate agent-view camera for system-level state parsing. The benchmark contains 1,470 teleoperated demonstrations across 14 card and chip manipulation primitives, a physical policy benchmark, a 36-problem agentic perception benchmark, and three released closed-loop system case studies.  My read: the paper is useful because it evaluates the full stack at the point where many embodied-agent demos become vague. A robot needs to decide that it should call, raise, reveal a card, or collect chips, then parse a changing tabletop state, translate the decision into executable low-level primitives, manipulate thin cards and small chips with a dexterous hand, preserve the table for the next decision, and recover when perception or execution is slightly wrong. DexHoldem makes those failure surfaces measurable.  Paper and Resources  The paper is “DexHoldem: Playing Texas Hold’em with Dexterous Embodied System” by Feng Chen, Tianzhe Chu, Li Sun, Pei Zhou, Zhuxiu Xu, Shenghua Gao, Yuexiang Zhai, Yanchao Yang, and Yi Ma. It is available as arXiv:2605.18727, with the project page at dexholdem.github.io/Dexholdem. The dataset is released as TexasPokerRobot, the policy code is in DexHoldem/Dexholdem-Policy, and the embodied-agent runtime is in DexHoldem/DexHoldemSKills.  The policy repository is organized around six public training recipes: DP(DINO), DP_transformer_resnet, DP_unet, ACT, RDT_small, and RDT_FT. The released pipeline downloads the Hugging Face dataset, organizes demonstrations with five validation trajectories per primitive, optionally precomputes DinoV2 or SigLIP visual features, and serves trained checkpoints through a ZeroMQ policy server. The skills repository exposes the agent loop as a coding-agent-native workflow: install the skill, launch a game loop, parse the table, route the state, and dispatch robot primitives.  Why Poker Is a Good Benchmark Shape  Texas Hold’em is not used here as a test of poker strategy. It is a controlled tabletop domain where semantic state and physical state are tightly coupled. Cards are thin, chips are small and denomination-specific, bets change the legal action space, and every physical movement can disturb later decisions. The robot may need to pick up a left or right hole card, show a card, place it back face down, push a 50-chip, pull a 100-chip, or compose several such actions into a higher-level poker move.  This makes the benchmark more demanding than isolated hand skills. A local manipulation success can still be a system failure if a non-target card shifts, a chip stack becomes unreadable, or the next agent step cannot trust the table state. DexHoldem therefore scores primitive rollouts with four outcome levels: scene-preserving success, disruptive completion, retryable task failure, and disruptive failure requiring reset. The distinction between task completion and scene preservation is the main metric design choice.  Benchmark Structure  DexHoldem separates the system into two levels. The primitive level defines callable dexterous skills for data collection, policy training, and physical rollouts. The agent level defines the perception, routing, verification, recovery, and human-help branches that arise when those primitives are composed into a poker hand.  The policy benchmark covers 14 instruction-conditioned primitives:                    Group        Primitives                            Card pickup        pick_up_left, pick_up_right                    Chip push        push_5, push_10, push_50, push_100                    Chip pull        pull_5, pull_10, pull_50, pull_100                    Card placement/reveal        put_down_left, put_down_right, show_left, show_right              Each primitive has 105 accepted teleoperated demonstrations, split into 100 training and 5 validation trajectories. During deployment, each low-level policy receives top-down, third-person, and wrist RGB-D observations, 30-dimensional arm-hand proprioception, and a task condition. The action is a short-horizon sequence of 30-dimensional joint-position targets: 6 dimensions for the UR10e arm and 24 for the ShadowHand.  The agentic perception benchmark isolates the visual parsing problem. Each of 36 problems asks the perceiver to recover a structured game state from a real tabletop image plus predecessor context. The schema is split into eight scored challenges: loop stage, turn ownership, blind information, community cards, current bet chips, robot chip inventory, opponent chip inventory, and showdown outcome. The important detail is that overall correctness is a strict exact match over all applicable fields, so a single wrong chip dictionary can make a state unusable for routing.  The Closed-Loop Agent  The embodied loop is capture -> perceive -> route -> execute. A dedicated agent-view camera captures the table. The perceiver writes structured state. A deterministic router loads persistent game-state memory, validates fields, handles waiting, verifies primitive outcomes, retries harmless failures, and escalates unsafe states to human help. The main agent is queried only when multiple high-level branches are legal, such as a new poker decision at an idle state.  High-level agent primitives are translated into low-level policy primitives. For example, view_card(L) becomes pick_up_left -> perceive -> put_down_left, while call dispatches chip-push primitives according to the difference between the opponent’s bet and the robot’s bet. Chip actions use a min-count rule over denominations in the order 100, 50, 10, 5, so a failed atom can be retried without discarding the whole high-level action.  This architecture is deliberately mundane in the right places. The router carries workflow constraints that should not be left to the language model at every frame, while the learned policy handles contact-rich motion. The result is a benchmark where an agent’s perception errors, policy errors, verification delays, and recovery decisions all show up in the same physical trace.  Main Results  On the 80-trial primitive schedule covering all 14 skills, π0.5 obtains the best task completion rate at 61.2%. On the stricter scene-preserving success rate, π0.5 and π0 tie at 47.5%. RDT reaches 30.0% SPSR and 46.2% TCR; DP(DINO) is the strongest task-specific imitation baseline at 26.2% SPSR and 36.2% TCR. Smaller or simpler baselines remain much lower: ACT reaches 10.0% SPSR, BAKU 6.2%, and DP-UNet 1.2%.  The gap between SPSR and TCR is the key signal. For π0.5, the score rises from 47.5% to 61.2% when disruptive completions are counted. For RDT, it rises from 30.0% to 46.2%. These are not small bookkeeping differences; they show that a policy can complete a local primitive while damaging the table state needed by the next agent step.  The primitive-group breakdown is also revealing. π-series models reach 100.0/100.0 SPSR/TCR on pickup, but chip motion remains much harder: π0.5 reaches 25.0/35.0 on chip push and 15.0/30.0 on chip pull. Put-down/show has a larger completion-preservation gap, reaching 50.0/80.0 for both π0.5 and π0. Cards can be placed or revealed while still disturbing the scene enough to hurt continuation.  For agentic perception, Opus 4.7 gives the best strict problem-level exact match at 34.3%, while GPT 5.5 gives the best average field-wise accuracy at 66.8%. This split is important. Individual fields can be recognizable, yet full routing-relevant state recovery remains fragile because many fields must be right together. Blind information is nearly saturated, and turn ownership is high for several models, while current bet chips and opponent chip inventory are weak: the best CB score is 45.8%, and the best OCI score is 43.8%.  The three system-level case studies pair GPT 5.5 with the π0 dexterous policy. They are not presented as a statistically powered success-rate estimate. Their value is operational: the traces show repeated waiting, verification, continuation of multi-atom actions, recovery dispatches, and occasional human-help requests. One 23-state trajectory views both hole cards, raises, checks, calls a 200-chip bet, and reveals both cards, with about a third of states spent in the wait branch.  RDT Fine-Tuning Diagnostic  The RDT data-scaling study is a useful caution against overreading pretraining. The authors compare random initialization with a gripper-pretrained RDT checkpoint under 10%, 20%, 50%, and 100% of the DexHoldem training split. At 10% data, pretrained initialization reduces validation loss by only 1.2% relative to random initialization. The reductions grow to 9.0%, 10.7%, and 11.3% at 20%, 50%, and 100%, but both curves follow ..."
    },{
      "title": "DeformGen: Dynamics-Based Topology Augmentation for Deformable Manipulation Policy Learning",
      "url": "/posts/2026/06/deformgen-paper-notes/",
      "date": "2026-06-26",
      "tags": ["Deformable Manipulation","Data Augmentation","Imitation Learning","Simulation","Robot Learning","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  DeformGen is a data augmentation framework for deformable object manipulation. It starts from sparse demonstrations, generates new physically plausible deformable object states through localized physical disturbances and forward simulation, then transfers the original manipulation trajectory to each new state using deformation-field warping. The goal is to turn one or a few demonstrations into many usable training episodes for rope, cloth, toy, and other deformable manipulation tasks.  My read: the paper is useful because it states the failure mode of rigid-object demonstration augmentation precisely. Rigid augmentation assumes equivariance: if the object pose changes by an SE(3) transform, the end-effector trajectory can receive the same transform and remain valid. Deformable objects break this assumption. Their state is high-dimensional, physically constrained, and non-equivariant under local deformation. DeformGen therefore augments both sides together: the object topology and the behavior attached to that topology.  Paper and Resources  The paper is “DeformGen: Dynamics-Based Topology Augmentation for Deformable Manipulation Policy Learning” by Zili Lin, Wenyao Zhang, Yuyang Zhang, Zekun Qi, Junyan Lin, Hanxin Zhu, Jiaolong Yang, Zhibo Chen, Yao Mu, Xiaokang Yang, Xin Jin, and Wenjun Zeng. The affiliations include Shanghai Jiao Tong University, Eastern Institute of Technology, Ningbo, Tsinghua University, The Hong Kong Polytechnic University, University of Science and Technology of China, Zhongguancun Academy, and Microsoft Research.  The paper is available as arXiv:2606.25939. The project page is zili2002.github.io/DeformGen, and the code is at Zili2002/DeformGen.  Why Rigid Augmentation Fails  For rigid manipulation, a single demonstration can often be reused by changing the object pose and applying a matching rigid transform to the robot trajectory. This works because distances between material points are preserved, and the relative pose between gripper and object can be kept consistent.  Deformable manipulation has two different problems.  First, the object state is not a 6-DoF pose. The paper models a deformable object as \\(N\\) particles:  \\[s = (p_1, \\ldots, p_N) \\in \\mathbb{R}^{3N}.\\]  Only a small subset \\(S_{\\mathrm{real}} \\subset \\mathbb{R}^{3N}\\) corresponds to physically plausible configurations. A random perturbation in particle space can easily create disconnected ropes, impossible cloth folds, self-intersections, or shapes with unrealistic internal stress. Low-dimensional rigid pose perturbations stay valid, but they cover only spatial diversity, not shape or topological diversity.  Second, trajectory transfer is non-equivariant. When a rope bends or a cloth drapes, nearby material points move differently. A global rigid transform of the end-effector trajectory may misalign the grasp pose, miss the local geometry, or fail to compensate for the changed global shape. For deformables, valid data augmentation must synthesize a new object state and a new trajectory that matches that state.  State Augmentation: Simulate Into Plausible Topologies  DeformGen’s first component is Dynamic Topological Transformation. Instead of directly editing the particle positions, it starts from a known valid state \\(s_0 \\in S_{\\mathrm{real}}\\), applies localized physical disturbances, and forward-simulates the object dynamics:  \\[s_{\\mathrm{aug}} = \\Phi_{\\mathrm{sim}}(s_0, f, \\Delta t).\\]  Here \\(f\\) is a localized force field and \\(\\Phi_{\\mathrm{sim}}\\) is the simulator rollout. The key assumption is conditional: a good simulator approximately preserves physical plausibility when it evolves from a valid state, while it cannot reliably repair an invalid state produced by arbitrary geometric perturbation.  This is the core design choice. DeformGen does not ask the simulator to fix broken shapes after the fact. It uses the simulator as the generator itself. Localized forces can produce bending, twisting, folding, draping, and compression while the internal constraints, contacts, and settling dynamics remain coupled through simulation.  In the implementation, the simulator does not expose a direct external-force API. The authors implement disturbances by commanding an xArm7 gripper to execute randomized Cartesian perturbations while contacting the object. Rope and toy use 180 random steps; cloth uses 260. After perturbation, the object is stabilized for 30 to 40 simulation steps.  Trajectory Augmentation: Warp the Demonstration Through the Deformation Field  Generating a new object state is only half the problem. The source demonstration trajectory must also be adapted. DeformGen uses Deformation-Field Warping.  Let \\(p^{\\mathrm{orig}}, p^{\\mathrm{def}} \\in \\mathbb{R}^{N \\times 3}\\) be the source and deformed point clouds. Each particle displacement is:  \\[\\delta_i = p^{\\mathrm{def}}_i - p^{\\mathrm{orig}}_i.\\]  For an end-effector waypoint \\(x_t\\), DeformGen finds its \\(K\\) nearest source particles and interpolates their displacements with inverse-distance weights:  \\[d(x_t) = \\sum_j \\tilde{w}{t,j}\\delta{\\mathrm{nn}_j(x_t)}.\\]  The warped waypoint is:  \\[x_t^{\\mathrm{warp}} = x_t + \\alpha_t d(x_t).\\]  The scalar \\(\\alpha_t) is a decay function. It lets the trajectory strongly follow local deformation near the grasp phase while gradually reverting toward the original path when useful for the manipulation phase.  Orientation is adapted by estimating a local Jacobian from the original KNN neighborhood to the deformed neighborhood. The induced rotation is projected onto \\(SO(3)\\), then interpolated with the original orientation using SLERP:  \\[R_t^{\\mathrm{warp}} = \\mathrm{SLERP}(R_t, R’_t, \\alpha_t).\\]  The practical detail is important: the grasp pose uses small \\(K\\), because grasp alignment depends on nearby geometry. The manipulation phase uses all object points, because the motion must compensate for the global deformation of the object.  Policy Training Setup  The experiments use Real2Sim-Eval with PhysTwin for soft-body dynamics and rendering. The robot is an xArm7 with two RGB cameras: third-person and wrist-mounted, both at 848 by 480 and 30 Hz. The policy outputs an 8D action: end-effector position, quaternion orientation, and gripper opening.  The paper evaluates three tasks:                    Task        Goal        Success criterion                            Rope routing        Thread a rope through a clip        enough rope intersections with upper and lower clip planes                    Toy packing        Put a stuffed toy into a container        enough object points inside a scaled oriented bounding box                    Cloth folding        Fold cloth into a triangle        projected mask matches a fitted triangle by contour and IoU criteria              For each task, the authors collect one teleoperation source demonstration, generate more than 1,200 distinct object states, synthesize trajectories, and keep successful rollouts. They train ACT, Diffusion Policy, SmolVLA, and \\(\\pi_0\\) with LoRA, then evaluate on held-out object states unseen during training.  Baselines and Ablations  The comparison is structured to separate state augmentation from trajectory transfer:                    Method        State augmentation        Trajectory transfer                            1 Src.        none        source demo only                    SoftMimicGen*        rigid state perturbation        deformation-field warping                    DeformGen*        topological state augmentation        local rigid trajectory transfer                    DeformGen        topological state augmentation        deformation-field warping              This setup isolates two questions. Comparing SoftMimicGen* with DeformGen tests whether topology-diverse states help. Comparing DeformGen* with DeformGen tests whether deformation-field warping improves over local rigid transfer on the same augmented states.  Results  The headline result is that DeformGen improves policy learning in most settings. In Table 2, the single-source baseline is close to zero success on most tasks. Full DeformGen achieves the best average success for three of four policy architectures:      ACT: 59.00% average success with DeformGen, compared with 1.33% for one source demo.    SmolVLA: 56.50% with DeformGen, compared with 2.50% for one source demo.    \\(\\pi_0\\): 56.67% with DeformGen, compared with 2.33% for one source demo.    Diffusion Policy is the exception where DeformGen* slightly exceeds full DeformGen on average, but the overall trend still supports the value of topological augmentation. The strongest numbers appear in rope routing, where DeformGen reaches 90.50% with ACT, 92.00% with SmolVLA, and 99.00% with \\(\\pi_0\\).  The state coverage analysis supports the mechanism. Rigid augmentation clusters near the source state with near-zero non-rigid residual. ..."
    },{
      "title": "ViserDex: Visual Sim-to-Real for Robust Dexterous In-hand Reorientation",
      "url": "/posts/2026/06/viserdex-paper-notes/",
      "date": "2026-06-24",
      "tags": ["Dexterous Manipulation","Sim-to-Real","3D Gaussian Splatting","Reinforcement Learning","RGB Perception","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  ViserDex builds an RGB-only sim-to-real pipeline for goal-conditioned dexterous in-hand reorientation. The system uses a monocular wrist camera to estimate object pose, feeds that pose into a recurrent student policy, and deploys the policy on a 16-DoF Allegro Hand. Its main technical move is to use 3D Gaussian Splatting (3DGS) as the visual simulator and perform domain randomization before rasterization, directly on Gaussian attributes. This gives photorealistic but diverse pose-estimation training data without the cost of full ray-traced scene randomization.  My read: the paper is less about inventing a new hand policy and more about making visual state estimation usable enough for dexterous control. The RL policy still learns from privileged simulation, but real-world deployment depends on a monocular RGB pose estimator that survives occlusion, motion, texture variation, and adversarial lighting. The useful lesson is that perception fidelity and perception diversity have to be engineered together; a prettier renderer alone does not close the visual sim-to-real gap.  Paper and Resources  The paper is “ViserDex: Visual Sim-to-Real for Robust Dexterous In-hand Reorientation” by Arjun Bhardwaj, Maximum Wilder-Smith, Mayank Mittal, Vaishakh Patil, and Marco Hutter from ETH Zurich and NVIDIA. It is available as arXiv:2604.11138, and the project page is rffr.leggedrobotics.com/works/viserdex. The project page lists the work as accepted to RSS 2026 and includes videos, visual comparisons, and the BibTeX entry.  The task is continuous goal-conditioned in-hand object reorientation: given a target orientation, the hand must repeatedly rotate an object until the target is reached, then move to the next target. The deployed system uses an Allegro Hand, an Intel RealSense D435i wrist camera, a per-object segmentation module, a keypoint-based pose estimator, and a recurrent control policy running at 30 Hz.  Why This Problem Is Hard  Dexterous in-hand reorientation is a cruel perception problem. The hand itself occludes the object, the object moves quickly, contact changes are discontinuous, and small pose errors can accumulate into control failures. Prior RGB-based demonstrations often relied on simple objects, multiple cameras, or expensive automatic domain randomization. Tactile and depth sensing help in different ways, but each adds its own hardware and modeling assumptions.  ViserDex chooses a modular decomposition:      A privileged teacher policy learns contact-rich reorientation in simulation.    A recurrent student policy learns to act from noisy proprioceptive and exteroceptive observations.    A monocular RGB pose estimator predicts 2.5D object keypoints from segmented RGB crops.    A 3DGS rendering pipeline generates the pose-estimator training data with structured appearance randomization.    This division is important. End-to-end RL from pixels would need to learn manipulation, hidden-state inference, and visual sim-to-real transfer at the same time. ViserDex separates the load: RL solves control under state-like observations, while 3DGS-based data generation solves the hard visual transfer problem for object pose estimation.  Control Pipeline  The teacher policy is trained with PPO in simulation and has access to privileged observations. Its action is a 16-dimensional target joint-position command for the Allegro Hand. The reward combines an orientation-tracking term, a sparse success bonus, and regularization penalties for smoother, lower-energy actions. A success is counted when the orientation error is within a threshold; episodes end if the object is dropped, if the policy goes too long without success, or after a long successful sequence.  The student policy removes privileged access. It receives noisy proprioception, noisy object pose, the goal, action history, and timing information. To handle partial observability, the student uses a recurrent belief encoder. The training objective combines behavior cloning from the teacher and a reconstruction loss:  \\[L = L_{\\mathrm{BC}} + \\lambda L_{\\mathrm{recon}}.\\]  The reconstruction target includes exteroceptive and privileged information. In effect, the recurrent state learns to smooth and infer latent physical context from imperfect observations. The paper’s appendix shows why this matters: during injected pose-estimation failures, the belief decoder can reject high-amplitude outliers such as a 180-degree flip and keep the controller stable for several timesteps.  The teacher training also uses a performance-based curriculum instead of heavy ADR over many parameters. As the moving average of consecutive successes improves, the curriculum increases regularization penalties, action-latency randomization, and the pressure to complete goals faster. The ablation is sharp: removing the regularization-penalty curriculum or using no curriculum causes near-zero learning progress in the reported curves.  3DGS as the Visual Simulator  The perception module needs object pose from RGB. ViserDex trains a ResNet-34 pose estimator that predicts nine normalized 2.5D keypoints: eight object-specific keypoints plus the centroid. The 2.5D keypoints are converted to a 6D pose through rigid Procrustes alignment.  The training data is generated in simulation, but the renderer is unusual. Each object is represented as a 3D Gaussian scene, where each Gaussian has position, covariance, opacity, and spherical harmonic color coefficients. For a viewing direction \\(d\\), color is computed from spherical harmonics:  \\[c(d) =\\mathrm{Sigmoid}\\left(\\sum_{\\ell=0}^{L}\\sum_{m=-\\ell}^{\\ell}k_{\\ell}^{m}Y_{\\ell}^{m}(d)\\right).\\]  During manipulation, the object moves while the camera remains fixed. ViserDex keeps the Gaussian scene static by applying the inverse object transform to the camera before rendering. This preserves vanilla 3DGS assumptions while producing images from the correct object pose. Since object-only 3DGS would ignore finger occlusions, the system also renders a low-fidelity hand depth map from the physics simulator and masks out Gaussian pixels hidden behind the hand.  The result is a pragmatic hybrid: the object appearance is photorealistic, the hand occlusion is physically aligned, and the rendering cost remains compatible with high-throughput simulation.  Pre-Rasterization Gaussian Augmentations  The core perception idea is to randomize the Gaussian representation before rasterization. Standard image augmentation is cheap but operates in 2D and often ignores geometry. Full scene randomization through ray tracing is physically grounded but expensive. ViserDex works between these extremes by perturbing Gaussian attributes, especially spherical harmonic coefficients, before rendering.  The key is structured perturbation. Randomizing each Gaussian independently creates noisy artifacts. Real lighting, material, exposure, and wear usually affect coherent spatial regions or material-like groups. ViserDex therefore clusters Gaussians and perturbs clusters:                    Augmentation group        What it changes        Intuition                            Random noise        Individual Gaussian attributes        Sensor artifacts, tiny appearance errors                    Spatial clusters        Position-correlated Gaussian groups        Local shadows, marks, damage, patch effects                    Color clusters        SH0 color-correlated groups        Material-specific albedo and reflectance shifts                    Global shift        Whole Gaussian scene        Exposure, color temperature, saturation, ambient light              This is the part I would keep from the paper. The authors do not simply say “use 3DGS because it looks real.” They show that realism without diversity can transfer poorly. In Table II, the naive Gaussian Splatting baseline under adversarial lighting reaches only 36.5% mean strict pose accuracy, below randomized tiled rendering at 47.2%. The full ViserDex augmentation pipeline reaches 56.3%. The visual simulator helps because it is both high-fidelity and controllably randomized.  The ablation makes the mechanism clearer. Under adversarial lighting, removing global shift collapses mean pose accuracy to 23.6%, and the adversarial mean rotation error rises to 38.9 degrees. Spatial and color cluster removals also hurt substantially. Global appearance shifts are the most important for lighting robustness, while structured cluster perturbations help local and material-level generalization.  Results  For pose estimation, ViserDex reports strict accuracy using an ADD-style threshold: prediction error below 10 mm and 10 degrees. Across five objects, the full method reaches 65.4% mean accuracy under nominal lighting and 56.3% under adversarial lighting. It also improves adversarial rotation error to 14.6 degrees, compared with 17.7 degrees for domain-randomized tiled rende..."
    },{
      "title": "Towards Bridging the Gap: Systematic Sim-to-Real Transfer for Diverse Legged Robots",
      "url": "/posts/2026/06/pace-sim2real-paper-notes/",
      "date": "2026-06-21",
      "tags": ["Legged Robots","Sim-to-Real","System Identification","Actuator Modeling","Reinforcement Learning","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  PACE is a sim-to-real pipeline for legged robots that treats actuator and joint dynamics as the main reality gap. The key move is practical: collect short fixed-base, in-air encoder trajectories, fit a compact physical parameterization in simulation with CMA-ES, then train locomotion policies directly in the fitted simulator without dynamics randomization. The paper is useful if you are thinking about parameter identification as an alternative to ActuatorNet-style black-box actuator modeling.  My read: PACE is strongest as a joint-space dynamics alignment recipe. It does not try to learn a full residual simulator. It fits per-joint effective inertia, viscous damping, Coulomb friction, joint bias, and one global command delay. That gives a small parameter vector, enough physical meaning to debug, and a workflow that works even when the robot has only joint encoders and no joint torque sensors.  Paper and Resources  The paper is “Towards Bridging the Gap: Systematic Sim-to-Real Transfer for Diverse Legged Robots” by Filip Bjelonic, Fabian Tischhauser, and Marco Hutter from ETH Zurich’s Robotic Systems Lab. It is available as arXiv:2509.06342, with the project code at leggedrobotics/pace-sim2real, documentation at pace.filipbjelonic.com, and an ETH Research Collection dataset for actuator model identification and locomotion experiments at PACE Dataset for Sim-to-Real Transfer in Legged Robots.  The repository frames PACE as Precise Adaptation through Continuous Evolution. In the public code path, the basic example collects excitation data and runs scripts/pace/fit.py, which estimates actuator and joint parameters with CMA-ES for Isaac Lab / Isaac Sim 5.0-style workflows.  Why Actuator Modeling Matters Here  For legged locomotion, the actuator model can dominate sim-to-real transfer. A policy trained against a URDF-only model may learn joint trajectories that look plausible in simulation but land in the wrong phase and energy regime on hardware. Earlier ETH work used actuator networks: learned models that map histories of commands and joint states to torques, often requiring torque-instrumented data and careful data collection.  PACE takes a more physically constrained route. It asks whether the important low-level discrepancy can be captured by a small number of joint-space parameters. The paper’s answer is yes for the tested quadrupeds: a fitted simulator can match in-air joint trajectories, generalize across gains and trajectories, and remain competitive with an ActuatorNet baseline on ANYmal while using much less and less specialized data.  The important distinction is interpretability. An actuator network can be very expressive, but its errors are harder to attribute. In PACE, if the fitted \\(I_a\\) is too large, you can reason about rotor inertia, CAD link inertia, firmware compensation, and apparent inertia. If damping shifts, you can look at gearbox, motor, compensation, temperature, or motor-constant mismatch. That makes the method feel more like a debugging loop than a pure function approximator.  Data Collection Setup  PACE collects data with the robot base fixed and the legs moving freely in air. This removes unmeasured contact forces and avoids base-motion coupling. The authors excite all joints simultaneously with chirp signals, usually 20-60 seconds per sequence, with high-rate synchronized logging. The target is not to reproduce locomotion contact directly; the target is to isolate the joint and drive dynamics that will later shape contact behavior.  Three practical details matter:      Fixed base: the simulated replay uses the same base pose as the real experiment.    No contact: legs move in air, so the loss does not need foot forces or contact estimates.    Excitation bandwidth: the chirp should cover the frequencies the policy can excite, or at least twice the highest frequency expected in the actual walking motion.    The paper also emphasizes PD gains. If gains are too high, the closed-loop poles move to high frequencies, making the required excitation bandwidth hard or unsafe. PACE uses small gains for identification and policy training so that the characteristic dynamics are visible in the collected data.  The joint transfer function used to explain this is:  \\[H_q(s) = e^{-sT_d}\\frac{P_\\tau}{I_a s^2 + (d + D_\\tau)s + P_\\tau}\\]  Here \\(I_a\\) is effective armature inertia, \\(d\\) is viscous damping, \\(T_d\\) is lumped delay, and \\(P_\\tau,D_\\tau\\) are the joint-level PD gains.  Parameter Identification  The fitted parameter vector is deliberately small:  \\[\\mathbf{p} =[\\mathbf{I}_a,\\mathbf{d},\\boldsymbol{\\tau}_f,\\tilde{\\mathbf{q}}_b,T_d]^\\top\\in \\mathbb{R}^{4n+1}\\]  For \\(n\\) actuated joints, PACE fits per-joint effective inertia \\(\\mathbf{I}_a\\), viscous damping \\(\\mathbf{d}\\), Coulomb friction \\(\\boldsymbol{\\tau}_f\\), joint bias \\(\\tilde{\\mathbf{q}}_b\\), and one global command delay \\(T_d\\). For the robots in the paper, this gives about 49 parameters, small enough for evolutionary search in massively parallel simulation.  The simulator runs \\(N=4096\\) environments in parallel. Each environment samples a candidate parameter vector, replays the recorded real joint targets, and compares simulated joint positions to the measured ones:  \\[\\ell_e =\\frac{1}{k}\\sum_{i=1}^{k}\\left|\\mathbf{q}i^{\\mathrm{real}}-\\mathbf{q}{i,e}^{\\mathrm{sim}}\\right|^2\\]  PACE then solves:  \\[\\mathbf{p}^{\\ast} =\\arg\\min_{\\mathbf{p}}\\mathbb{E}[\\ell_e]\\]  using CMA-ES over the parallel population. The choice makes sense: gradients through the full simulator are not required, the dimension is moderate, and the objective can have local traps caused by delay, saturation, compensation, and friction.  At the single-joint level, the reference model is:  \\[I_a\\ddot q+d\\dot q=\\tau_i+\\tau_{\\mathrm{comp}}+\\tau_f\\]  and the practical closed-loop form is:  \\[I_a\\ddot q+d\\dot q=\\mathrm{sat}\\left(P_\\tau(\\hat q-q+\\tilde q_b)-D_\\tau\\dot q+\\tau_{\\mathrm{comp}}\\right)+\\tau_f\\]  This equation is the heart of the paper for me. The authors are not just matching trajectories; they are choosing a parameterization that absorbs the effects that matter at the joint: inertia-like terms, damping-like terms, Coulomb friction, bias, firmware compensation, and saturation.  One subtle but important point: PACE does not co-optimize PD gains with the dynamics. If \\(I_a,d,P_\\tau,D_\\tau\\) are scaled together, the same trajectories can be preserved, which creates non-uniqueness. The gains are treated as known, and the fit focuses on the physical simulator parameters.  PACE versus ActuatorNet  The paper’s ANYmal comparison is the cleanest place to read PACE against ActuatorNet. The authors compare three settings: URDF-only, actuator network, and PACE. URDF-only diverges in in-air replay and fails in forward walking. Both actuator network and PACE transfer, but PACE has smaller delta phase-portrait spread in the reported in-air comparison and avoids the joint-position bias visible in the actuator-network baseline.  The data story is also different. PACE uses roughly 20 seconds of encoder-only in-air data per robot. Actuator networks generally need minutes of torque-instrumented data, and the deployed vendor LSTM baseline was likely trained on an even larger dataset. That changes the engineering trade-off: ActuatorNet is appealing when torque sensing and broad training logs are available; PACE is attractive when you want a lower-data, encoder-only path with parameters that can be inspected.  For me the useful mental model is:                    Method        What it learns        Data pressure        Debuggability                            ActuatorNet        black-box actuator mapping, often recurrent        higher, often torque-instrumented        lower                    PACE        compact physical joint-space parameters        lower, encoder-only in-air trajectories        higher              The paper does not make ActuatorNet obsolete. It shows that for many PMSM-driven legged robots, a small physically meaningful parameter set can cover the main gap well enough to train blind locomotion without dynamics randomization.  Results and What to Keep  The single-drive experiments validate that the fitted inertia tracks known analytic changes in load. At the full-robot level, Tytan, ANYmal, and Minimal show close real-sim trajectory overlays in in-air replay. The fitted simulators generalize across unseen gains and trajectories, which is important because a parameter fit that only memorizes one chirp would be much less useful.  On the locomotion side, policies are trained in fitted simulation and deployed zero-shot. The paper reports deployment across three main platforms and more than ten additional robots. It also reports an energy result: ANYmal D reaches full Cost of Transport 1.27, about 32% lower than the state-of-the-art ANYmal C reference in the paper’s comparison. Tytan..."
    },{
      "title": "EgoInfinity: A Web-Scale 4D Hand-Object Interaction Data Engine",
      "url": "/posts/2026/06/egoinfinity-paper-notes/",
      "date": "2026-06-21",
      "tags": ["Robot Learning","Human Videos","Egocentric Data","Retargeting","Manipulation","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  EgoInfinity is best read as a data engine: a system for turning arbitrary in-the-wild RGB manipulation videos into robot-usable 4D hand-object interaction data. The output includes metric hand trajectories, object point clouds and meshes, 6-DoF object poses, contact-relevant interaction states, and robot-specific retargeted trajectories.  The central idea is to make internet video actionable. EgoInfinity starts from web-scale Action100M-style clips, filters for manipulation, reconstructs hands and objects in a shared metric 3D frame, repairs hand-object drift with interaction-aware rules, then compiles the recovered hand motion into executable robot trajectories through a learned root-frame estimator plus IK. The most useful takeaway is the representation boundary:  \\[H_t={M_t^h,K_t^h,{}^c p_t^h,P_t^o,M^o,{}^c p_t^o}\\]  This state is agent-agnostic: it describes the human hand, object geometry, and object pose in metric 4D space before choosing a target robot embodiment. That is the reason the same recovered interaction can be retargeted to Unitree G1, Robonaut2, dual Franka arms, and a LEAP hand policy setting.  Paper and Resources  The paper is “EgoInfinity: A Web-Scale 4D Hand-Object Interaction Data Engine for Any-View Robot Retargeting and Video-to-Action Robot Learning” by Gaotian Wang, Kejia Ren, Andrew Morgan, Yiting Chen, Howard H. Qian, Podshara Chanrungmaneekul, and Kaiyu Hang. It is available as arXiv:2606.17385. The project page is the EgoInfinity Hugging Face Space, and the public dataset entry is Rice-RobotPI-Lab/egoinfinity.  The arXiv paper reports the data engine as corpus-agnostic, with the current scale bounded by Action100M: 14.6 years of footage, 147M action segments, and a table-level comparison listing 127K hours. The released browser/dataset is a curated, inspectable subset: the paper reports 106 processed manipulation videos, 277 objects, 12,107 frames, and median 5.8 s clip duration; the Hugging Face dataset page currently exposes 1,150 rows and about 5.49 GB.  The Problem: Video Is Huge, Robot Data Is Actionable  Egocentric and internet videos contain enormous manipulation diversity, but most of that signal is trapped in pixels. A robot needs metric geometry, object state, contact timing, feasible motion, and embodiment-specific commands. Existing robot datasets provide actions but are expensive and hardware-bound; existing human-video datasets scale better but usually stop at observation, narration, pose, or weak action labels.  EgoInfinity tries to close that gap by turning RGB videos into an intermediate form that is closer to robot action but still independent of any one robot. This is the right design pressure. If the engine produced only 2D tracks, downstream policies would still need to infer geometry. If it produced only robot actions, the output would be tied to one embodiment. The paper instead builds a metric 4D HOI representation and adds retargeting as a compilation step.  Data Engine Pipeline  The pipeline has two passes. The first pass is cheap: scan videos, identify hand-present segments, filter by hand-motion statistics and camera-motion cues, and retain clips likely to contain useful manipulation. The second pass runs the full reconstruction stack only on the active segments. The current paper focuses on approximately static-view videos, which is a practical assumption for many tutorial and how-to clips.  The full stack has four conceptual blocks:      Metric geometry and hand tracking. MoGe-2 estimates focal length and global metric scale; Flow3R predicts dense depth; GeoCalib estimates gravity. WiLoR reconstructs MANO hand parameters and hand meshes, with infilling and smoothing to handle missing frames.    Object discovery and reconstruction. Text annotations from Action100M are used to extract object prompts. SAM-3 detects target regions, SAM-2 tracks masks through time, SAM-3D reconstructs object meshes, and FoundationPose++ estimates 6-DoF object pose trajectories.    Interaction-aware refinement. MEMFOF optical flow, hand keypoints, object masks, and point clouds are used to classify object state. The main text compresses this to \\(s_t\\in\\{\\mathrm{static},\\mathrm{grasped},\\mathrm{moving}\\}\\), while the appendix uses finer labels such as left-hand grasp, right-hand grasp, and bimanual grasp.    Coordinate cleanup and exo-to-ego reframing. The system can erode masks, filter depth boundaries, remove outliers, and synthesize an egocentric view by rigidly re-rendering the recovered 3D scene from a hand-centered virtual camera.    The important implementation point is cross-module metric calibration. EgoInfinity turns a collection of foundation-model outputs into one shared metric camera-world frame: hand predictions, object predictions, dense depth, camera calibration, gravity, masks, and object priors are all aligned before contact reasoning. Without that, hand-object contact reasoning would collapse into scale mismatch and pose drift.  Interaction-Aware Refinement  The refinement stage is the paper’s strongest systems idea. Pure visual object tracking is brittle under occlusion, especially when the hand covers the manipulated object. EgoInfinity uses interaction state to decide which geometric signal should drive object pose.  For each frame, it first forms an object proposal:  \\[\\tilde p_t^o=(R_{\\mathrm{cano}},\\operatorname{center}(S_t^o\\odot D_t))\\]  Here \\(R_{\\mathrm{cano}}\\) comes from the canonical SAM-3D orientation, \\(S_t^o\\) is the object mask, and \\(D_t\\) is depth. Then the state determines the trusted pose source:      If the object is static, lock it to a robust point-cloud centroid.    If it is grasped, bind it rigidly to the hand frame with a canonical palm-aligned transform.    If it is moving but not confidently grasped, keep the visual proposal and smooth it.    In the appendix, the state classifier is more concrete. A global-static gate catches fixtures whose mask centroid barely moves; a Schmitt trigger stabilizes per-frame motion detection; grasp signals combine 2D mask overlap, fingertip-to-cloud distance, and wrist-to-cloud distance; temporal smoothing fills short gaps and removes short false runs. This is a nice example of using simple geometry and temporal logic to make foundation-model outputs more physically usable.  Functional Retargeting  The retargeting section is another key piece. Internet videos often show only hands, partial arms, or arbitrary viewpoints, so exact human body-pose imitation is a fragile target. EgoInfinity uses functional retargeting: preserve task-relevant hand motion and choose a feasible robot root frame, without requiring full human kinematic recovery.  Given recovered hand trajectories and optional gravity, the retargeter estimates a robot-specific kinematic root frame:  \\[{}^c p_r=({}^c R_r,{}^c t_r)\\in SE(3)\\]  The estimator is an SE(3)-equivariant Vector Neuron network trained entirely in MuJoCo simulation. It predicts plausible root frames from hand trajectories, using flow matching to represent ambiguity: many torso/root poses can explain the same observed hand path. At inference time, the system samples multiple root-frame hypotheses, clusters them, interpolates smooth root trajectories across windows, then scores candidates by IK convergence, residual tracking error, manipulability, joint-limit margin, and smoothness.  This design makes the representation boundary clean:  RGB video    -> metric 4D hand-object state    -> robot-specific root-frame estimate    -> IK and smoothing    -> executable joint trajectory    Finger motion is handled separately for dexterous hands through robot-specific mapping from MANO keypoints. Arm joints follow wrist-level IK targets; finger joints use hand geometry.  Experiments  The curated Action100M subset contains 106 clips, 277 objects, and 12,107 frames. The paper reports that 88% of clips and 47% of objects involve manipulation, with a mix of static, moving, left-hand, right-hand, and bimanual grasp states. Object categories include containers, tools, food, hardware, appliances, textiles, electronics, decor, paper, and hygiene objects; top verbs include place, add, show, season, arrange, hold, pick, pour, present, remove, insert, and slice.  For cross-embodiment motion retargeting, the paper evaluates Unitree G1, Robonaut2, and dual Franka FR3:                    Robot        IK rate        Position error        Orientation error                            Unitree G1        0.821        2.86 cm        6.73 deg                    Robonaut2        0.774        6.67 cm        8.25 deg                    Dual-Franka        0.706        10.27 cm        12.17 deg              The trend makes sense. Unitree G1 has stronger whole-body reachability for many human-like hand motions; dual Franka is constrained by a tabletop-style bimanual setup and larger morphology mismatch. The ..."
    },{
      "title": "TopoRetarget: Interaction-Preserving Retargeting for Dexterous Manipulation",
      "url": "/posts/2026/06/toporetarget-paper-notes/",
      "date": "2026-06-19",
      "tags": ["Dexterous Manipulation","Motion Retargeting","Reinforcement Learning","Trajectory Optimization","Sim-to-Real","Robot Learning","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  TopoRetarget shifts dexterous retargeting from Human Pose -> Robot Pose to Human Hand-Object Interaction -> Robot Hand-Object Interaction. The central object in the method is a shared interaction mesh that contains both hand keypoints and object surface points. On top of that mesh, the method optimizes topology-aware Laplacian coordinates so that the robot hand preserves the local hand-object interaction pattern in the human demonstration.  The important part of the paper is Sections 3.2-3.4. The method first builds a reasonable robot-hand warm start from relative bone directions, then constructs a shared hand-object graph, then refines the robot hand by matching local topology in place of absolute keypoint positions. The RL controller comes after this retargeting stage: it tracks the generated reference with residual joint-position actions and absorbs dynamics, timing, and sim-to-real robustness.  Paper: “TopoRetarget: Interaction-Preserving Retargeting for Dexterous Manipulation”. arXiv: 2606.16272. Project page: toporetarget2026.github.io/TopoRetarget.  Core Framing  TopoRetarget moves the goal away from direct keypoint fitting. Human and robot hands differ in bone lengths, joint layout, palm shape, finger arrangement, and feasible contact surfaces. A retargeted pose can match fingertips in Euclidean space while losing the actual manipulation structure, especially when useful contact occurs on phalanges, finger sides, or palm regions.  The paper therefore treats manipulation retargeting as preservation of local hand-object interaction. The robot should keep the same local relationship between hand regions and object regions as the human demonstration. In this view, object-relative geometry becomes the object of imitation. The optimization target shifts from “where is this human keypoint in global space” to “where is this hand point relative to its neighboring hand and object points”.  3.2 Relative Bone-Direction Initialization  The first stage provides a robot-hand initialization. Since human and robot hands have different geometry, TopoRetarget uses the local bending pattern of the fingers as the initialization signal and avoids direct copying of absolute hand keypoint positions.  For each non-terminal keypoint \\(k\\), define the bone direction \\(d_k\\) as the unit vector from the current keypoint to its child keypoint. \\(d_k^s\\) denotes the source human bone direction, and \\(d_k^r(q)\\) denotes the robot bone direction under joint configuration \\(q\\).  The key design is to compare relative changes between adjacent bones on the same finger. For adjacent bone pairs \\((k,l)\\in A_B\\), the bone-direction mismatch is:  \\[E_{bone}(q)=\\sum_{(k,l)\\in A_B}\\left|(d_k^r(q)-d_l^r(q))-(d_k^s-d_l^s)\\right|_2^2 .\\]  This loss captures local articulation. It describes how a finger bends from one bone to the next and avoids forcing a single bone to point in an absolute direction. This distinction matters for cross-embodiment transfer because link lengths and palm frames are mismatched across hands.  The initialization solves:  \\[\\tilde q_t^r=\\arg\\min_q\\lambda_{warm}E_{bone}(q)+\\lambda_{smooth}|q-\\tilde q_{t-1}^r|_2^2 .\\]  The first term makes the robot reproduce the local hand shape. The second term keeps the initialization temporally continuous. The result \\(\\tilde q_t^r\\) serves as a warm start for the final retargeting output. Its purpose is to put the robot hand near a plausible local articulation state before the interaction-aware refinement begins.  3.3 Interaction Mesh Construction  Matching hand shape alone is insufficient for manipulation. The core signal is the relationship between the hand and the object.  At frame \\(t\\), TopoRetarget forms a source vertex set and a robot vertex set:  \\[V_t^s=[P_t^h;O_t],\\qquadV_t^r(q)=[P_t^r(q);O_t].\\]  \\(P_t^h\\) is the human hand keypoint set, \\(P_t^r(q)\\) is the robot hand keypoint set, and \\(O_t\\) is a set of points sampled from the object surface. The graph has \\(N_v=21+N_o\\) vertices: the first 21 are hand keypoints, and the remaining vertices are object surface points.  The paper runs Delaunay tetrahedralization on the source vertices \\(V_t^s\\) to obtain the interaction edge set \\(\\mathcal I_t\\). This gives a source graph:  \\[G_t^s=(V_t^s,\\mathcal I_t).\\]  Then it reuses the same connectivity for the robot graph:  \\[G_t^r(q)=(V_t^r(q),\\mathcal I_t).\\]  This shared connectivity is the main structural move. Human and robot graphs now have the same local neighborhood structure, so the optimizer can compare corresponding hand-object interactions directly. The method avoids manually specifying which fingertip, phalanx, or palm point should contact which object region. The interaction mesh encodes those local neighborhoods from the source demonstration.  3.4 Topology-Aware Laplacian Refinement  Once the shared graph exists, TopoRetarget compares local geometry through weighted Laplacian coordinates.  The edge weights \\(w_{ij,t}\\) are computed from spatial distances in the source graph. Close neighbors receive high weight, distant neighbors receive low weight. These weights are computed once on the source graph and then reused for the robot graph.  For vertex \\(v_i\\), the weighted Laplacian coordinate is:  \\[\\Delta_t(V)i=\\sum{j\\in\\mathcal N_t(i)}w_{ij,t}(v_i-v_j).\\]  When the weights are normalized so that \\(\\sum_j w_{ij,t}=1\\), the same expression becomes:  \\[\\Delta_i=v_i-\\sum_jw_{ij,t}v_j.\\]  This is “the current point minus the weighted center of its neighbors.” It describes local structure in place of absolute position. It is naturally insensitive to global translation and fits cross-embodiment comparison better than coordinate matching.  The interaction-mesh energy is:  \\[E_{IM}(q)=\\frac1{N_v}\\sum_{i=1}^{N_v}\\left|\\Delta_t(V_t^r(q))_i-\\Delta_t(V_t^s)_i\\right|_2^2 .\\]  This objective asks the robot graph to match the human graph’s Laplacian coordinates. Put differently, it preserves how a hand point sits relative to surrounding hand points and object surface points. The retained quantity is local hand-object topology: which regions are near each other, in what local direction, and under what local neighborhood geometry.  Final Optimization  The final retargeting problem combines the interaction objective with the hand-shape prior and feasibility terms:  \\[(q_t^{r,\\ast},s_t^\\ast)=\\arg\\min_{q,s}\\lambda_{IM}E_{IM}(q)+\\lambda_{bone}E_{bone}(q)+E_{reg}(q;q_{t-1}^{r,\\ast})+\\frac{w_s}{2}\\sum_{i\\in Q_t}s_i^2 .\\]  \\(E_{IM}\\) is the core term. It keeps local hand-object interaction consistent with the human demonstration. \\(E_{bone}\\) preserves the local articulation prior from initialization. \\(E_{reg}\\) provides temporal smoothness and floating-base regularization. The slack variables \\(s_i\\) belong to the penetration constraints and are penalized so that the optimizer can tolerate small controlled violations while rejecting severe penetration.  The paper also adds signed-distance constraints \\(\\phi_i(q)\\) for robot-hand and object geometry. The design combines a soft tolerance with a hard bound: the optimization can absorb minor geometric noise, while large interpenetration is blocked. This is important because retargeting is often driven by noisy hand-object capture, where tiny contact errors are common and strict zero-penetration constraints can make the problem brittle.  Where RL Fits  The RL part should be read as a tracking layer on top of the retargeted reference. TopoRetarget first produces \\(q_t^{r,\\ast}\\) and object-aligned references; then a PPO controller learns to track them.  The policy uses residual joint-position control:  \\[q^{target}_t = q^{ref}_t + a_t .\\]  The observation contains robot proprioception, object state, and current/lookahead reference information. The reward combines object tracking, hand-link tracking, joint tracking, and smoothness. Domain randomization is used for physical robustness.  This RL section matters because it clarifies the division of labor. Retargeting encodes the contact topology and generates a meaningful reference. RL handles dynamical tracking, residual correction, and robustness. The policy executes and corrects a topology-preserving reference, which reduces the burden of discovering the manipulation sequence from scratch.  Limitations  The most important limitation is source quality. TopoRetarget can preserve local hand-object relations that exist in the captured motion. If the source trajectory contains a virtual contact, where a finger is meant to interact with the object but fails to touch or approach the surface, the interaction mesh has no correct relation to preserve. This suggests that upstream contact completion or motion cleanup may be necessary for noisy egocentric or monocular data.  Another limitation is that Laplacian topology is a geometric proxy. It preserves local neighborhoods, relative directions, and object-relative struc..."
    },{
      "title": "T-Rex: Tactile-Reactive Dexterous Manipulation",
      "url": "/posts/2026/06/t-rex-paper-notes/",
      "date": "2026-06-19",
      "tags": ["Tactile Sensing","Dexterous Manipulation","Vision-Language-Action","Robot Learning","Imitation Learning","Bimanual Manipulation","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  T-Rex argues that tactile feedback should be treated as a high-frequency control signal for dexterous manipulation. In this framing, touch is part of the control loop: vision and language provide slow semantic planning, while touch provides local, fast correction when contact changes inside an action chunk.  The paper combines a tactile-synchronized bimanual robot dataset, a variable-rate Mixture-of-Transformer-Experts (MoT) policy, and a three-stage training recipe. Across 12 real-world contact-rich tasks, T-Rex reaches 65% average success, compared with 35% for EgoScale, the strongest baseline. A useful warning in the results is that π0.5 + tactile drops to 6%, below π0.5 without tactile, showing that tactile signals need architecture and training alignment to help.  Paper Info  The paper is “T-Rex: Tactile-Reactive Dexterous Manipulation” by Dantong Niu, Zhuoyang Liu, Zekai Wang, Boning Shao, Zhao-Heng Yin, Anirudh Pai, Yuvan Sharma, Stefano Saravalle, Ruijie Zheng, Jing Wang, Ryan Punamiya, Mengda Xu, Yuqi Xie, Yunfan Jiang, Letian Fu, Konstantinos Kallidromitis, Matteo Gioia, Junyi Zhang, Jiaxin Ge, Haiwen Feng, Fabio Galasso, Wei Zhan, David M. Chan, Yutong Bai, Roei Herzig, Jiahui Lei, Fei-Fei Li, Ken Goldberg, Jitendra Malik, Pieter Abbeel, Yuke Zhu, Danfei Xu, Jim Fan, and Trevor Darrell.  It is available as arXiv:2606.17055. The project page is tactile-rex.github.io, the code is released at ZhuoyangLiu2005/T-Rex, and the dataset is on Hugging Face as zekaiwang/trex_dataset.  Core Argument  Many VLA policies can interpret instructions and visual context, but contact-rich dexterity often fails at a shorter time scale. Turning a page, extracting a card, squeezing toothpaste, handling an egg, or opening a lock requires quick reactions to force, slip, deformation, and contact geometry. Those signals are local and high-frequency, and the useful correction may need to happen before a slower visual policy replans.  T-Rex addresses this frequency mismatch directly. It keeps a slow visual-language-action pathway for task progress and adds a fast tactile pathway for contact-level refinement. The result is a policy whose control loop is shaped by the sensing modality: vision carries broad context; touch adjusts the action when the physical interaction changes.  Dataset and Training Recipe  The T-Rex Dataset is collected on a fixed-base Dexmate Vega-1 robot with two Sharpa Wave dexterous hands. The setup uses a head camera, two wrist cameras, five fingertip tactile sensors per hand, tactile force vectors, tactile deformation maps, Manus gloves, and VIVE trackers. The full dataset described in the paper contains 100 hours of teleoperation, 7700+ trajectories, 22 motor primitives, 200+ daily objects, and synchronized RGB, tactile, robot-state, action, and language streams. The public release currently contains about 50 hours and 5400+ trajectories in LeRobot v3.0 format.  The data is designed for more than task cloning. By covering elementary motor primitives and object interactions, it gives the model reusable contact-rich building blocks during robot mid-training. The training recipe has three stages: first, the latent and action experts inherit broad visuomotor priors from EgoScale-style pretraining on 22,889 hours of egocentric human video; second, robot mid-training on the T-Rex Dataset aligns those priors with bimanual actions and synchronized tactile feedback; third, task-specific post-training adapts the model with about 100 demonstrations per downstream task. The recipe suggests that tactile reactivity can be learned efficiently in a dedicated robot stage, after large-scale visual pretraining.  Variable-Rate MoT  The model uses a Mixture-of-Transformer-Experts policy with three experts:                    Expert        Role        Rate                            Latent Expert        Future visual latent prediction        Low-rate                    Action Expert        Low-frequency action denoising        About 5 Hz                    Tactile Expert        High-frequency tactile refinement        About 20 Hz              The action expert first produces an intermediate action chunk through flow matching, then the tactile expert refines it using fresh tactile observations. In the implementation, the action chunk length is 16, denoising uses 10 Euler steps, the split is τ_split = 0.4, the action expert runs 6 slow steps, and the tactile expert runs 4 fast refinement steps. Tactile updates are triggered at offsets {0, 4, 8, 12} inside the chunk, so the model can react to new contact without rerunning the full vision-language stack.  The tactile encoder also matches the nature of the signal. For fingertip force, a per-finger VQ-VAE compresses recent 6D force/torque history into temporal tokens while preserving the current force vector for instantaneous contact. For deformation, a convolutional encoder processes tactile maps. The final tactile tokens combine temporal force, current force, and spatial deformation features, allowing the policy to distinguish events such as force spikes, gradual slip, and local surface deformation.  Empirical Evidence  The benchmark contains 12 real-world tactile-reactive tasks: Flip Page, Transfer Egg, Wipe Plate, Apply Toothpaste, Split Cup, Sort Mahjong, Open Lock, Refill Tablet, Acid-Base Neutralization, Extract Card, Deal Poker, and Screw Lightbulb. Each task is evaluated with 16 rollouts under randomized object poses, and multi-stage tasks use progress-based scoring.  Average success across the 12 tasks:                    Method        Average Success                            ViTacFormer        3%                    RDP        6%                    Tactile-VLA        15%                    EgoScale        35%                    π0.5        17%                    π0.5 + tactile        6%                    T-Rex        65%              The main result supports the paper’s central claim: tactile feedback is most useful when the policy can react with a separate fast pathway. The tactile ablation tells the same story:                    Configuration        Average                            Full T-Rex        65%                    w/o Tactile        42%                    MLP Force + Deform        58%                    Deform only        54%                    MLP Force + VQ-VAE Force        59%                    w/o Async        60%              Removing tactile drops success from 65% to 42%. Force and deformation both help, the temporal VQ-VAE improves force modeling, and asynchronous refinement adds a smaller but still meaningful gain. The training ablation further supports the recipe:                    Recipe        Average                            No pretraining, no mid-training        18%                    Pretraining only        34%                    Mid-training only        45%                    Full recipe        65%              The full system wins because it combines broad human-video priors with tactile-grounded robot mid-training. The code release reflects this split: the main branch includes post-training, inference, dataset quickstart, tactile VQ-VAE tools, and robot-side code, while pretraining and mid-training scripts are provided in the full-pipeline branch with released checkpoints.  Limitations and Takeaway  The paper notes that long-horizon tasks with tight contact tolerances remain difficult to teleoperate and learn from demonstrations alone. Reinforcement learning or online interaction-based refinement may be needed for those cases. It also highlights hardware constraints: tactile sensor distortion, calibration drift, cross-device variation, and the lack of dense palm sensing make tactile foundation policies harder to scale. A practical adoption issue is that T-Rex depends on a rich dexterous platform with fingertip tactile sensing, so broader use will depend on tactile hardware becoming more common and standardized.  The clear takeaway is that tactile feedback changes the control problem. For dexterous manipulation, the policy needs slow vision-language planning plus fast tactile-reactive refinement. T-Rex is valuable because it turns that principle into a dataset, architecture, training recipe, and real-world benchmark result.  本文支持通过顶部导航栏的语言切换按钮在 English / 中文 之间切换。  TL;DR  T-Rex 的核心观点是：在灵巧操作里，触觉反馈应该被当作高频控制信号。按照这个视角，触觉属于控制循环的一部分：视觉和语言负责较慢的语义规划，触觉负责在一个 action chunk 内接触状态变化时做局部、快速的修正。  论文把 tactile-synchronized bimanual robot dataset、variable-rate Mixture-of-Transformer-Experts (MoT) policy 和三阶段训练 recipe 结合起来。在 12 个真实 contact-rich tasks 上，T-Rex 平均成功率达到 65%，而最强 baseline EgoScale 是 35%。结果里一个很有价值的警告是：π0.5 + tactile 只有 6%，低于不加 tactile 的 π0.5，说明触觉信号需要匹配的 architecture 和 training alignment 才能真正发挥作用。  Paper Info  论文标题是 “T-Rex: Tactile-Reactive Dexterous Manipulation”，作者包括 Dantong Niu, Zhuoyang Liu, Zekai Wang, Boning Shao, Zhao-Heng Yin, Anirudh Pai, Yuvan Sharma, ..."
    },{
      "title": "RGB-S: Image-Aligned Tactile Saliency for Robust Dexterous Manipulation",
      "url": "/posts/2026/06/rgb-s-paper-notes/",
      "date": "2026-06-19",
      "tags": ["Visuo-Tactile","Dexterous Manipulation","Tactile Sensing","Imitation Learning","Diffusion Policy","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  RGB-S makes tactile sensing look like something a standard visual encoder can already use: an image-space saliency map. Instead of feeding touch as a robot-centric vector and asking the policy to discover where each taxel belongs in the camera view, the method uses forward kinematics and camera calibration to project tactile sensor locations into the RGB image plane. Contact force is then rendered as a Gaussian saliency channel and concatenated with RGB:\\[X_t = \\mathrm{Concat}(I_t, S_t) \\in \\mathbb{R}^{H \\times W \\times 4}\\]  Here (I_t) is RGB and (S_t) is the tactile saliency map. The fourth channel is added to a pretrained ResNet-18 with zero initialization, so the encoder begins as an ordinary RGB encoder and gradually learns how projected touch should affect the visual representation. The central result appears under occlusion: in real-world dexterous manipulation, RGB-S reaches 51.7% average success, compared with 25.0% for the strongest implicit visuo-tactile baseline, a +26.7 percentage-point gain.  Paper Info  The paper is “RGB-S: Image-Aligned Tactile Saliency for Robust Dexterous Manipulation” by Shengcheng Luo, Kefei Wu, Xiaoying Zhou, Wanlin Li, Ziyuan Jiao, and Chenxi Xiao.  It appears on arXiv as arXiv:2606.08765, with v2 dated June 11, 2026. The project page is touch-as-saliency.github.io.  核心论点  Dexterous manipulation needs both broad scene context and direct evidence of contact. Vision provides the first, and pretrained image encoders make it cheap to reuse. Touch provides the second, especially when the hand or object hides task-relevant pixels. The hard part is alignment: RGB images live in a dense 2D coordinate system, while tactile readings are sparse, low-dimensional, and tied to a robot hand. If a policy only receives tactile features through concatenation, FiLM, or attention, it must learn the taxel-to-image correspondence from demonstrations.  RGB-S inserts a geometric prior before learning begins. Given robot proprioception, tactile sensor offsets, camera intrinsics, and camera extrinsics, each tactile node can be projected into the image. A contact becomes a visual cue near the image location where interaction is happening. Under occlusion, this matters because the tactile map can still mark where the robot is touching even when the corresponding RGB region is masked or unreliable.  Force-Aware Kinematic Projection  The method starts with tactile readings:\\[f_t = \\{f_{i,t}\\}_{i=1}^{M}\\]  where each (f_{i,t}) is a scalar force magnitude or contact intensity from tactile sensor node (i).  For each tactile node, the 3D world position is computed with forward kinematics:\\[P_{i,t} = \\mathrm{FK}(s_t, L_i)\\]  Here, (s_t) is robot proprioception, and (L_i) is the fixed local offset of the tactile sensor relative to its attached robot link.  Then, for camera view (c), the node is projected into the image plane:\\[[u^c_{i,t}, v^c_{i,t}, 1]^\\top\\simK_c(R_cP_{i,t}+t_c)\\]  where (K_c) is the camera intrinsic matrix, and (R_c, t_c) are extrinsics. Nodes outside image bounds are discarded.  The projected sparse contacts are rendered as a dense saliency map:\\[S^c_t(u,v)=\\max_{i \\in V^c_t}\\tilde{f}_{i,t}\\exp\\left(-\\frac{(u-u^c_{i,t})^2 + (v-v^c_{i,t})^2}{2\\sigma^2}\\right)\\]  The force is normalized with:\\[\\tilde{f}_{i,t} =\\tanh(\\gamma f_{i,t}/F^i_{limit})\\]  The Gaussian absorbs uncertainty from calibration, kinematics, and contact localization. Force modulation preserves more information than a binary contact map, and max aggregation keeps the saliency map bounded when multiple contacts overlap.  Network Architecture  After building the RGB-S input, the policy can use a standard visual stack. The ResNet-18 first convolution is expanded from three input channels to four:\\[z^c_t = W_{rgb} * I^c_t + W_s * S^c_t\\]  The RGB weights (W_{rgb}) come from pretrained ResNet-18, while the tactile weights (W_s) start at zero. This makes initialization conservative: the model initially behaves like the original RGB encoder, then fine-tuning learns how the saliency channel should modulate perception. The feature map is compressed with SpatialSoftmax into 32 keypoints, giving a 64-dimensional feature per camera view; multi-view features are concatenated with proprioception and passed to downstream policies. The same RGB-S representation is evaluated with Behavior Cloning MLP, ACT, and Diffusion Policy, so the contribution centers on image-aligned tactile representation across policy families.  Experiments  The simulation suite covers pick-and-place, cube-push, and rotate-cross. Policies are trained with normal vision and evaluated under both normal and occluded views, where black masks cover task-relevant image regions only at test time. Tactile readings come from an ETac-based simulator, and RGB-S maps are rendered for each camera view.  Across policy families and tasks, RGB-S is usually best or second-best. For Diffusion Policy, the average success rates are:                    Fusion        Pick-and-Place Avg        Cube-Push Avg        Rotate-Cross Avg                            Vision-only        39.7        60.9        52.0                    Concat        43.8        57.5        59.0                    FiLM        42.6        66.7        53.0                    CLiP-style        43.4        64.2        56.0                    Cross-Attn        38.4        59.2        61.0                    RGB-S        59.1        68.3        69.0              The improvement is most visible when RGB is degraded. For Diffusion Policy on pick-and-place, RGB-S reaches 39.7% success under occlusion while vision-only reaches 7.4%. On rotate-cross, RGB-S reaches 50.0% under occlusion, ahead of vision-only at 26.0%. The pattern is important: extra tactile input alone is not enough, since implicit fusion methods improve some settings and hurt others. RGB-S makes touch spatial before policy learning, so the visual encoder receives contact as a location-aware cue.  The real-world platform uses an xArm6 with a LEAP Hand, four fingertip TwinTac sensors, twelve FSR sensors, and 44 projected tactile nodes. Visual input comes from two calibrated RealSense D435 cameras, with EasyHEC used for camera extrinsics. Observations include proprioception, two RGB views, and tactile readings; actions are 22-dimensional targets for the arm and hand. Demonstrations are collected through VR teleoperation with Meta Quest 3 and Manus Quantum Metaglove. On pick-and-place, open-drawer, and flip-box, Diffusion Policy gives the following real-world results:                    Method        Normal Avg        Occluded Avg                            Vision-only        56.7        10.0                    Concat        55.0        13.3                    Cross-Attn        30.0        25.0                    RGB-S        66.7        51.7              The per-task occluded results show the same picture:                    Method        Pick & Place        Open Drawer        Flip Box                            Vision-only        0/20        4/20        2/20                    Concat        1/20        6/20        1/20                    Cross-Attn        0/20        4/20        11/20                    RGB-S        7/20        10/20        14/20              RGB-S improves occluded real-world average success by 26.7 percentage points over Cross-Attn, the strongest implicit baseline in this table. This is the central result: explicit image-space grounding of touch helps most when vision loses the task-relevant region.  Ablations  The ablations keep the same message tight. Under Diffusion Policy on pick-and-place, rendering contact as a force-aware saliency map beats binary contact and RGB overlay:                    Variant        Normal        Occluded        Average                            Vision-only        71.9        7.4        39.7                    RGB Overlay        65.3        33.1        49.2                    Binary RGB-S        65.3        27.3        46.3                    Force-aware RGB-S        78.5        39.7        59.1              Binary RGB-S already shows the value of contact location, while force-aware RGB-S adds interaction strength. Spatial alignment matters most under occlusion: with a 25 px random tactile-map offset, simulated occluded success drops from 39.7% to 32.2%; at 100 px, it falls to 9.9%. Normal vision is more tolerant because RGB still carries usable object information. Architecture also matters, with early zero-initialized fusion performing best:                    Architecture        Normal        Occluded                            Late fusion        73.6        35.5                    Intermediate fusion        73.6        22.3                    Early RGB-S        78.5        39.7              Injecting saliency at the first visual layer lets tactile information flow through the full visual hierarchy while keeping initializati..."
    },{
      "title": "Human Universal Grasping",
      "url": "/posts/2026/06/human-universal-grasping-paper-notes/",
      "date": "2026-06-19",
      "tags": ["Dexterous Grasping","Human Data","Robot Learning","Flow Matching","Sim-to-Real","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  Human Universal Grasping (HUG) is built around a direct scaling argument: human egocentric grasp data can become dexterous robot grasp supervision if every grasp is mapped into a canonical human hand space and then retargeted at deployment. The paper collects smart-glasses recordings of people grasping everyday objects, fits each terminal grasp to MANO, trains an RGB-D and point-conditioned flow-matching model, and retargets the predicted human grasp to robot hands.  The headline assets are 1M-HUGs, a 27.8-hour dataset with 1M egocentric grasp frames, 6,707 object instances, and 41 buildings; HUG, the flow-matching grasp model; and HUG-Bench, a 90-object benchmark with metric-scale meshes for paired simulation and real-world evaluation.  On the 30-object HUG-Bench test set, HUG reaches 73.0% success in MuJoCo simulation, 66.7% real-world tabletop success, and 62.0% in-the-wild success. In tabletop trials, it beats Dex1B by +23% and CAP by +34%.  Paper Info  The paper is “Human Universal Grasping” by Kevin Yuanbo Wu, Tianxing Zhou, Isaac Tu, Billy Yan, Irmak Guzey, David Fouhey, Dandan Shan, and Lerrel Pinto.  It appears on arXiv as arXiv:2606.17054, dated June 15, 2026. The project page is grasping.io, with code, data, benchmark, checkpoints, and an interactive demo.  Problem and Motivation  Dexterous grasping is still bottlenecked by data. Simulation can generate many grasps, but sim-to-real transfer for multi-fingered hands remains brittle; teleoperation collects real robot grasps, but it is slow and embodiment-specific. HUG shifts the source of supervision to ordinary human behavior. People already grasp thousands of objects in natural settings, and smart glasses now provide calibrated egocentric RGB-D, camera motion, and hand tracking for that behavior.  The core pipeline is compact: collect in-the-wild human grasps, fit them into a shared MANO hand space, learn a conditional distribution over human grasps from RGB-D observations and target points, then retarget the generated grasp to a robot hand. The paper’s central claim is that this route gives robot dexterity a scalable data source while keeping the learned representation independent of any single robot embodiment.  1M-HUGs Dataset  1M-HUGs is collected with Aria Gen 2 smart glasses. In each recording, the wearer first looks around a target object for 15-30 seconds while the hands stay out of view, then reaches in with the right hand and grasps the object. This protocol turns one physical grasp into many training pairs: the final grasp pose is propagated backward through camera poses and paired with earlier object-only RGB-D frames from different viewpoints.  Each curated training entry contains a 224 x 224 RGB or grayscale frame, intrinsics, metric depth, an object mask, and the terminal MANO hand pose plus wrist transform in the camera frame. The pipeline uses a vision-language model for object identification, SAM3 for mask propagation, heuristics for grasp-frame selection, and human review in a web annotation interface. After filtering, the dataset has 1M RGB frames and 1M grayscale stereo-left frames, roughly 2M training entries, from 6,707 recordings, 41 buildings, and about 1.5K unique objects.  MANO as the Common Hand Space  Aria provides sparse 21-hand-landmark tracking, so HUG optimizes a full MANO hand for each frame. MANO separates shape (\\beta), which controls hand size and proportions, from pose (\\theta), which controls joint articulation. HUG fixes shape to one canonical hand during training, so different collectors do not introduce different hand scales into the grasp target.  This is the representation choice that makes the whole system portable. The model learns wrist placement and articulated finger pose in a stable human hand coordinate system. The same canonical MANO hand can be exported to MuJoCo for simulation, and the predicted hand pose can later be retargeted to robot hands with different kinematics.  HUG Model  The model takes a single RGB-D observation and a 2D click ((u, v)) on the target object. Depth and camera intrinsics lift the click to a 3D query point (p_q), giving the model a compact way to specify which object or object part should be grasped.  The output is a 99-dimensional grasp state:\\[x = [t, R_{6d}, \\theta_{6d}] \\in \\mathbb{R}^{99}\\]  Here (t \\in \\mathbb{R}^3) is wrist translation in the camera frame, (R_{6d} \\in \\mathbb{R}^6) is wrist rotation in the continuous 6D representation, and (\\theta_{6d} \\in \\mathbb{R}^{15 \\times 6}) represents the 15 MANO finger joints.  The perception stack combines RGB semantics with local 3D geometry. A frozen DINOv2-Base with register tokens encodes the image, while a trainable PointNeXt U-Net encodes 4096 points cropped within 0.3 m of the query point. The two streams meet through point painting: point-cloud centroids are projected into the image, DINO features are sampled at those locations, and the concatenated RGB/3D features are refined by a transformer. The grasp generator is a flow-matching transformer that tokenizes translation, wrist rotation, and finger pose, conditions on the fused scene tokens, and integrates the learned velocity field with 50 Euler steps at inference.  Training Objective  The learning objective combines velocity prediction in normalized grasp space with geometric hand supervision. The paper reconstructs the predicted clean grasp, runs it through MANO, and applies an L1 loss to 3D hand landmarks in the camera frame:\\[L = \\lambda_v L_v + \\lambda_{3D}(1 - t)L_{3D}\\]  The weights are (\\lambda_v = 1) and (\\lambda_{3D} = 20). The ((1 - t)) factor emphasizes near-clean denoising steps, where the reconstructed hand is physically meaningful. This 3D loss is crucial: removing it drops HUG-Bench test success from 73.0% to 32.7% and raises fingertip contact error from 14.6 mm to 35.7 mm. The full model trains for 100K steps with AdamW, batch size 128, and two RTX 5090 GPUs, taking about 10 hours including MuJoCo validation.  HUG-Bench  HUG-Bench contains 90 unseen everyday objects, arranged by five geometric categories (cylindrical, spheroidal, prismatic, appendaged, and amorphous) and three size bins. Each category-size cell has four validation objects and two test objects, giving 30 test objects for real-world evaluation. The set is deliberately awkward: small and large items, handles, articulated structures, and objects that require structure-aware contact, including glue stick, pepper shaker, wine bottle, strawberry, football, storage bin, picnic basket, rubber duck, grapes, headphones, and easel.  The benchmark also contributes the simulation assets needed to evaluate dexterous grasps. The authors build metric meshes from short Aria recordings by extending Multi-view SAM3D with Aria intrinsics, extrinsics, and stereo depth, then manually align, make meshes watertight, and compute convex decompositions for MuJoCo. The released scan-to-asset pipeline is called aria2mesh.  Simulation Results  In MuJoCo, the canonical MANO hand executes an open-loop pre-grasp, grasp, and lift rollout. A grasp succeeds if the object is lifted away from the surface.  The main simulation results are:                    Method        Val SR        Test SR        Test fingertip contact error                            RGB + PC full HUG        71.5%        73.0%        14.6 mm                    without point cloud crop        61.2%        58.0%        25.7 mm                    without point painting        61.8%        58.3%        23.3 mm                    without 3D loss        39.2%        32.7%        35.7 mm                    PC only        64.2%        70.7%        22.1 mm                    RGB only        26.8%        29.7%        108.6 mm                    Human grasp oracle        90.3%        94.0%        7.4 mm              The ablations point to a useful division of labor. Point-cloud geometry carries the main spatial signal, RGB adds semantic grounding and improves fingertip placement, and the full RGB+PC model gives the best contact accuracy. The human grasp oracle exposes the remaining gap from tracking noise, asset imperfections, and open-loop execution. Scaling is equally important: from 25K to 1M RGB frames, test success rises from 33% to 73%, while fingertip contact error drops from 54.2 mm to 14.6 mm. The curve has not saturated at 1M, so the system still looks data-bound.  Real-World Results  The real-world evaluation uses the 30 HUG-Bench test objects, with 10 trials per object and 300 trials per method.  In tabletop experiments, HUG and Dex1B are deployed on a 6-DoF Ability hand mounted on a 7-DoF xArm, using a third-person ZED stereo camera. CAP uses its published parallel-jaw configuration with an iPhone wrist camera.                    Method        Overall tabletop success        Objects with at least one success                            Dex1B        43.7%      ..."
    },{
      "title": "Fast-WAM: Do World Action Models Need Test-time Future Imagination?",
      "url": "/posts/2026/06/fast-wam-paper-notes/",
      "date": "2026-06-19",
      "tags": ["World Action Models","Video World Models","Robot Learning","Vision-Language-Action","Flow Matching","Manipulation","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  Fast-WAM asks whether World Action Models need to generate future videos at test time. Its answer is practical: keep WAM-style video supervision during training, then run as a direct action policy at inference. The model uses a pretrained video DiT to form latent world representations from the current observation and language, and predicts action chunks without the expensive imagine-then-execute stage.  The main empirical message is that training-time video co-training matters more than test-time future imagination. Fast-WAM reaches 91.8% average success on RoboTwin 2.0 and 97.6% on LIBERO without embodied pretraining, while running at 190 ms latency on a single RTX 5090D V2 GPU. Variants that still generate future videos are close in success but much slower; removing video co-training causes a larger accuracy drop.  Paper Info  The paper is “Fast-WAM: Do World Action Models Need Test-time Future Imagination?” by Tianyuan Yuan, Zibin Dong, Yicheng Liu, and Hang Zhao from IIIS, Tsinghua University and Galaxea AI. It is available as arXiv:2603.16666, with a project page at yuantianyuan01.github.io/FastWAM and code at yuantianyuan01/FastWAM. The authors also release checkpoints and preprocessed LIBERO/RoboTwin datasets on Hugging Face.  Core Argument  Many WAMs follow an intuitive pipeline: observe the scene and language command, imagine future visual states, then predict actions conditioned on that imagined rollout. The intuition is attractive, but video diffusion is slow, and a robot policy ultimately needs low-latency closed-loop control.  Fast-WAM separates two roles that are often bundled together. Future video prediction can be a training objective that shapes the representation; future video generation can also be a test-time procedure used before action prediction. The paper’s controlled comparison shows that the first role carries most of the benefit on its benchmarks. In other words, video modeling is most useful as supervision for learning latent world structure, while inference can stay direct:  current observation + language  -> latent world representation  -> action chunk    This makes Fast-WAM look like a bridge between VLA policies and WAMs. At deployment time, it behaves like a direct action policy. During training, it still receives world-model supervision from future video latents.  Method  Fast-WAM is built on Wan2.2-5B, reusing the video Diffusion Transformer, pretrained T5 text encoder, and video VAE. On top of this video backbone, the authors add a 1B action expert DiT, giving the full model roughly 6B parameters. The model groups tokens into clean current-frame latents, noisy future-video latents used for training, and action tokens used for action chunk generation.  The most important implementation detail is the structured attention mask. During training, video tokens learn to predict future latent frames, and action tokens learn to predict actions from the clean current observation. Action tokens cannot attend to future video tokens, which prevents future-frame leakage and keeps the ablation fair: the action branch can benefit from a video-shaped backbone, but it cannot directly see ground-truth future frames. At inference, the future-video branch is removed; the current observation goes through the video backbone once, and the action expert denoises the action chunk.  The training objective is a joint flow-matching loss over action chunks and future video latents:  L_act = L_FM(action chunk)L_vid = L_FM(future video latents)L = L_act + lambda * L_vid    Reported implementation details include a hidden dimension of 1024 for the action expert, action horizon 32, future video horizon 9 frames after 4x temporal downsampling, multi-camera image concatenation before VAE encoding, 10 action denoising steps at inference, AdamW with learning rate 1e-4, mixed precision training, and gradient clipping. The official repository mirrors this structure through src/fastwam/, model and data configs for LIBERO/RoboTwin, evaluation managers under experiments/, scripts/train.py, DeepSpeed launch scripts, checkpoint evaluation instructions, ActionDiT preprocessing, and T5 embedding precomputation.  Controlled Variants  The paper tests the core claim by comparing variants that change whether video is used during training and whether future generation is used during inference:                    Variant        Training Video Co-Training        Test-Time Future Generation        Meaning                            Fast-WAM        Yes        No        Main method                    Fast-WAM-Joint        Yes        Yes        Joint video/action denoising                    Fast-WAM-IDM        Yes        Yes        Generate future video, then predict action                    Fast-WAM w/o video co-train        No        No        Tests whether video co-training matters              This design isolates the real question: does performance come from learning with video, or from explicitly imagining video during deployment?  Simulation Results  Fast-WAM is evaluated on RoboTwin 2.0 and LIBERO. On RoboTwin, it is close to pretrained LingBot-VA and clearly above non-pretrained baselines:                    Method        Embodied Pretraining        Average Success                            π0        Yes        62.2                    π0.5        Yes        79.8                    Motus        Yes        87.8                    Motus from Wan2.2        No        77.3                    LingBot-VA        Yes        92.2                    LingBot-VA from Wan2.2        No        80.6                    Fast-WAM        No        91.8              On LIBERO, Fast-WAM remains competitive with strong WAM/VLA baselines even though it does not rely on embodied pretraining:                    Method        Embodied Pretraining        Average Success                            OpenVLA        Yes        76.5                    π0        Yes        94.1                    π0.5        Yes        96.9                    LingBot-VA        Yes        98.5                    Motus        Yes        97.7                    Fast-WAM        No        97.6              The ablations are the heart of the paper. Fast-WAM stays close to variants that perform explicit future imagination, while removing video co-training causes a much larger drop:                    Variant        RoboTwin Average Success        LIBERO Average Success                            Fast-WAM        91.8        97.6                    Fast-WAM-Joint        90.6        98.5                    Fast-WAM-IDM        91.3        98.0                    Fast-WAM w/o video co-train        83.8        93.5              Real-World Towel Folding  The real-world experiment uses a Galaxea R1 Lite platform on a long-horizon towel-folding task. The authors collect 60 hours of teleoperated demonstrations and train for 30k steps. The result is useful because it measures both success and completion time: a policy that eventually succeeds after repeated corrections may still be weak for deployment.  The real-world pattern matches the simulation story. Pretrained π0.5 remains the strongest baseline, but Fast-WAM variants with video co-training substantially outperform π0.5 without pretraining, and Fast-WAM without video co-training drops to 10% success. The latency comparison is also decisive: Fast-WAM runs at 190 ms, compared with about 580 ms for Fast-WAM-Joint and 810 ms for Fast-WAM-IDM. Within this model family, direct Fast-WAM gives the best deployment tradeoff: strong success with far lower inference latency.  Strengths and Limitations  The strongest part of the paper is the clean experimental question. Instead of making model scale the main story, it asks where the benefit of WAMs actually comes from and answers with controlled variants. The released code, checkpoints, and preprocessing/evaluation instructions also make the result easier to inspect and reuse.  There are still important boundaries. The conclusion is tested at a specific scale, with Wan2.2-5B plus a 1B action expert; larger video backbones, larger embodied datasets, or different action decoders may shift the tradeoff. Fast-WAM also still uses diffusion-style action denoising, so 190 ms is fast for a WAM comparison but still meaningful for high-frequency contact-rich control. The real-world evaluation focuses on towel folding on one robot platform, leaving open how well the conclusion transfers across rigid, articulated, deformable, and more contact-heavy manipulation. Finally, skipping future video generation removes an interpretable visual rollout that could help debugging, planning, or human inspection.  Takeaway  Fast-WAM’s takeaway is concise: for WAMs, future video prediction may be more valuable as a training objective than as a test-time procedure. This gives robot learning a useful design path: use video world mod..."
    },{
      "title": "DexJoCo: A Benchmark and Toolkit for Task-Oriented Dexterous Manipulation on MuJoCo",
      "url": "/posts/2026/06/dexjoco-paper-notes/",
      "date": "2026-06-18",
      "tags": ["Dexterous Manipulation","MuJoCo","Benchmark","Imitation Learning","Vision-Language-Action","Robot Learning","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  DexJoCo is best read as an integrated benchmark and workflow for task-oriented dexterous manipulation. Its contribution is the full stack around policy learning: MuJoCo environments, a Franka Panda + Allegro Hand robot setup, low-cost human teleoperation, replayable demonstrations, LeRobot/Zarr data conversion, policy training and evaluation, and robustness tests under visual and dynamics randomization.  The paper builds 11 functional tasks and collects 1.1K human demonstration trajectories. The tasks cover tool use, button pressing, folding, grasping with tongs, watering, peg insertion, articulated-object interaction, bimanual sequencing, and language-conditioned reasoning. My main takeaway is that DexJoCo turns dexterous manipulation from a set of isolated simulator puzzles into a reproducible robot-learning pipeline, while also showing how fragile current imitation-learning and VLA policies remain once fine contact, hand coordination, temporal memory, and visual generalization matter.  Paper and Resources  The paper is “DexJoCo: A Benchmark and Toolkit for Task-Oriented Dexterous Manipulation on MuJoCo” by Hanwen Wang, Weizhi Zhao, Xiangyu Wang, Siyuan Huang, He Lin, Boyuan Zheng, Rongtao Xu, Gang Wang, Yao Mu, He Wang, Lue Fan, Hongsheng Li, Zhaoxiang Zhang, and Tieniu Tan. It is available as arXiv:2605.16257, with a project page at dexjoco.github.io, code at brave-eai/dexjoco, and LeRobot-format data on Hugging Face at DexJoCo/DexJoCo-Datasets-LeRobot.  核心论点  Many dexterous manipulation benchmarks emphasize hand-only control, in-hand manipulation, or simple pick-and-place. DexJoCo argues for a more task-oriented setting where the full arm-hand system must produce a functional outcome: a nail is hammered, a mouse is clicked, glasses are folded, a plant is watered, a peg is inserted, an iPad is unlocked, or a camera shutter is pressed. This design makes success depend on sequence, pose, articulated state, and contact:  G = {g_seq, g_pose, g_joint, g_contact}    That framing is important because dexterity is evaluated through what the hand accomplishes, with visual plausibility treated as insufficient. The benchmark includes single-arm and bimanual tasks around a MuJoCo model of a Rethink mount, Franka Panda arm, and Allegro Hand. Observations include third-person and wrist RGB/RGB-D views, object poses, robot states, end-effector pose, and hand joint angles; demonstrations use target absolute end-effector poses plus target absolute hand joint angles.  The task suite is broad enough to stress different failure surfaces without turning the post into a catalog: Hammer Nail and Water Plant test tool use, Click Mouse and Photograph test precise button interaction, Pinch Tongs tests finger coordination, Fold Glasses and Microwave Cook test articulated objects, Assembly and Hanoi test alignment and sequencing, Pick Bucket tests long-horizon object handling, and Unlock iPad adds language-conditioned reasoning to bimanual control.  Teleoperation and Data Pipeline  A strong part of DexJoCo is that the benchmark comes with a practical data path. The authors use a roughly $2,300 USD teleoperation setup built from Rokoko Smartgloves, HTC Vive Trackers, HTC Base Stations, and a 3D-printed connector. The glove captures hand motion while avoiding camera-occlusion issues common in vision-only hand tracking; the wrist tracker drives the Franka end-effector. Human hand motion is retargeted to the Allegro Hand with GeoRT, using a self-supervised objective over fingertip direction, workspace coverage, sensitivity, pinch behavior, and self-collision:  L = L_dir + λ1 L_cover + λ2 L_flat + λ3 L_pinch + λ4 L_col    The released repository mirrors this end-to-end story. dexjoco/ contains the MuJoCo environments and task wrappers, teleoperation/ documents the Vive/Rokoko/GeoRT hardware workflow, scripts/record_demos_zarr.py and scripts/replay_demos_zarr.py support recording and replay, dexjoco-data-converter/ converts demonstrations into LeRobot datasets and Diffusion Policy-style Zarr buffers, openpi/ supports π0.5 training and evaluation, and docs/custom_policy_integration.md describes the observation/action contract for custom policies.  One useful engineering detail is the OpenPI-style server-client evaluation pattern. The policy server emits action chunks, while the DexJoCo evaluation client buffers and executes those actions in simulation, requesting the next plan before the buffer runs dry. That is closer to deployed policy execution than a purely synchronous one-step inference loop.                    Setup        Policy Action        Environment Action                            Single-arm        22D [xyz, rotvec, hand16]        23D [xyz, quat, hand16]                    Bimanual        44D [r_xyz, r_rotvec, r_hand16, l_xyz, l_rotvec, l_hand16]        46D quaternion layout              The state logs include privileged environment information for replay, but policy training should use only robot proprioception: the first 23 dimensions for single-arm tasks and the first 46 dimensions for bimanual tasks.  Robustness and Evaluation  DexJoCo evaluates ACT, Diffusion Policy Transformer (DP-T), Diffusion Policy CNN (DP-C), π0.5, and GR00T N1.5. ACT and Diffusion Policy are trained from scratch with vision and proprioception; π0.5 and GR00T N1.5 are LoRA fine-tuned and condition on language. Because default VLA action heads do not directly match bimanual dexterous action dimensions, the authors adapt the heads, including partial reinitialization for extra dimensions.  The robustness design is compact but revealing. rand-obj randomizes object placement and table height. rand-full adds third-person camera pose, lighting direction/color, and tabletop texture randomization. The replay system lets users apply visual randomization by replaying the same trajectories under different rendering settings, and the code also exposes --randomize-dynamics for parameters such as joint friction, stiffness, and object mass.                    Model        rand-obj Avg. Success        rand-full Avg. Success                            DP-T        50.4%        20.0%                    DP-C        47.6%        28.4%                    ACT        35.5%        22.7%                    π0.5        52.5%        34.1%                    GR00T N1.5        40.2%        30.5%              The table gives the main result in one glance: π0.5 has the strongest average success, but all methods degrade sharply under fuller visual randomization. Smaller Diffusion Policy variants remain competitive on several settings, which suggests that dexterous manipulation still depends heavily on action representation, temporal memory, and contact-level control instead of scaling vision-language pretraining alone.  Limitations  The failure modes are the most useful part of the benchmark. Policies can look semantically correct while failing physically: they pick up a camera but miss the shutter, reach a button but fail to press it, align near a peg but miss insertion, or start a bimanual sequence and lose timing. Pinch Tongs exposes repeated open-close memory, Assembly and Hanoi expose precise alignment, and several bimanual tasks show how quickly action dimensionality and asymmetric hand roles become bottlenecks.  The benchmark also inherits limits from simulation and sensing. Vision-only policies lack force and tactile cues for contact-rich manipulation. Current VLA models are still mostly pretrained on gripper-heavy robot data, so high-DoF hand action heads require adaptation and can remain brittle. Domain randomization improves coverage, but sim-to-real transfer will still need stronger physical, visual, and sensing fidelity. The iPad password setting also hints that language grounding can collapse into action bias when instructions require arithmetic or paraphrased reasoning.  Takeaway  DexJoCo is most valuable as infrastructure: it packages functional dexterous tasks, accessible teleoperation, replayable demonstrations, data conversion, policy integration, and robustness evaluation into one benchmark pipeline. For research, it is a good place to test whether a method can actually complete contact-rich functional interactions beyond plausible arm-hand motion. For practice, the code release matters because the benchmark can be extended, replayed, converted, and evaluated with modern imitation-learning and VLA tooling.  本文支持通过顶部导航栏的语言切换按钮在 English / 中文 之间切换。  TL;DR  DexJoCo 更适合被理解为一个面向 task-oriented dexterous manipulation 的 integrated benchmark 和 workflow。它的贡献集中在 policy learning 周围的完整栈：MuJoCo environments、Franka Panda + Allegro Hand 机器人设置、低成本 human teleoperation、可 replay 的 demonstrations、LeRobot/Zarr 数据转换、policy training/evaluation，以及 visual 和 dynamics randomization 下的鲁棒性测试。  论文构建了 11 个功能性任务，并收集了 1.1K 条 human demonstration trajectories。这些任务覆盖工具使用、按按钮、折叠、夹钳操作、浇水、peg insertion、articulated-obj..."
    },{
      "title": "Tac2Real: Reliable and GPU Visuotactile Simulation for Online Reinforcement Learning and Zero-Shot Real-World Deployment",
      "url": "/posts/2026/06/tac2real-paper-notes/",
      "date": "2026-06-17",
      "tags": ["Visuotactile Simulation","Tactile Sensing","Reinforcement Learning","Sim-to-Real","Robot Manipulation","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  Tac2Real is a tactile simulation and sim-to-real recipe for contact-rich robot learning. The paper argues that online RL with vision-based tactile sensors needs two things at the same time: a physically meaningful tactile simulator and enough throughput to run many parallel environments. Tac2Real tries to sit exactly at that tradeoff point.  The simulator uses PNCG-IPC, a GPU-friendly variant of Incremental Potential Contact, to generate GelSight-style marker displacement fields. Instead of rendering full tactile RGB images, it outputs a low-dimensional 9 x 7 marker displacement field, which is more sensitive to contact modes such as pressing, sliding, and collision direction, while being easier to use in RL.  The second contribution is TacAlign, a four-stage alignment pipeline for zero-shot deployment: robot controller alignment, baseline IPC material calibration, task-based contact calibration, and domain randomization. This part is the real bridge from high-fidelity simulation to real-world success.  On blind peg insertion, Tac2Real reaches 91.7% zero-shot real-world success over 60 trials. In the same real-world setting, TacSL reaches 15.0%, Tacchi reaches 8.3%, and no tactile feedback reaches 6.7%. The key lesson is that simulation success alone is not enough: TacSL performs similarly to Tac2Real in simulation, but collapses in real deployment because its tactile fields are less physically aligned with reality.  Paper Info  The paper is “Tac2Real: Reliable and GPU Visuotactile Simulation for Online Reinforcement Learning and Zero-Shot Real-World Deployment” by Ningyu Yan, Shuai Wang, Xing Shen, Hui Wang, Hanqing Wang, Yang Xiang, and Jiangmiao Pang.  It was submitted to arXiv on March 30, 2026 as arXiv:2603.28475. The project page is ningyurichard.github.io/tac2real-project-page, and it links to the official code.  Problem and Motivation  Vision-based tactile sensors such as GelSight Mini are powerful because they convert local contact into dense visual or marker signals. For manipulation, this is especially useful when object pose is hidden, camera feedback is ambiguous, or the task becomes dominated by subtle contact geometry.  The difficulty is that tactile simulation is caught between two bad extremes:      Fast approximations can scale to online RL, but often miss deformation, friction, slip, and realistic contact modes.    High-fidelity physics methods can model soft contact better, but are usually too slow or unstable for thousands of online RL environments.    Tac2Real targets the middle ground: physically grounded enough to transfer, but lightweight and parallel enough to train policies online.  The paper positions existing methods along this axis. Tacchi uses MPM and can be visually plausible, but struggles with numerical instability and adhesion-like artifacts under large deformation. TacSL is very fast because it uses SDF and penalty-based contact, but its tactile field can deviate from real marker displacement. IPC-based methods are more robust for contact, but need careful acceleration to be useful in RL.  Tac2Real Simulation Framework  Tac2Real uses Preconditioned Nonlinear Conjugate Gradient Incremental Potential Contact (PNCG-IPC) for the tactile sensor gel. IPC formulates elastodynamic contact as an optimization problem. With implicit Euler integration, the next positions are obtained by minimizing an energy:\\[E(x) =\\frac{1}{2}(x - \\hat{x})^\\top M(x - \\hat{x})+ h^2 \\Psi(x)+ B(x)+ D(x)\\]  The terms correspond to inertia, hyperelastic energy, log-barrier contact potential, and frictional potential.  Standard IPC often uses Newton-style optimization with Hessian assembly, factorization, and continuous collision detection based line search. That is accurate but expensive. PNCG-IPC replaces Newton steps with nonlinear conjugate gradient updates. The computation mainly needs gradients, diagonal Hessian entries, and vector dot products, which map well to GPUs. It also uses an analytical step-size bound to avoid costly CCD in each line search.  This is a deliberate engineering tradeoff. Each iteration is less accurate than Newton’s method, but the iteration is cheap enough that the solver can reach sufficient tactile accuracy at interactive speed.  Why Marker Displacement Fields?  Tac2Real focuses on GelSight Mini marker displacement fields rather than tactile RGB images. A GelSight Mini can produce either RGB tactile images or a marker displacement field depending on gel type. The paper’s test shows that marker displacement fields change clearly across stationary, press-down, move-forward, and move-backward contact modes, while RGB tactile images show subtler changes.  For RL, this matters for three reasons:      The representation is compact: 9 x 7 x 2 marker displacement values.    It directly exposes contact direction and shear-like deformation cues.    It avoids the cost and uncertainty of optical rendering.    In simulation, marker positions are mapped to initial IPC mesh nodes using k-nearest neighbors, then interpolated from the deformed mesh state.  Integration with Robot Simulators  Tac2Real is designed as a plugin outside the main physics engine. The robot simulator, such as Isaac Lab, advances the robot and object dynamics. Tac2Real receives relative quantities between the tactile sensor and the contacted object:      relative position,    relative rotation,    relative linear velocity,    relative angular velocity.    It then runs tactile simulation and returns marker displacement fields. These tactile fields are concatenated with the base robot observation and passed back to the RL policy.  This interface is important because it makes Tac2Real cross-engine compatible. The tactile backend only needs relative sensor-object quantities, so the same idea can be attached to Isaac Lab, Isaac Gym, MuJoCo, PyBullet, or similar environments.  For throughput, Tac2Real uses a Ray cluster over multiple nodes and GPUs. Each GPU owns a Ray-wrapped tactile simulation worker responsible for a subset of environments. During rollout, tactile fields are computed in parallel and gathered for policy training.  TacAlign: Closing the Gap in Layers  Tac2Real’s strongest practical idea is that simulation fidelity alone is not enough. The paper separates the sim-to-real gap into structured and stochastic parts, then introduces TacAlign to attack both.  1. Robot Control Alignment  Both simulated and real Franka robots use Cartesian impedance control. A naive approach would try to match controller gains directly, but the paper shows that similar gains do not necessarily imply similar end-effector trajectories because of actuator delays, friction, and unmodeled dynamics.  Instead, TacAlign minimizes trajectory discrepancy over six canonical motions: three translations and three rotations. It alternates between optimizing simulation gains and real-controller gains. The initial average translational discrepancy is 11.11 mm, which is already larger than the roughly 8 mm socket hole in the peg insertion task. After alignment, the discrepancy drops to 2.521 mm translation and 0.454 degrees rotation.  2. Baseline IPC Calibration  The tactile gel’s material parameters are calibrated against real GelSight Mini measurements. The parameters are:      Young’s modulus (E),    Poisson’s ratio (\\nu),    density (\\rho),    friction coefficient (\\mu).    The authors use a 6-DOF positioning stage and four 3D-printed indenters: cube, cylinder, moon, and triangle. Each indenter performs pressing, sliding, and rotating interactions. The objective is the MSE between simulated and real marker displacement fields, optimized with CMA-ES.  3. Task-Based Calibration  Baseline indentation is not enough for the actual peg insertion task. TacAlign also fine-tunes Isaac Lab contact parameters, especially contact friction and compliant contact settings, using task-relevant states:      stationary grasping,    press-down collision,    forward collision,    backward collision.    This stage turns out to be especially important in ablation. Removing task-based calibration drops Tac2Real’s real-world success rate from 91.7% to 25.0%.  4. Randomization  Finally, TacAlign adds randomization for residual uncertainty. It randomizes controller gains, friction, socket position, object pose, hand pose, end-effector pose noise, and IPC movement perturbations. This complements deterministic calibration by making the policy robust to errors that cannot be cleanly identified.  Online RL Setup  The evaluation uses two contact-rich simulation tasks:      Random Orientation Peg Insertion. A Franka robot inserts a cylindrical peg into a socket. The peg orientation in the gripper is randomized within ([-35^\\circ, 35^\\circ]). The peg and socket hole diameter are both 8 mm.    Random Orientation Nut Threading. The robot places a randomly oriented nut onto a bolt and rotates th..."
    },{
      "title": "ARM: Advantage Reward Modeling for Long-Horizon Manipulation",
      "url": "/posts/2026/06/arm-paper-notes/",
      "date": "2026-06-17",
      "tags": ["Reward Model","Robot Learning","Long-Horizon Manipulation","VLA","Behavior Cloning","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  ARM trains a reward model for long-horizon manipulation, but the important move is subtle: it avoids asking humans or VLMs to assign an absolute progress score to every frame. Instead, it asks a simpler relative question: over a short interval, did the robot make progress, regress, or stay stagnant?  This gives the paper its core primitive:\\[y \\in \\{-1, 0, +1\\}\\]  where (+1) means Progressing, (0) means Stagnant, and (-1) means Regressing. A MIMO temporal transformer then turns these tri-state advantage labels into dense progress curves. Those curves are used to weight imitation learning data through Advantage-Weighted Behavior Cloning (AW-BC).  The result is strong on a real long-horizon towel-folding task. Standard BC with GR00T-N1.5 reaches 62.1% success. RA-BC with SARM reaches 78.5%. AW-BC with ARM reaches 99.4%, with better throughput and folding precision. My read: ARM is less about a new policy architecture and more about a scalable reward-supervision interface for messy, non-monotonic robot demonstrations.  Paper Info  The paper is “ARM: Advantage Reward Modeling for Long-Horizon Manipulation” by Yiming Mao, Zixi Yu, Weixin Mao, Yinhao Li, Qirui Hu, Zihan Lan, Minzhao Zhu, and Hua Chen.  It appears on arXiv as arXiv:2604.03037, with v2 dated April 21, 2026. The project page is aiming1998.github.io/ARM.  Problem and Motivation  Long-horizon manipulation is hard for reinforcement learning because sparse rewards are too thin for credit assignment. A binary success signal at the end of a 120-second towel-folding episode does not tell the policy which grasp, pull, fold, recovery, or placement mattered.  Dense progress rewards are the usual answer, but they are expensive and brittle. A human annotator can label subtask boundaries, but this requires careful temporal localization. A VLM can be prompted to segment videos, but it is noisy, slow, and often lacks the geometric grounding needed for fine robot state changes. A scalar progress score also assumes something that is often false: progress should monotonically increase over time.  Real robot demonstrations contain backtracking, corrections, pauses, and recovery motions. In towel folding, for example, a local adjustment can temporarily move the towel away from the final shape while still being necessary for the next fold. A progress model that only knows “later means better” will punish exactly the kind of recovery behavior that long-horizon manipulation needs.  ARM reframes the reward-labeling problem around relative advantage. Instead of asking “how complete is this frame?”, it asks “did this interval improve the state relative to the recent past?” That makes the annotation more local, more robust to non-monotonic behavior, and easier to scale.  Core Idea: Tri-State Advantage Labels  The labeling scheme has three classes:      Progressing (+1): the state advances toward the task goal.    Regressing (-1): the state deviates from the goal, hits an error, or moves toward failure.    Stagnant (0): no meaningful progress is made, such as waiting or idle motion.    This is much easier than asking for frame-level normalized progress (P \\in [0, 1]). Humans do not need to decide whether a towel is 0.43 or 0.47 complete. They only need to judge the local direction of change.  The paper also uses this tri-state supervision as a cold start. After the initial human labels, ARM can run over large unlabeled trajectory datasets and generate pseudo-labels automatically. This is where the approach becomes more scalable than manual subtask segmentation.  Advantage Reward Model Architecture  ARM is a multimodal temporal reward model. For each timestep, it combines:      CLIP visual features from video frames,    robot proprioceptive state,    task instruction text.    Each input is projected into a shared latent space:\\[x_i = \\mathrm{MLP}(v_i) + \\mathrm{MLP}(s_i) + \\mathrm{MLP}(g)\\]  where (v_i) is the visual feature, (s_i) is the robot state, and (g) is the language task embedding.  The model uses an 8-layer Transformer encoder over a causal window:\\[W_t = \\{o_{t-4k}, \\dots, o_t\\}\\]  The key architectural choice is MIMO: Multi-Input Multi-Output. Instead of producing one scalar progress estimate for one window, ARM predicts multiple interval advantage labels in one forward pass. This lets the model share temporal context across adjacent predictions and reduces redundant sliding-window inference.  ARM has two heads:      Multi-frame advantage classification head. Predicts tri-state interval transitions (\\Delta \\hat{y}) between consecutive hidden states.    Task completion head. Predicts whether a state is a successful terminal state.    The total loss is:\\[\\mathcal{L}_{ARM} = \\lambda_{int}\\mathcal{L}_{int} + \\lambda_{succ}\\mathcal{L}_{succ}\\]  The interval loss uses cross entropy over tri-state labels. The completion loss uses focal loss, because successful terminal states are rare in long continuous trajectories.  Global Progress Reconstruction  Tri-state labels are local. Policy learning, however, benefits from a dense progress signal over the whole episode. ARM reconstructs such a curve in three steps.  First, the MIMO model runs over clipped video segments in parallel, predicting interval advantages efficiently.  Second, segments are aligned and padded when needed, with synthetic padding ignored during final aggregation.  Third, the predicted relative transitions are integrated into a global progress curve (P_t), anchored by the task completion head. A successful terminal frame provides the absolute anchor, such as (P_T = 1.0), while earlier frames are reconstructed from accumulated predicted gains.  This turns local labels such as “progressing” or “regressing” into a dense reward-like signal that can detect dips, pauses, and recoveries instead of forcing a staircase-shaped subtask curve.  AW-BC: Using ARM for Policy Improvement  The downstream policy training method is Advantage-Weighted Behavior Cloning (AW-BC). The idea is simple: use ARM’s reconstructed progress gains to weight action chunks. Good transitions get high weight; regressive or low-value transitions are suppressed.  For an action chunk with horizon (H), ARM defines a length-adaptive gain:\\[\\Delta G_t = (P_{t+H} - P_t) \\cdot \\frac{L_{seq}}{\\bar{L}}\\]  Here, (P_t) is reconstructed progress, (L_{seq}) is the current episode length, and (\\bar{L}) is the average episode length. This normalization helps avoid a bias where short episodes get artificially steep progress slopes.  Weights are computed from the batch gain distribution. If (\\mu) and (\\sigma) are the mean and standard deviation of gains, the paper clips gains between (\\mu - 2\\sigma) and (\\mu + 2\\sigma), then maps them into ([0, 1]):\\[\\tilde{w}_i =\\mathrm{clamp}\\left(\\frac{\\Delta G_i - b_{lower}}{b_{upper} - b_{lower} + \\epsilon},0,1\\right)\\]  The weighted BC objective is:\\[\\mathcal{L}_{AW-BC}(\\theta)=\\mathbb{E}_{(s,a)\\sim\\mathcal{D}}[-\\tilde{w}(s,a)\\log \\pi_\\theta(a|s)]\\]  This is offline policy improvement in the spirit of advantage-weighted regression: stay close to the data, but prioritize the parts of the data that actually move the task forward.  Task and Dataset  The main benchmark is a real-world bimanual towel-folding task using an AgileX ALOHA-style teleoperation setup. A successful episode requires an 8-stage sequence:      extract exactly one towel from a cluttered pile;    place it on the central tabletop;    flatten it into a planar initial state;    perform a bottom-to-up longitudinal fold;    perform a top-to-bottom longitudinal fold;    perform a right-to-center lateral fold;    perform a left-to-right lateral fold;    transport and deposit the folded towel into a target box.    The trial must complete within 120 seconds, with a single towel neatly folded and fully inside the box.  The dataset contains 972 towel-folding episodes, about 20 hours total:      809 expert demonstrations,    163 DAgger-augmented error-correction episodes.    Unlike approaches that discard slow or messy trajectories, ARM keeps them because they contain valuable recovery behavior.  The real robot setup uses three RGB views: a high global view plus left and right wrist cameras. The proprioceptive state and action are both 14-dimensional, covering bimanual joint positions and gripper states.  Reward Model Results  ARM is compared with SARM on reward reconstruction and terminal success identification.                    Metric        SARM        ARM                            Progress reconstruction MSE ↓        0.0059        0.0014                    Standard episode terminal ID        83.3% (10/12)        100.0% (12/12)                    Failure episode terminal ID        91.6% (11/12)        100.0% (12/12)              Qualitatively, SARM produces stepped progress curves around subtask boundaries. ARM produces smoother dense curves and captures temporary downward dips du..."
    },{
      "title": "SynManDex: Synthesizing Human-like Dexterous Grasps",
      "url": "/posts/2026/06/synmandex-paper-notes/",
      "date": "2026-06-16",
      "tags": ["Dexterous Manipulation","Robot Learning","Human Priors","Synthetic Data","Grasp Synthesis","Paper Notes"],
      "excerpt": "",
      "text": "TL;DR  SynManDex is a synthetic-data pipeline for bimanual dexterous grasping. Its core argument is simple and useful: let human priors propose where a functional grasp should live, then let the robot embodiment decide whether the grasp can be contacted, reached, lifted, and used for policy learning.  The paper uses generated digital-human pre-grasps as affordance-aware proposals. These proposals encode approach direction, wrist orientation, and coarse finger coordination, while robot-native modules retarget them to XHand, refine contacts with force-closure optimization, check arm-hand IK, and admit only demonstrations that survive a lift rollout. The important shift is from human-like pose imitation to executable grounding.  The results make the staging decision credible: 86.4% force-closure success on a 312-object, 25-class grasp-quality manifest, 4.67/5 combined human-likeness, 65.8% lift-admitted trajectory rate, 80.7% held-out simulated policy success, and 25/30 real-robot successes on a 36-DoF bimanual UR5e-XHand platform.  Paper Info  The paper is “SynManDex: Synthesizing Human-like Dexterous Grasps from Synthetic Human Pre-Grasps” by Yanming Shao, Zanxin Chen, Wenwei Lin, Mingjie Zhou, Tianxing Chen, Xiaokang Yang, Yichen Chi, and Yao Mu. The arXiv version is 2606.09798, submitted as v1 on June 8, 2026. The project page is tsunami-kun.github.io/SynManDex.  Core Argument  Dexterous grasping is hard because functional plausibility and robot executability are different filters. A person holding a camera, flute, bottle, teapot, or phone chooses contacts that preserve use: a handle stays accessible, a lens points outward, one hand stabilizes while the other can release or reposition fingers. A robot hand then faces its own geometry, joint limits, palm shape, collision model, actuation, and arm reachability. A MANO-like human pose can look plausible while missing load-bearing contacts, penetrating the object, or putting the wrist outside the robot’s reachable set.  SynManDex places the human prior at the proposal stage. The generated human pre-grasp suggests a functional search basin; the robot pipeline resolves the final contact state and rejects samples that fail physical or kinematic checks. I find this the most important design choice in the paper, because it treats human data as a guide to intent while keeping validity tests on the target embodiment.  The whole pipeline can be written compactly as:\\[h_0 \\sim p_\\theta(h \\mid M)\\]\\[q_{init} = R_\\psi(h_0, M)\\]\\[q^\\star = \\Pi_{phys}(q_{init}, M)\\]\\[\\tau = \\Pi_{exec}(q^\\star, M)\\]  Here (M) is the object mesh, (h_0) is a generated digital-human pre-grasp, (q_{init}) is the retargeted robot seed, (q^\\star) is the robot-grounded keyframe, and (\\tau) is the admitted executable trajectory. The admission gate is the real dataset boundary:\\[A(\\tau) = A_{coll} \\wedge A_{FC} \\wedge A_{IK} \\wedge A_{lift}\\]  A sample enters the dataset only after collision, force-closure, inverse-kinematics, and lift checks succeed. This gate is what turns a visually plausible human-inspired grasp into a robot demonstration.  Method  SynManDex-Human is an object-conditioned diffusion model trained on hand-object interaction resources such as GRAB and ContactPose. Instead of generating the final closed grasp, it generates a single pre-contact frame. For temporal human-object sequences, the authors locate the first contact frame by minimum hand-object distance and supervise the frame 0.2 seconds earlier; static grasps provide pose priors. This pre-contact choice matters because it gives the robot an approach and role assignment without forcing it to reproduce human contact geometry.  The diffusion objective follows the usual DDPM form:\\[L_{diff} =\\mathbb{E}_{t,h_0,\\epsilon}\\left[\\left\\|\\epsilon -\\epsilon_\\theta(\\sqrt{\\bar{\\alpha}_t}h_0 + \\sqrt{1-\\bar{\\alpha}_t}\\epsilon, t, M)\\right\\|_2^2\\right]\\]  After diffusion sampling, the system has a digital-human hand seed (X^H) with 21 hand keypoints. SynManDex retargets it to an open XHand pre-grasp, preserving motion direction, coverage, local flatness, pinch relations, and self-collision margins:\\[L_{GeoRT} =\\lambda_{dir}L_{dir}+ \\lambda_{cov}L_{cov}+ \\lambda_{flat}L_{flat}+ \\lambda_{pinch}L_{pinch}+ \\lambda_{self}L_{self}\\]  It then solves for wrist pose and joints:\\[(\\theta_0, T_0) =\\arg\\min_{\\theta,T}\\sum_i \\|\\bar{x}^H_i - T x^R_i(\\theta)\\|_2^2+ \\lambda_\\psi \\|\\theta - g_\\psi(X^H)\\|_2^2+ \\lambda_{self}L_{self}(\\theta)\\]  This retargeted seed is deliberately incomplete. It serves as the starting point for a robot-native module that refines the wrist-hand configuration:\\[q^\\star =\\arg\\min_qw_c C_{coll}(q)+ w_f L_{FC}(q)+ w_r \\|q - q_{init}\\|^2\\]  The force-closure score is based on a discretized friction-cone wrench margin:\\[Q_{FC}(q) =\\min_{\\|w\\|_2=1}\\max_{f \\in F(q), \\|f\\|_1 \\le 1}w^\\top G(q)f\\]  The paper uses this score as an admission signal, with simulation rollout still serving as the execution check. That separation is important: a discretized contact model can guide contact search, while lift rollout tests whether the grasp survives the dynamics the policy will later imitate.  The ablation table captures the method’s main claim:                    Method        G1        Pen. mm        Contact        FC        Combined human-likeness        PCD                            SynManDex full        7.2        0.6        89.2%        86.4%        4.67        0.41                    Optimization-only        4.6        0.67        71.6%        79.1%        2.81        0.11                    Retarget-only        0.4        8.3        34.7%        12.3%        4.18        0.09              Retarget-only preserves the human silhouette and loses contact quality. Optimization-only improves contact and stability while reducing human-likeness. The full pipeline keeps the human-functional basin and adds robot-grounded contact refinement.  Grounded floating-hand grasps still need arms. SynManDex checks arm-hand reachability with cuRobo and rolls out approach, closure, squeeze, and lift phases in simulation. A trajectory is admitted only if it passes a vertical lift test:\\[y =\\mathbf{1}\\left[\\max_{t \\ge t_{lift}}(z_t - z_0) > \\tau_z\\right]\\]  Under a fixed 240 candidates per object budget, the funnel looks like this:                    Stage        Pass signal                            Grounded keyframes        86.4% force-closure among optimized XHand candidates                    IK-valid trajectories        82.3% IK-valid among grounded candidates                    Lift-admitted demonstrations        65.8% lift-admitted among grounded candidates              This is where SynManDex becomes more than a static grasp generator. The admitted rollouts train a closed-loop point-cloud policy whose observation is the union of scene geometry and rendered robot proprioceptive points:\\[P_t = P^{scene}_t \\cup P^{robot}_t\\]  The policy predicts a 36-DoF bimanual action chunk with a truncated-normal distribution and is trained by negative log-likelihood:\\[p_\\phi(a_{t:t+H-1} \\mid P_t)=\\prod_{\\tau=0}^{H-1}\\text{TN}(a_{t+\\tau}; \\mu_{t+\\tau}, \\sigma_{t+\\tau}, a_{min}, a_{max})\\]  The training loss is negative log-likelihood:\\[L_{policy} =-\\sum_{\\tau=0}^{H-1}\\log p_\\phi(a_{t+\\tau} \\mid P_t)\\]  At inference, the policy replans in a receding-horizon loop. The architecture uses PointNet++ features and action-query tokens; the input point budget is 2048, chunk size is 16, and control dimension is 36.  The policy ablation shows that demonstration quality dominates the learning result:                    Configuration        Success        Avg. L2                            Full SynManDex policy        80.7%        0.474                    No human prior        37.1%        0.622                    No force closure        22.9%        0.893                    No pre-validation        42.9%        0.561                    Scene-only point cloud        45.7%        0.539                    MLP pooling without action queries        40.0%        0.601              SynManDex also uses validated keyframes as an interface to a VLM agent. The VLM sees multi-view renders, object metadata, contact regions, hand-role candidates, admission metrics, and an allowed primitive library, then emits a JSON task specification with functional goals, hand roles, object-relative waypoints, release conditions, terminal predicates, and risk flags. The executor still checks IK, collision, possession, and task success. This keeps semantic planning attached to a physically vetted grasp state.  Experiments and Main Results  The main benchmark compares SynManDex with pose-only and trajectory-generation baselines:                    Method        Artifact        Bimanual        Pen. mm        FC        Bench success        IK/lift                            Dexonomy-XHand        pose        no        4.7        42.5%        36.8%        28.3%                    DexGraspNet        pose        no        3.4     ..."
    },{
      "title": "RoboPaint: From Human Demonstration to Any Robot and Any View",
      "url": "/posts/2026/06/robopaint-paper-notes/",
      "date": "2026-06-16",
      "tags": ["Robot Learning","Dexterous Manipulation","Human Demonstrations","VLA","Tactile Sensing","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  RoboPaint is a Real-Sim-Real pipeline that turns instrumented human manipulation into robot-view VLA data. The core story is compact: capture human motion, vision, and tactile contact; retarget the hand motion across embodiments with tactile constraints; reconstruct the real scene; render robot trajectories from arbitrary views; then train policies on the generated image-action data.  The paper is primarily a data-generation paper. Its strongest technical idea is Dex-Tactile retargeting, which maps glove states to dexterous robot-hand states while preserving contact timing and force-related cues. The headline numbers are substantial for this kind of pipeline: retargeted trajectories reach 84% real-world replay success across 10 objects, and Pi0.5 trained only on generated Real-Sim-Real data reaches 80% average success on pick-and-place, pushing, and pouring.  Paper Info  The paper is “RoboPaint: From Human Demonstration to Any Robot and Any View” by Jiacheng Fan, Zhiyue Zhao, Yiqian Zhang, Chao Chen, Peide Wang, Hengdi Zhang, and Zhengxue Cheng from Paxini Tech, Shanghai Jiao Tong University, and Zhejiang University.  The PDF is available as arXiv:2602.05325. The data-processing toolkit referenced by the paper is px-DataCollection/px_omnisharing_dataprocess_kit.  Core Technical Story  RoboPaint starts from the observation that passive human videos lack robot actions and tactile information, while direct robot teleoperation is slow, expensive, and tied to one embodiment. The paper therefore shifts the source of demonstrations to instrumented human operation. Operators wear custom gloves and manipulate objects in standardized capture rooms. The system records 11 RGB streams at 1200 x 1920, 3 RGB-D streams at 720 x 1280, 29-DoF glove joint angles, tactile readings from glove sensors, and synchronized timestamps. The tactile channel count is reported as 14-channel tactile signals in the abstract and 15 tactile channels in the contribution list, so I would treat that detail as slightly inconsistent in the paper text.  This capture setup is the foundation of the method. The demonstrations are stored as a physical trace of manipulation: what the operator sees, how the wrist and fingers move, where the object is, and where contact forces appear. Wrist 6D poses are estimated with ArUco markers on wristbands, object 6D poses with FoundationPose, and glove kinematics/tactile maps from the instrumented glove. Calibrated extrinsics then move these quantities from camera coordinates into the robot/simulation frame.  Cross-Embodiment Retargeting  Dex-Tactile retargeting maps human glove inputs:\\[(J^{Glove}, P^{Glove}, \\Gamma^{Glove})\\]  to target robot dex-hand states:\\[(J^{Dex}, P^{Dex}, \\Gamma^{Dex})\\]  The optimization has two terms:\\[L = L_{kin} + L_{tac}\\]  The kinematic term aligns fingertip positions and orientation vectors:\\[L_{kin} =\\frac{1}{N}\\sum_i\\lambda_{pos}\\|p_i^{Glove} - p_i^{Dex}\\|_2+ \\lambda_{dir}\\|d_i^{Glove} - d_i^{Dex}\\|_2\\]  The tactile term gives higher weight to active contact regions. Each tactile point on the glove is mapped to a corresponding location on the robot hand surface, and the normalized force controls the contact weight:\\[w_j^g = [1 + \\exp(-20(F_j - 0.5))]^{-1}\\]  This weighting is the key design choice. Human and robot hands differ in finger length, topology, joint limits, actuation, and contact geometry, so pose matching alone can produce the wrong grasp. RoboPaint uses tactile correspondences to keep contact-heavy regions important during optimization, then synthesizes robot tactile signals by attenuating the original glove tactile readings according to spatial mismatch. The generated robot record can therefore include an optional ObjTac-style tactile heatmap channel.  Scene Reconstruction and Rendering  The rendering side gives the pipeline its “any view” claim. RoboPaint reconstructs the deployment workspace with 3D Gaussian Splatting, aligns the 3DGS scene to the robot/simulation coordinate system using a known-size ArUco marker and a similarity transform, exports it as a USD asset, and imports it into Isaac Sim 5.1. Static background appearance comes from 3DGS, while dynamic objects and robots are rendered with mesh models.  At each time step, the robot arm joints are computed by IK from retargeted TCP poses, dex-hand joints come from Dex-Tactile retargeting, object poses follow the estimated object trajectory, and observations can be rendered from arbitrary cameras. The VLA record is:\\[d_t = [a_t, img_t^{visual}, (img_t^{tactile})]\\]  where:\\[a_t = [pos_t, rot_t, j_t^{Dex}]\\]  Here, the action stores TCP translation, TCP rotation direction, and dex-hand joint angles. The released toolkit also describes a practical data stack: DF-1 for preprocessed raw data, DF-2 for parsed encoder/tactile data with bimanual and object poses, DF-2R for dex-hand retargeting, and DF-3 for LeRobot-format training data.  Results  The experiments test geometric alignment, real-world replay, and downstream policy learning. In simulation validation, the authors reproject estimated 3D gloves, object poses, and tactile contact points back onto RGB frames, then replay retargeted manipulation in Isaac Sim. The reported average tactile contact error is 3.86 mm.  For real-world replay, the setup is UR5 plus Paxini DexH13 across 10 objects, with 10 demonstrations per object. Replaying retargeted end-effector trajectories and dex-hand joint angles reaches 84% average success. The paper reports higher success for simpler stable-contact objects and above-80% success even for harder objects such as a plastic cup and camera.  The policy experiment compares real teleoperation data with RoboPaint-generated Real-Sim-Real data on pick-and-place, push cuboid, and pour bottle. The most important table is:                    Model / Camera Setting        Tele Avg.        Paint Avg.                            Diffusion Policy        76.6%        50.0%                    Pi0.5 with wrist camera        100.0%        80.0%                    Pi0.5 without wrist camera        83.3%        46.6%              The best generated-data result is Pi0.5 with a wrist camera: 80% average success from painted data, compared with 100% from teleoperation. The gap is still real, but it is a meaningful tradeoff because the data source is much faster to collect and can be re-rendered across views.  The collection-time comparison explains why the paper cares about this tradeoff. For 100 successful demonstrations, human data collection is consistently faster, with speedups growing as tasks become longer or more dexterous:                    Task        Teleoperation        Human Demo        Speedup                            Pick and place        about 1h30m        about 35m        2.57x                    Open box        about 2h        about 30m        4.00x                    Push cuboid        about 2h        about 30m        4.00x                    Bagging fruits        about 10h        about 2h20m        4.28x                    Table bussing        about 12h        about 2h30m        4.80x                    Fold clothes        about 16h        about 3h        5.33x              Limitations and Takeaway  RoboPaint is still an infrastructure-heavy approach. It needs specialized capture rooms, instrumented gloves, calibrated cameras, object scans, environment reconstruction, tactile correspondences, accurate object poses, and feasible IK. Errors in any part of that chain can produce trajectories that look plausible in rendering but fail physically. The policy results are also narrow: three downstream tasks, a small replay object set, and a remaining gap against teleoperation data.  The clear takeaway is that RoboPaint should be read as a data-scaling recipe for dexterous VLA systems. Its contribution is the connection between multimodal human capture, tactile-aware cross-embodiment retargeting, 3DGS + Isaac Sim rendering, and policy training from generated Real-Sim-Real data. For my taxonomy, I would label it as:  Human-Demonstration-to-Robot Data Generation / Tactile-Aware Retargeting / Real-Sim-Real VLA Data Pipeline  这篇笔记支持通过页面顶部导航栏进行 English / 中文 切换。  TL;DR  RoboPaint 是一套 Real-Sim-Real 管线，把带传感器的人类操作转换成机器人视角的 VLA 数据。核心技术故事很清楚：采集 human motion、vision 和 tactile contact；用触觉约束做跨 embodiment 的手部 retargeting；重建真实场景；从任意视角渲染机器人轨迹；最后用生成的 image-action data 训练 policy。  这篇论文主要贡献在数据生成。最强的技术点是 Dex-Tactile retargeting：它把 glove states 映射成 robot dex-hand states，同时保留 contact timing 和 force-related cues。关键结果也比较有分量：retargeted trajectories 在 10 个物体上达到 84% real-world replay success；只用生成的 Real-Sim-Real 数据训练的 Pi0.5，在 pick-and-place、pushing、pouring 上达到 80% average success。  Paper Info  论文标题是 “RoboPaint: From Human Demonstration to Any Robot and Any View”，作者是 Jiacheng Fan, Zhiyue Zhao, Yiqian Zhang, Chao Chen, Peide Wang, Hengdi Zhang, and Zhengxue Cheng，来自 Paxini Tech、上海交通大学 和..."
    },{
      "title": "Qwen-VLA: Unifying Vision-Language-Action Modeling",
      "url": "/posts/2026/06/qwen-vla-paper-notes/",
      "date": "2026-06-16",
      "tags": ["VLA","Robot Learning","Embodied AI","Egocentric Data","Flow Matching","Paper Notes"],
      "excerpt": "",
      "text": "TL;DR  Qwen-VLA is best read as a scaling recipe for embodied generalist models. Its contribution is less about inventing a new robot controller in isolation and more about aligning three pieces that usually fight each other: a strong vision-language backbone, a continuous action expert, and a mixed embodied data interface that can absorb robots, navigation, synthetic trajectories, and human egocentric motion.  The core technical story has five parts. First, Qwen-VLA keeps Qwen3.5-4B as the semantic and spatial reasoning backbone. Second, it attaches a 1.15B DiT-style flow-matching action expert for continuous action chunks. Third, it uses embodiment-aware prompting plus a shared padded tensor interface so different robots can keep their native control conventions. Fourth, it introduces T2A, a text-to-action pretraining stage that teaches the action decoder a language-conditioned motor prior before visual grounding. Fifth, it builds a broad training mixture where robot data remains central, while human ego data, synthetic data, navigation, and auxiliary vision-language tasks provide coverage and regularization.  The takeaway is simple: Qwen-VLA treats VLA as a data and representation unification problem. The model does not claim that every embodiment shares a pure semantic action ontology. It instead creates a practical interface where prompts describe the body, masks select valid action channels, and the action expert learns to decode future motion under those constraints.  Problem Framing  The paper, “Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodiments”, is by the Qwen Team. It was submitted to arXiv on May 28, 2026, with v2 on June 1, 2026. The PDF is available at arXiv:2605.30280, and the official repository is QwenLM/Qwen-VLA.  Qwen-VLA frames manipulation, navigation, human hand motion, and trajectory prediction as one conditional prediction problem:\\[p_\\theta(y_{t:t+H-1} \\mid o_t, x, e, z)\\]  Here, (o_t) is visual context, (x) is the instruction, (e) is the embodiment description, and (z) is an optional task identifier. The target (y) may be an end-effector command, joint action, gripper state, dexterous-hand action, navigation waypoint, or human hand motion. The unifying move is the model interface: predict a future action or trajectory chunk while using prompts, masks, and dataset-specific normalization to keep the channel semantics interpretable.  Qwen-VLA supports multiple robot platforms through text prompts. A training example is prefixed with a description like:  The robot is {robot_tag} with {single arm / dual arms}[, waist][, and mobile base].The control frequency is {FPS} Hz.Please predict the next {chunk_size} control actions to execute the following task: {instruction}.    This prompt carries the platform, arm configuration, control frequency, horizon, and control convention. Actions then enter a fixed tensor:\\[Y \\in \\mathbb{R}^{H \\times K}\\]  If an embodiment uses only (c \\le K) channels, its valid values occupy the prefix of the vector and the rest are zero-padded. A binary mask (M \\in {0,1}^{H \\times K}) tells the loss which channels and timesteps are valid. The paper does not need a single declared semantic meaning for every coordinate of (K); the combination of embodiment prompt, dataset convention, per-dataset quantile normalization, and mask makes one action expert usable across many control spaces.  Action Expert and T2A  The architecture has a clean division of labor. The Qwen3.5-4B vision-language backbone handles perception, instruction following, visual grounding, and spatial reasoning. The DiT-style flow-matching action expert generates continuous action chunks. It concatenates VLM hidden states with a noisy action chunk, applies joint self-attention with AdaLN timestep conditioning, and learns a velocity field for denoising actions. At inference time, actions are produced with a small number of Euler integration steps. This keeps continuous motor prediction out of the language-token channel and gives the policy head capacity for high-frequency control.  The training recipe then builds the model in four stages: T2A, CPT, SFT, and RL. T2A is the key idea. During Stage I, the VLM is frozen, images are removed, and only the DiT action decoder learns from text plus embodiment prompts. The paper treats this as a compression-decompression problem: a compact instruction such as “pick up the red cup” must expand into a long, structured real-valued trajectory. T2A teaches the decoder the shape of plausible actions before the model has to solve visual grounding at the same time.  The ablation makes the point concrete. On Simpler-WidowX after SFT:                    T2A setting        SFT success                            No T2A        60.9%                    Full-sequence T2A with about 20% synthetic + 80% real text-action data        71.1%              Several details sharpen the story. Removing images during T2A helps the decoder focus on language-action structure and reduces cost. Full-sequence prediction performs better than chunk-only prediction because it exposes global temporal structure and termination patterns. Synthetic-only and real-only T2A both trail the mixed setting: synthetic trajectories broaden instruction coverage, while real trajectories anchor the prior in physical motion. The paper also reports that T2A can overfit when run too long, which is a useful reminder that pretraining a motor prior is still pretraining on a finite corpus.  After T2A, CPT unfreezes the VLM and action expert and trains on the heterogeneous embodied plus vision-language mixture. SFT uses curated downstream manipulation, navigation, grounding, and VQA data with task-balanced and embodiment-balanced sampling. RL starts from SFT and uses PPO with sparse binary success rewards in SimplerEnv; because flow-matching policies do not naturally expose token-style log probabilities, the paper injects controlled noise into Euler denoising transitions so PPO can recompute Gaussian log probabilities at the action-chunk level.  Data Recipe and Human Ego Actions  The pretraining mixture is the real engine of Qwen-VLA. The paper reports this sampling composition:                    Data source        Proportion                            Robot manipulation trajectories        74.2%                    Human egocentric trajectories        6.0%                    Navigation trajectories        7.5%                    Synthetic simulation trajectories        3.7%                    General vision-language data        3.4%                    Spatial grounding 2D        2.5%                    Autonomous driving VQA        2.4%                    Fine-grained embodied action caption        0.2%              Robot manipulation dominates the mixture, with public sources such as RobotSet, Galaxea, AgiBot World, RoboCOIN, RoboMIND, RDT-1B, DROID, BridgeData V2, RH20T, RT-1, and BC-Z, plus more than 1,000 hours of in-house real-robot trajectories and simulation-based manipulation data. The important design choice is that Qwen-VLA preserves source action formats: delta end-effector commands, absolute joint commands, gripper states, and dexterous-hand joints remain dataset-native, then get normalized and disambiguated through prompts. Camera views are also explicitly tagged with boundary tokens such as ego, cam_left_wrist, and cam_right_wrist.  Human egocentric data is only 6.0% of the mixture, but it is conceptually important because it supplies scalable manipulation priors from human activity. Qwen-VLA uses Ego4D and EPIC-KITCHENS subsets processed by VITRA, plus EgoDex, EgoVerse, and Xperience. For each egocentric sample, the model predicts a future bimanual hand action chunk. Each wrist is represented as 3D relative translation plus 3D axis-angle rotation, giving 6 wrist dimensions per hand.  Finger articulation is compressed. A MANO hand pose has 45 axis-angle dimensions, so Qwen-VLA applies PCA over the 45D hand pose across the human datasets and keeps the first 10 principal components. These coefficients are the eigengrasps.  For each hand, the action is therefore:\\[6 \\text{ wrist dims} + 10 \\text{ eigengrasp dims} = 16\\]  For two hands:\\[2 \\times 16 = 32\\]  So human ego contributes a 32D action per timestep: relative bimanual wrist motion plus compact hand articulation. This representation gives the model reusable hand-shape priors without forcing it to predict all MANO joint angles directly. It also has a clear boundary: eigengrasps compress human hand pose, while executable robot behavior still depends on robot trajectories, embodiment prompts, contact dynamics, and downstream fine-tuning.  Synthetic data plays two roles. The vision-language-action branch uses an internal ROBOINF-style pipeline to build tabletop scenes, generate tasks and success checks, produce motion programs, and roll out successful trajectories; the paper reports about 359,848 successful full trajectories i..."
    },{
      "title": "EgoEngine: From Egocentric Videos to Robot Demonstrations",
      "url": "/posts/2026/06/egoengine-paper-notes/",
      "date": "2026-06-16",
      "tags": ["Dexterous Manipulation","Imitation Learning","Human Videos","Synthetic Data","Robot Learning","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  EgoEngine treats egocentric human videos as raw material for robot demonstrations. Given a human RGB video, it builds a digital twin, removes the human body from the visual stream, renders the robot into the same scene, and optimizes a robot action trajectory that can reproduce the observed object motion. The generated training pair is:\\[(\\tilde{o}_t, \\tilde{a}_t)\\]  where (\\tilde{o}_t) is the robot-view observation and (\\tilde{a}_t) is the executable robot action. The paper’s strongest message is that the action side is the real bottleneck. Visual conversion helps, but downstream success rises mainly when the human video is converted into physically meaningful robot actions.  Paper Info  The paper is “EgoEngine: From Egocentric Human Videos to High-Fidelity Dexterous Robot Demonstrations” by Yangcen Liu, Shuo Cheng, Xinchen Yin, Woo Chul Shin, Alfred Cueva, Yiran Yang, Zhenyang Chen, Chuye Zhang, and Danfei Xu, from Georgia Institute of Technology and Tsinghua University. The arXiv PDF is 2606.12604, submitted as v1 on June 10, 2026. The project page is egoengine.github.io.  Core Argument  Egocentric human videos contain rich contact behavior, object motion, and task intent, but a robot policy cannot train directly from them. The visual stream contains human arms and hands, and the action stream is missing: human motion has different morphology, kinematics, actuation, and contact dynamics from the robot hand. EgoEngine’s central move is to make the video actionable by grounding both perception and control in a reconstructed digital twin.  The digital twin is object-centric. For Aria data, Aria Gen2 provides RGB frames and 3D hand poses; FoundationStereo estimates depth; SAM2 supplies hand, arm, and object masks; FoundationPose tracks 6D object pose; object meshes and calibration connect the egocentric camera, robot-mounted Aria frame, and robot base. For TACO, which lacks AprilTags, the base pose is estimated from object geometry with a fixed offset heuristic. This scene reconstruction is the shared coordinate system: the visual branch renders the robot in it, and the action branch uses it to optimize robot behavior against the human-observed object trajectory.  Action Branch Dominance  The action branch starts with human-centric retargeting. EgoEngine solves an IK problem with MINK to align robot fingertips and wrist orientation to the tracked human hand:\\[q_t^\\star =\\arg\\min_{q \\in Q}L_{tip}(q;t) + \\lambda_w L_{wrist}(q;t)\\]  The output reference trajectory (\\tau_{ref} = {q_t^\\star}_{t=1}^T) is useful as a motion prior, but it is still only a kinematic guess. The stronger step is object-centric refinement. Let (T_o^t) be the object pose tracked from the human video and (\\hat{T}_o^t) be the object pose produced by the robot in simulation. EgoEngine optimizes against the pose error:\\[e_t =\\sqrt{\\lambda_p d_p(\\text{trans}(\\hat{T}_o^t), \\text{trans}(T_o^t))^2+ \\lambda_R d_R(\\text{rot}(\\hat{T}_o^t), \\text{rot}(T_o^t))^2}\\]  Within the feasible region (e_t \\le C), the object reward is:\\[r_{obj}^t = C - e_t\\]  Rollouts terminate when the error exceeds (C). Auxiliary terms keep the robot close to the retargeted reference, smooth the actions, encourage useful contacts, and reward lifting when the task calls for it. This shifts supervision from copying the human hand to reproducing the object’s state change with the robot’s own embodiment.  The visual branch is still necessary. It removes human arms with SAM2 and Inpaint-Anything v2, renders the robot according to the optimized trajectory, and uses two-pass differential rendering to build an occlusion-aware robot mask:\\[\\tilde{M}_r^t(p) =\\mathbf{1}\\left[\\|I_{rob}^t(p) - I_{bg}^t(p)\\| > 0\\right]\\]  The final observation blends the rendered robot (R_t) with the inpainted background (\\bar{I}_t):\\[\\tilde{o}_t^{(r)} =\\tilde{M}_r^t \\odot R_t+ (1 - \\tilde{M}_r^t) \\odot \\bar{I}_t\\]  The ablation shows why the paper emphasizes actions. Visual editing alone barely changes real-robot success, while executable action generation explains most of the gain:                    Training data        Average real-robot SR                            Human Videos        0.03                    + Visual branch        0.05                    + Action branch        0.43                    EgoEngine        0.51              Chunk-Wise Solver Escalation  Dexterous trajectories are long, contact-rich, and expensive to optimize with full RL everywhere. EgoEngine decomposes each trajectory into chunks and escalates solver strength only when the cheaper choice fails the object-centric threshold. Replay directly follows the retargeted reference; MPC samples short-horizon corrections around that reference; RL trains a residual hand policy for difficult chunks:\\[\\delta a_t \\sim \\pi_\\phi(\\cdot \\mid s_t),\\qquada_t = a_t^{base} + \\delta a_t\\]  The paper calls this an MCTS-style mode switcher. The important detail is that it is a lightweight heuristic tree over chunk-level solver choices, not a full learned-value MCTS. At each chunk boundary, EgoEngine tries Replay, then MPC, then RL. A two-chunk window optimizes the current and next chunks together, executes only the current chunk, and then replans. This design keeps hard contact segments strong while avoiding full RL cost on easy segments.                    Dataset        Method        SR        Step        Reward        Cost                            TACO        Replay        0.17        0.29        0.29        1.00                    TACO        MPC        0.25        0.42        0.39        7,923                    TACO        RL        0.83        0.86        0.70        73,675                    TACO        EgoEngine        0.83        0.84        0.67        34,842                    Aria        Replay        0.10        0.66        0.62        1.00                    Aria        MPC        0.20        0.69        0.65        4,382                    Aria        RL        0.90        0.94        0.85        20,237                    Aria        EgoEngine        0.90        0.91        0.83        16,560              On Aria, EgoEngine reports 2.88 demos/hour on one RTX 4090 without parallelization, compared with 2.36 demos/hour for full RL. The larger TACO cost gap shows why chunk-wise escalation matters more as trajectories get longer.  Training and Evaluation Implications  After generation, EgoEngine aggregates synthetic robot demonstrations:\\[\\tilde{D}_{robot} = \\{(\\tilde{o}, \\tilde{a})\\}\\]  The policy is trained on RGB observations and proprioceptive states with an imitation objective:\\[\\min_\\theta\\mathbb{E}_{(\\tilde{o},\\tilde{a}) \\sim \\tilde{D}_{robot}}\\left[\\|\\pi_\\theta(\\tilde{o}) - \\tilde{a}\\|_2^2\\right]\\]  In the appendix, the policy uses a ResNet-18 visual stem, a proprioceptive stem, transformer token fusion, and a flow-matching action decoder. The key evaluation condition is clean: the policy trained from EgoEngine demonstrations uses no real-robot teleoperation data.  The real setup uses a single-arm RB-Y1 with one XHand and Aria Gen2. Across four Aria tasks, direct human video training and Phantom-style visual conversion are near zero, while EgoEngine reaches non-trivial zero-shot real-robot performance:                    Method        Mustard        Drawer        Flower        Hammer                            Human Video        0.00        0.10        0.00        0.00                    Phantom        0.00        0.05        0.00        0.00                    Real Robot        0.80        0.80        0.70        0.25                    EgoEngine        0.40        0.35        0.70        0.60              The task pattern is useful. Mustard and Drawer still favor real teleoperation, likely because precise pinch-like contact is hard to reconstruct and optimize from egocentric video. Flower and Hammer are friendlier to smooth human motion priors and power-grasp structure, where EgoEngine matches or exceeds real-robot demonstrations.  Limitations  The paper’s limitations are practical and concrete. Visual synthesis still relies on blending-based composition, so contact edges, occlusion, lighting, and robot-object interaction artifacts may become more important as tasks scale. Digital twin construction depends on object assets, pose tracking, calibration, and scene reconstruction; severe occlusion, deformable or transparent objects, and cluttered environments remain hard. Action optimization is cheaper than full RL but still simulation-heavy. The real-robot evaluation is also compact: four tasks, one real hardware setup, and a limited task distribution.  Takeaway  EgoEngine’s most transferable lesson is that action fidelity is the bottleneck in human-video robot learning. A realistic robot-view video is useful, but the decisive supervision is the executable action sequence tied to task outcome. EgoEngine uses object motion as the bridge: the..."
    },{
      "title": "From Egocentric Human Videos to Executable Robot Actions",
      "url": "/posts/2026/06/ego-video-to-robot-action-notes/",
      "date": "2026-06-16",
      "tags": ["Egocentric Video","Robot Learning","Imitation Learning","Reinforcement Learning","Dexterous Manipulation","Paper Notes"],
      "excerpt": "",
      "text": "TL;DR  Egocentric human video is becoming a practical source of robot-learning data, but the central problem is no longer just making the video look robotic. The hard step is converting a human interaction into an action sequence the robot can execute. A useful mental model for the field is:  human ego video  -> robot-looking video  -> robot-compatible supervision  -> executable robot trajectories  -> policy training data    My takeaway from Phantom, Masquerade, HumanEgo, and EgoEngine is that the field is shifting from visual embodiment transfer to contact-feasible action generation. Visual editing helps reduce the observation gap; hand-object representations capture the transferable structure of manipulation; executable trajectory refinement is what turns a video prior into usable robot demonstrations.  The Core Question  Egocentric videos are attractive because they are cheap, scalable, and naturally record hand-object interaction. A person can wear smart glasses or a head-mounted camera and collect many manipulation episodes without robot hardware. The conversion problem is much harder: the video contains human hands, arms, head motion, camera motion, human morphology, occlusions, and contact behavior that may not map cleanly to the target robot.  Most papers in this cluster can be read through two gaps. The visual gap asks what the policy should see: human observation or robot observation. The action gap asks what the robot can actually do: rough hand motion or feasible robot control. For simple reaching and pick-and-place, visual conversion may be enough to make robot data more sample-efficient. For dexterous, bimanual, or contact-rich tasks, action feasibility usually becomes the bottleneck.  Four Papers, One Trajectory  Phantom is the clean visual-editing baseline: estimate hand poses from human video, remove the human arm, render a robot, and train policies from robot-like demonstrations. Its strength is simplicity and zero-shot deployment without robot-specific data; its limitation is that the setup is curated and depends on good hand pose, depth, and an edited trajectory close enough to what the robot can execute. Source: project page, arXiv:2503.00779.  Masquerade scales the same intuition to in-the-wild egocentric videos such as Epic Kitchens. It estimates left/right hand trajectories, inpaints the human embodiment, overlays a rendered bimanual robot, pretrains a ViT encoder to predict future 2D robot keypoints, and co-trains with a diffusion policy head using only 50 robot demonstrations per task. The key empirical message is practical: imperfect robot overlays still help visual representation learning and long-horizon bimanual generalization. The limitation is that human videos mainly provide auxiliary supervision; the final action policy still needs task-specific robot demonstrations. Source: project page, arXiv:2508.09976.  HumanEgo shifts attention from edited pixels to interaction structure. Its Interaction-Centric Token (ICT) encodes hand and object poses, relative spatial relationships, and grasp state; a flow matching action generator is trained with dense auxiliary objectives such as 2D trace prediction, object motion prediction, and latent consistency. The conceptual point is strong: manipulation is defined by how hands approach, grasp, move, release, and coordinate around objects. The limitation is perception dependence: the pipeline needs reliable hand/object tracking, entity pose estimation, and relatively clean task-level demonstrations, so messy in-the-wild videos and harder contact remain important stress tests. Source: arXiv:2605.24934.  EgoEngine gives the most complete formulation because it generates both robot observation videos and task-aligned executable action trajectories. Given an egocentric RGB video, it builds a digital twin, renders robot-view observations, maps human motion into robot rollout, and refines the result under feasibility constraints. Its ablation is the clearest evidence that action generation matters more than visual conversion alone:                    Setting        Success Rate                            Human Videos        0.03                    + Visual branch        0.05                    + Action branch        0.43                    Full EgoEngine        0.51              These numbers are best read as a directional trend, with no claim of official contribution percentages. Visual conversion alone moves average success from 0.03 to 0.05, while adding the action branch reaches 0.43 and the full system reaches 0.51. Source: project page, arXiv:2606.12604.  Compact Comparison                    Paper        Main Data Processing        Robot Data Needed?        Action Fidelity        Best Use Case                            Phantom        Hand pose + inpainting + rendered robot        No        Medium        Simple zero-shot human-video-to-policy pipeline                    Masquerade        In-the-wild video editing + robot overlay + co-training        Yes, small task-specific set        Medium        Scaling visual representation from web ego data                    HumanEgo        Hand-object entity representation + flow matching        No        Medium to high, depending on perception        Data-efficient zero-shot transfer from clean ego demos                    EgoEngine        Digital twin + robot observation generation + executable trajectory refinement        No real robot demos for policy learning        High        Generating full robot demonstrations from human videos              Why Action Refinement Matters  The strongest conceptual paper in this group is EgoEngine because it treats action generation as the main object, not a side effect of visual editing. Masquerade is the most practical visual-representation system because it scales to in-the-wild videos and shows how small robot datasets can benefit from edited human clips. HumanEgo has the most interesting representation idea because ICT focuses directly on the hand-object relation. Phantom remains the foundation that made the data-editing route concrete.  The next step is likely ego-to-robot action refinement with physics feedback. In tabletop pick-and-place, a rough retargeted trajectory may work; in dexterous manipulation, bimanual coordination, tool use, insertion, twisting, wiping, folding, and other contact-rich tasks, small action errors dominate. The grasp pose may be slightly wrong, contact may arrive too early or too late, the object may rotate into an infeasible orientation, one hand may block the other, force may matter more than pose tracking, and release timing may decide success. These are contact and control problems.  A promising role for RL, MPC, or trajectory optimization is therefore refinement guided by a human prior, with no need to start from open-ended exploration:  human ego video  -> rough human hand/object trajectory  -> robot retargeting  -> simulation or digital twin rollout  -> RL / MPC / trajectory optimization refinement  -> executable robot demonstration  -> downstream imitation policy    In this pipeline, the learning or optimization step starts from a strong human prior and fixes what the video does not specify in robot coordinates: stable grasp pose, contact force and compliance, object reorientation, handover timing, bimanual coordination, release phase, and recovery from small tracking errors. The goal is to manufacture better demonstrations from human videos, then train a robust visuomotor policy on those demonstrations.  The clear takeaway is that egocentric video becomes truly useful for robot learning when it grows from a visual pretraining source into a robot demonstration engine. The representation should capture hand-object interaction; the learning step should refine that interaction into feasible robot action; the empirical signal from EgoEngine suggests that this action branch is the main lever.  TL;DR  Egocentric human video 正在变成机器人学习的一种现实数据来源，但核心问题已经从视觉外观迁移到动作可执行性。真正困难的一步，是把人类交互转换成机器人可以执行的动作序列。可以用下面这条线索理解这个方向：  人类第一视角视频  -> 看起来像机器人在做的视频  -> 机器人可用的监督信号  -> 可执行的机器人动作轨迹  -> policy 训练数据    我从 Phantom、Masquerade、HumanEgo 和 EgoEngine 里读到的主线是：这个方向正在从 visual embodiment transfer 走向 contact-feasible action generation。Visual editing 可以缩小 observation gap；手-物体表示能捕捉 manipulation 的可迁移结构；可执行轨迹 refinement 才能把视频先验变成真正可用的机器人 demonstrations。  核心问题  Egocentric videos 有吸引力，是因为它便宜、可扩展，而且自然记录手和物体的交互。一个人戴上智能眼镜或者头戴相机，就能在不依赖机器人硬件的情况下采集很多 manipulation episodes。难点在转换：视频里有人的手、手臂、头部运动、相机运动、人类形态、遮挡，以及目标机器人未必能直接复现的接触行为。  这组论文基本都在处理两个 gap。Visual gap 关心 policy 看到什么：human observation 还是 robot observation。Action gap 关心机器人真的能做什么：粗略的人手运动还是可行的机器人控制。对简单 reaching 和 pick-and-place 来说，visual conversion 可能已经能提高样本效率；对 dexterous、bimanual、contact-rich tasks 来说，动作可行性通常会变成瓶颈。  四篇论文，一条主线  Phantom 是很干净的 visual-editing baseline：从 human video 估计手部姿态，移除人的手臂，渲染机器人，再用 robot-like demonstrations 训练 policy。它的优点是简单，并且能在没有 robot-specific data 的情况下 zero-shot 部署；限..."
    },{
      "title": "DexterCap: Affordable and Automated Capture of Complex Hand-Object Interactions",
      "url": "/posts/2026/06/dextercap-paper-notes/",
      "date": "2026-06-16",
      "tags": ["Dexterous Manipulation","Motion Capture","Hand-Object Interaction","Dataset","Robot Learning","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  DexterCap is a low-cost optical motion-capture system for fine-grained hand-object interaction. The paper’s main contribution is a capture-and-reconstruction pipeline for data collection; policy learning is downstream. It shows that dense, identifiable visual markers plus automated geometry can collect subtle in-hand manipulation data with higher fidelity and less manual cleanup than typical low-cost capture setups.  The key mechanism is to attach many character-coded marker patches directly to relatively rigid hand regions and objects. CornerNet, EdgeNet, and BlockNet recover marker corners, valid block edges, and two-character IDs from multi-view grayscale video; triangulated 3D markers then drive MANO hand fitting and object pose reconstruction. The resulting DexterHand dataset contains long in-hand manipulation sequences over primitive objects and a Rubik’s Cube, making the work most useful as dataset infrastructure for dexterous robot learning.  Paper Info  The paper is “DexterCap: Affordable and Automated Capture of Complex Hand-Object Interactions” by Yutong Liang, Shiyi Xu, Yulong Zhang, Bowen Zhan, He Zhang, and Libin Liu, accepted to Eurographics 2026 / Computer Graphics Forum. The arXiv entry uses the title “DexterCap: An Affordable and Automated System for Capturing Dexterous Hand-Object Manipulation” and is available as arXiv:2601.05844. The project page is pku-mocca.github.io/Dextercap-Page, with code at PKU-MoCCA/dextercap and dataset access through Hugging Face.  Core Problem  Fine hand-object capture sits in an awkward middle ground. Commercial optical mocap is accurate but expensive and often needs cleanup when markers disappear or swap. Data gloves reduce visual occlusion but still face drift and finger-accuracy issues. Markerless RGB/RGB-D methods are cheaper, yet severe self-occlusion, close contact, motion blur, and small in-hand rotations remain difficult.  DexterCap chooses explicit sensing design. The prototype uses 13 Hikvision MV-CS050-10GM industrial GigE PoE cameras around a 2 x 1 x 2 m capture cage, recording grayscale video at 2048 x 2448 and 20 FPS. The reported hardware cost is under 6,000 USD. The system still needs one-time calibration, subject-specific MANO shape estimation, and a small amount of session labeling: 2-3 frames from each video captured by each camera, with about 180 labeled frames enough for the image models to generalize well in the same environment.  Method  DexterCap’s marker design is the center of the system. Each checkerboard-like white square contains a two-character ID drawn from uppercase letters and digits after removing visually confusing characters, giving 324 unique tags; an underscore marks orientation. Instead of putting markers on a glove that can stretch, wrinkle, or slide, the system attaches marker patches to finger knuckles, finger segments, dorsum, and palm. Each hand uses 19 patches and provides more than 500 detectable corners. Object surfaces are marked as well, so rigid objects can be solved by pose alignment and articulated objects can expose internal state.  The visual parser is a three-stage cascade. CornerNet predicts checkerboard-corner heatmaps on grayscale patches. EdgeNet decides whether two candidate corners form a valid block edge, which makes assembly much cheaper than exhaustive quadrilateral validation. BlockNet reads the two characters and orientation, after which a voting step uses the known marker-patch layout to correct local mistakes. The edge-first design is both a speed trick and a reliability trick: it reduces wrong correspondences before geometric reconstruction begins.  Once 2D marker IDs are recovered across cameras, DexterCap triangulates 3D marker positions, keeps markers observed by at least three cameras, and uses RANSAC to reject outlier observations. It then removes inconsistent 3D clusters within a marker patch, filters abnormal temporal z-scores, and linearly interpolates short gaps when neighboring frames contain observations.  Hand reconstruction uses MANO with shape parameters (\\beta \\in \\mathbb{R}^{10}) and pose parameters (\\theta \\in \\mathbb{R}^{45}). The authors define anatomically informed local joint coordinates and reduce the controllable hand pose to:\\[\\phi \\in \\mathbb{R}^{27}\\]  which maps differentiably into the full MANO pose space. For each subject, a coarse Structure Sensor scan mounted on an iPhone produces a mesh of about 6k vertices for shape fitting through Chamfer distance, optionally helped by finger-length measurements. Calibration ties physical markers to constrained MANO submeshes and fixes marker-to-surface barycentric coordinates; per-frame optimization then solves global translation, global orientation, and hand pose with pose-limit regularization. When a marker and downstream markers on the same kinematic chain are occluded, the corresponding joint DoFs are held at their previous-frame values for temporal stability.  Object reconstruction is simple for rigid objects and more interesting for the Rubik’s Cube. Rigid pose is estimated with the Kabsch algorithm by aligning observed object markers to canonical marker positions on the object mesh. For the 2 x 2 x 2 Rubik’s Cube, DexterCap uses 384 markers on external facelets, detects the rotating face through coplanarity analysis, decomposes the cube into two 1 x 2 x 2 blocks, registers them separately, and snaps accumulated rotation to discrete quarter turns. This example matters because it turns the system from rigid object tracking into structured articulated-state capture.  Dataset and Results  DexterCap is used to build DexterHand, an open-source dataset for in-hand manipulation over seven basic object shapes plus a Rubik’s Cube, including cuboids, cylinder, disk/plate, ring, and triangular prism variants. The selected sequences in Table 1 total 4936.65 seconds, about 82 minutes, with most sequences lasting around 7-12 minutes. The reported average hand-object penetration is 0.38 +/- 0.31 cm, which the authors interpret as consistent with real hand deformation under grasping forces.  Marker extraction is quantitatively strong: CornerNet reaches 94.7% precision, 81.6% recall, and 87.7% F1 at the image level; EdgeNet reaches 99.02% accuracy, 98.9% precision, 99.1% recall, and 99.0% F1; BlockNet reaches 98.39% orientation accuracy, 97.95% left-character accuracy, and 97.36% right-character accuracy. The edge-first assembly reduces the search from 5550 quads per frame to 83 blocks via 707 edges. Reconstruction errors are also small for this capture setting: triangulated detected markers have 1.42 px reprojection error, MANO marker reconstruction error is 0.77 +/- 0.28 mm during calibration and 2.06 +/- 1.09 mm during dynamic manipulation, and object marker fitting error is 1.512 mm.  For motion quality, the paper compares DexterHand with GRAB, ARCTIC, HUMOTO, HaMeR, and GigaHands. The table is worth keeping because it compresses the main empirical claim: DexterHand is competitive with mocap/data-glove datasets and much stronger than vision-only baselines for fine in-hand manipulation.                    Dataset        MSNR ↑        Jerk ↓        Diversity ↑        Coherence ↑                            DexterHand / Ours        9.31        0.76        0.97        0.68                    GRAB (Vicon)        7.29        3.68        0.91        0.70                    ARCTIC (Vicon)        7.82        0.91        0.90        0.81                    HUMOTO (Data Glove)        7.51        1.90        0.93        0.63                    HaMeR (Vision)        -0.05        23.76        0.90        0.81                    GigaHands (Vision)        3.50        2.62        0.91        0.73              Why It Matters  For robot learning, DexterCap is valuable because it records the things policies and tracking controllers often need but ordinary video rarely provides cleanly: fine hand articulation, object pose trajectories, contact-rich motion, long-horizon interaction, and articulated object state. DexterHand can therefore serve as a manipulation-prior source for later work such as ConTrack, which evaluates on DexterHand clips for continuous single-hand in-hand rotation.  The strongest idea is the full systems argument. DexterCap combines affordable camera hardware, dense explicit markers, learned visual parsing, geometric reconstruction, and MANO fitting into one capture loop. The Rubik’s Cube example expands that argument from 6-DoF rigid pose to structured object state, which is exactly the sort of signal future dexterous manipulation datasets need.  Limitations  DexterCap remains a vision-based marker system, so severe occlusion is still a real failure mode. The paper specifically mentions the ring-object case where fingers can be fully occluded, producing artifacts such as finger-object penetration. The dataset is useful but still limited in subject ..."
    },{
      "title": "ConTrack: Constrained Hand Motion Tracking with Adaptive Trade-off Control",
      "url": "/posts/2026/06/contrack-paper-notes/",
      "date": "2026-06-16",
      "tags": ["Dexterous Manipulation","Reinforcement Learning","Reference Tracking","Human Demonstrations","Robotics","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  Core Idea  ConTrack is a constrained reinforcement-learning controller for tracking dexterous hand-object demonstrations. The paper is “ConTrack: Constrained Hand Motion Tracking with Adaptive Trade-off Control” by Yutong Liang, Quanquan Peng, Ri-Zhao Qiu, and Xiaolong Wang from UC San Diego; the paper is available as arXiv:2606.03177, with a project page at lyt0112.com/projects/ConTrack.  The core argument is that human-to-robot tracking should be framed as a trade-off problem, beyond pure imitation. A human hand trajectory can contain unreachable robot postures, unstable contacts, or object motions that only work for a different embodiment. ConTrack therefore separates task from style: object tracking is treated as the primary requirement, while hand motion and contact fidelity are optimized as style once the object trajectory is under control.  Each reference clip is modeled as a finite-horizon MDP. The state includes robot joints, object poses, and velocities; the reference provides retargeted robot joint targets, object pose targets, link-level contact events, and object-local contact points. The policy predicts a residual around the reference,\\[q_t^{tar} = q_t^{ref} + a_t\\]  so the controller stays near the demonstration while retaining enough freedom to resolve physical conflicts. The reward decomposes into task tracking, style fidelity, and motion regularization:\\[r(s_t, a_t) = r_g(s_t, a_t) + r_s(s_t, a_t) + r_p(s_t, a_t)\\]  where (r_g) measures object pose tracking, (r_s) measures hand kinematics and contact fidelity, and (r_p) penalizes high-frequency or unstable motion. The constrained objective is the most compact statement of the paper:\\[\\max_\\pi J_s(\\pi) \\quad \\text{s.t.} \\quad J_g(\\pi) \\ge \\alpha J_g^\\star\\]  In words, ConTrack maximizes style fidelity while maintaining a target fraction of the best observed task-tracking return. The target ratio (\\alpha) becomes the knob that defines how much object-tracking safety the policy must maintain before it spends capacity on motion style.  Adaptive Task-Style Mixing  The main mechanism is an online controller for the task-style frontier. ConTrack uses a Lagrangian-style relaxation,\\[\\mathcal{L}(\\pi, \\lambda) =J_s(\\pi) -\\lambda \\left(\\alpha - \\frac{J_g(\\pi)}{J_g^\\star}\\right)\\]  where (\\lambda) controls how strongly the policy should emphasize object tracking. PPO updates the policy with a mixed advantage,\\[A_{mix} =w_{task} A_g +(1 - w_{task}) A_s +A_p\\]  with:\\[w_{task} = \\sigma(\\lambda)\\]  and the dual variable is updated online:\\[\\lambda \\leftarrow \\lambda +\\eta \\left(\\alpha - \\frac{\\hat{J}_g}{J_g^\\star}\\right)\\]  When normalized task return drops below (\\alpha), (\\lambda) rises and the policy shifts pressure toward object tracking. When task return is comfortably above the target, (\\lambda) falls and the policy can recover more hand-pose and contact style. This is the paper’s cleanest departure from fixed reward mixtures: the controller adapts the trade-off during training instead of requiring a single static reward balance for every sequence.  Contact Priors and Adaptive Resets  Contact priors are what keep the style term from degenerating into joint-angle imitation. The reference contains binary contact events between hand links and objects, plus contact points in the object frame. The style reward encourages the policy to match the reference contact timing and keep contact points near the annotated object-local targets. This matters because low object pose error alone can hide very different finger-object interactions.  The reset mechanism addresses the long-horizon side of the same problem. If every rollout begins at frame one, early failures dominate training and later contact phases receive little signal. ConTrack maintains a reset library indexed by reference frame, storing policy-reachable states instead of raw reference states. That detail is important: directly resetting to a human hand-object reference can create impossible contacts in simulation, while states visited by the current policy are physically consistent under the learned controller.  For each frame (k), ConTrack estimates a survival ratio,\\[u_k = \\frac{\\bar{\\ell}_k}{T-k}\\]  then samples reset frames with\\[p(k) \\propto \\exp(-u_k / \\tau)\\]  This favors segments where the policy fails quickly. As tracking improves, the sampled starts move with the current failure boundary, producing a curriculum that remains tied to reachable contact states.  Experiments and Main Results  ConTrack is evaluated on GRAB for bimanual rigid-object interaction, ARCTIC for articulated and multi-object bimanual interaction, and DexterHand for single-hand in-hand rotation. Each clip is trained as an independent tracking task for 5000 PPO updates under a fixed simulator-step budget. Evaluation starts from the first reference frame, so progress measures whether the controller can survive the full sequence instead of only selected resets.                    Method        Progress ↑        Obj pos ↓        Obj rot ↓        Finger err ↓        Contact F1 ↑        Contact pt ↓                            ConTrack        0.899        0.026 m        0.272 rad        0.163 rad        0.784        0.018 m                    ManipTrans        0.743        0.012 m        0.207 rad        0.277 rad        0.620        0.030 m                    DexMachina        0.246        0.038 m        0.348 rad        0.147 rad        0.708        0.024 m                    SPIDER        0.444        0.201 m        1.104 rad        0.157 rad        0.191        0.036 m              The table captures the main trade-off. ManipTrans has lower object pose error on frames it survives, while ConTrack reaches much higher progress and stronger contact fidelity. DexMachina keeps finger motion close to the reference but has limited progress under the same budget. SPIDER lacks a learned feedback policy and struggles once contact dynamics dominate.  The ablations line up with the method story: adaptive task-style mixing improves progress and contact fidelity over fixed mixing; the reset library improves progress over start-only and uniform mid-clip resets; contact prior rewards improve contact F1 and contact-point accuracy. The paper also reports a real-world feasibility study on a tabletop bimanual platform with two xArm7 arms and two xHands, where policy-predicted joint references are streamed from simulation to a real-time controller over TCP.  Relation to ManipTrans  ManipTrans is a useful comparison because it also transfers human bimanual manipulation to dexterous robot hands through residual learning. Its pipeline pretrains a generalist trajectory imitator for hand motion, then fine-tunes residual corrections under interaction constraints. ConTrack changes the emphasis: it turns object tracking into the explicit constraint, treats hand motion and contacts as style, and uses an online dual controller to adapt the balance during RL. It also makes mid-trajectory resets depend on policy-reachable states, which is especially relevant when direct reference resets produce inconsistent contact configurations.  My short taxonomy is: ManipTrans is a two-stage transfer pipeline centered on imitation plus residual correction; ConTrack is a constrained RL tracking controller centered on adaptive task-style allocation.  Strengths and Limitations  ConTrack’s strength is conceptual clarity. It names the central conflict in dexterous tracking: object success and motion fidelity often compete. The dual controller turns that conflict into an explicit optimization mechanism, and the reset library keeps training aligned with the simulator states the policy can actually reach. The metrics are also well chosen: progress, object errors, joint errors, contact F1, and contact point error make it harder for a method to look good by optimizing only one side of the task-style trade-off.  The limitations are also clear. The constrained formulation uses a running maximum of task return for normalization, so it behaves as a practical controller instead of a strict guarantee. Contact priors depend on the availability and quality of contact annotations. The hardest DexterHand Ring clip remains difficult under the fixed 5000-update budget; the appendix reports that longer training can reach 94% success with 100,000 PPO updates, which suggests feasibility while exposing compute sensitivity. The real-world study demonstrates executable joint-command streaming, but broader deployment would still need stronger perception and tighter sim-to-real alignment.  Takeaway  The practical recipe is compact: use human motion as a reference, give object tracking priority through a constrained objective, preserve hand motion and contact as style when the task allows it, train from policy-reachable mid-trajectory states, and report progress together with contact fidelity. For my own mental taxonomy, I ..."
    },{
      "title": "Contact-Grounded Policy: Dexterous Visuotactile Policy with Generative Contact Grounding",
      "url": "/posts/2026/06/contact-grounded-policy-paper-notes/",
      "date": "2026-06-16",
      "tags": ["Dexterous Manipulation","Tactile Sensing","Diffusion Policy","Contact-Rich Manipulation","Robot Learning","Paper Notes"],
      "excerpt": "",
      "text": "TL;DR  Contact-Grounded Policy (CGP) is a supervised visuotactile diffusion policy for contact-rich dexterous manipulation. Its main argument is that tactile prediction becomes useful for control only when it is grounded in what the low-level compliance controller can actually execute. CGP therefore predicts future actual robot states and tactile feedback together, then maps each predicted state-contact pair into a target robot state for the controller.    The figure captures the control issue: actual robot state, target robot state, and contact patches are coupled through the compliant controller. CGP learns this coupling from demonstrations, which turns contact from an observed consequence into a realizable control target.  Paper Info  The paper is “Contact-Grounded Policy: Dexterous Visuotactile Policy with Generative Contact Grounding” by Zhengtong Xu, Yeping Wang, Ben Abbatematteo, Jom Preechayasomboon, Sonny Chan, Nick Colonnese, and Amirhossein H. Memar. It is listed as Robotics: Science and Systems (RSS), 2026 on the project page. The paper is available as arXiv:2603.05687, with project materials at contact-grounded-policy.github.io.  Core Argument  Dexterous manipulation is hard because success depends on multi-point contacts that evolve continuously. A hand may need to roll, pinch, press, wipe, twist, or delicately hold an object while contact points migrate across fingers and palm. These interactions depend on object geometry, friction, slip, compliance, and partial observability.  Many visuomotor policies predict kinematic targets such as end-effector poses, hand joint angles, or action chunks. In contact-rich tasks, a geometrically plausible hand target can still slip, apply the wrong force, break a fragile object, or lose a contact patch. The paper’s key critique is that feeding tactile observations into a policy, or predicting tactile signals as an auxiliary objective, does not ensure that the resulting contact can be realized by the controller.  CGP frames contact under a fixed tactile sensor and compliance controller as a triplet: actual robot state, tactile feedback, and target robot state. The gap between target and actual state is meaningful because contact forces can push the robot away from its commanded reference. During contact, that deviation encodes physical interaction, so CGP learns the setup-specific mapping:  actual robot state + tactile feedback -> target robot state    This is the paper’s main mechanism. The diffusion model proposes future state-contact evolution, and the learned contact-consistency mapping turns that evolution into controller references.  Method  CGP has two coupled modules. First, a conditional diffusion model predicts a short-horizon trajectory of future actual robot states and future tactile latents. Second, the contact-consistency mapping converts every predicted actual-state/tactile pair into an executable target state:  predicted actual state + predicted tactile latent -> target robot state    At inference time, CGP runs in a receding-horizon loop. The diffusion model predicts the next 16 steps of actual state and tactile feedback, the mapper converts them to target states, and the controller executes 8 steps before replanning. Rollout runs at 5 Hz with 8 DDIM denoising steps.  The tactile stream is compressed before diffusion. In simulation, the Tesollo DG-5F hand uses dense tactile arrays with 768 sensing points, each reporting a 3D force vector. On hardware, each Digit360 fingertip sensor produces RGB tactile images. CGP uses a KL-regularized VAE: dense tactile arrays use a 1D ResNet-style encoder/decoder and a 32-dimensional latent, while four Digit360 sensors use shared 2D ResNet-style per-sensor encoders/decoders and an 80-dimensional latent, or 20 dimensions per sensor. The KL term matters because lower reconstruction error without KL can produce a less structured latent space and worse rollout performance.  Evaluation  The evaluation spans simulation and real hardware. In simulation, the robot is a UR5 arm with a Tesollo DG-5F five-finger 20-DoF hand; the arm uses operational-space impedance control, the hand uses joint-space PD control, and tactile sensing comes from dense whole-hand tactile arrays. On hardware, the system uses a Franka Panda arm, an Allegro V5 four-finger 16-DoF hand, four Digit360 fingertip tactile sensors, and two RGB views: an agent view and a wrist view. Demonstrations are collected through Meta Quest 3 VR teleoperation in simulation and through OptiTrack mocap plus an instrumented glove on the real robot, with retargeting and a shared compliant control stack.  The task suite is deliberately contact-heavy: simulated in-hand box flipping, simulated fragile egg grasping, simulated dish wiping, real jar opening, and real in-hand box flipping. CGP is compared against a visuomotor diffusion policy using vision and robot state, and a visuotactile diffusion policy using vision, robot state, and tactile observations without contact grounding.                    Task        CGP        Visuotactile DP        Visuomotor DP                            In-Hand Box Flipping (Sim)        66.0%        58.0%        53.2%                    Fragile Egg Grasping (Sim)        74.8%        70.0%        53.2%                    Dish Wiping (Sim)        58.4%        43.6%        42.4%                    Jar Opening (Real)        93.3%        66.7%        73.3%                    Real In-Hand Box Flipping        80.0%        60.0%        60.0%              The largest gains appear where contact evolution is central: dish wiping, jar opening, and real in-hand flipping. The time-aligned predicted and observed tactile visualizations strengthen the result because CGP predicts tactile trajectories, converts them into controller targets, and then observes future tactile frames that closely match the earlier predictions. That makes the tactile prediction part of the executed behavior, not merely an auxiliary forecast.  The paper also isolates the contact-consistency mapping with a hand-configuration prediction task using 150 teleoperated grasping episodes, 4114 frames, and 11 objects in simulation. The strongest setting combines state and tactile input, a ResNet-style tactile encoder, and residual target prediction. MAE drops from 8.80 in absolute mode to 5.94 in residual mode; removing either modality hurts, with 10.64 for state-only residual prediction and 12.15 for tactile-only residual prediction even with the ResNet encoder. This supports the central hypothesis: tactile feedback and actual state together encode the controller-realizable contact structure.  Limitations  The main limitation is specificity. The contact-consistency mapping is tied to a particular hand, tactile sensor layout, and compliance controller; changing the sensor type, controller gains, or hand embodiment may require retraining or adaptation. The evaluation is also single-task: each policy is trained and evaluated on one task, so the paper does not yet establish cross-task transfer of contact knowledge across different objects, objectives, and contact regimes.  CGP also depends on specialized tactile hardware. Digit360 fingertip sensors are powerful, but broader deployment would need more common tactile sensing or robust cross-sensor adaptation. Finally, the policy is supervised from demonstrations. This avoids reward engineering, while still leaving scale dependent on collecting diverse contact-rich demonstrations.  Takeaways  CGP is worth reading because it clarifies a useful principle for dexterous manipulation: contact prediction should be shaped by the controller that must realize it. The reusable recipe is compact: represent contact through actual state, tactile feedback, and target state; predict future actual state and tactile feedback together; learn the contact-consistency mapping into executable controller references; compress tactile observations into a KL-regularized latent before diffusion; and treat the compliance controller as part of the learned contact model.  My taxonomy for this paper:  Supervised Visuotactile Diffusion Policy / Contact-Grounded Dexterous Manipulation / Controller-Aware Tactile Prediction  TL;DR  Contact-Grounded Policy (CGP) 是一个用于 contact-rich dexterous manipulation 的 supervised visuotactile diffusion policy。它的核心论点是：tactile prediction 只有和 low-level compliance controller 的可执行目标连接起来，才真正对控制有用。因此，CGP 同时预测未来的 actual robot states 和 tactile feedback，再把每个 predicted state-contact pair 映射成 controller 的 target robot state。    这张图直接画出了控制问题：actual robot state、target robot state 和 contact patches 通过 compliant controller 耦合在一起。CGP 从 demonstrations 中学习这个耦合关系，让 contact 从被动观测结果变成可实现的控制目标。  Paper Info  论文标题是 “Contact-Grounded Policy: Dexterous Visuotactile Policy with Generative Contact Grounding”，作者是 Zhengtong Xu, Yeping Wang, Ben Abbatematteo, Jom Preechayasomboon, Sonny Chan, Nick Colonnese, and Amirhossein H. Memar。项目页标注为 Robotics: Science and Systems (RSS), 2026。论文..."
    },{
      "title": "RigidFormer: Learning Rigid Dynamics using Transformers",
      "url": "/posts/2026/05/rigidformer-paper-notes/",
      "date": "2026-05-13",
      "tags": ["World Models","Robotics","Rigid Dynamics","Point Clouds","Transformers","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  RigidFormer is a mesh-free learned simulator for multi-object rigid-body dynamics. It takes two recent object-level point-cloud states, a temporal step size, and optional control signals, then predicts the next point-cloud state. Instead of updating every point independently, it compresses each object into an object token, lets object tokens interact through a Transformer, advances a small set of anchors per object, and then recovers a rigid transform with differentiable Kabsch alignment.  The paper is strong as a simulator-style world model. It improves the efficiency and scalability of rigid contact rollout by moving from vertex-level interaction to object-level and anchor-level interaction. The key move is not only “use a Transformer”; it is the choice to respect the low-dimensional structure of rigid motion while still reading contact-relevant local geometry from point clouds.  The most useful critical lens here is to separate “world model” into two concepts:  Action-Conditioned World Model:world state + action/control -> future world stateWorld-Action Model:world observation/change + goal -> action-relevant belief, affordance, risk, or action    RigidFormer mostly belongs to the first category. It asks: if the physical scene continues under this dynamics/control condition, what will the future object state be? But a policy may not need a full future reconstruction. For action, the important question is often: what changed, what matters, and how should I react?  Paper Info      Title: RigidFormer: Learning Rigid Dynamics using Transformers    Authors: Zhiyang Dou, Minghao Guo, Haixu Wu, Doug Roble, Tuur Stuyck, Wojciech Matusik    Affiliations: MIT and Meta    arXiv: 2605.09196    Project page: people.csail.mit.edu/frankzydou/projects/RigidFormer    Code: Frank-ZY-Dou/Dynamics-Modeling, with the current README noting that code release is coming soon.    The Problem  Rigid-body dynamics matters for robotics, graphics, and embodied AI because manipulation and interaction are full of contact: blocks collide, objects slide, stacks fall, and articulated parts push against each other. Classical physics engines can simulate this well when we have clean meshes, calibrated material parameters, and carefully tuned contact models. In real perception pipelines, those assumptions are often weak. We may have point clouds, incomplete geometry, noisy segmentation, and approximate material parameters.  Prior learned simulators often rely on mesh connectivity or vertex-level message passing. That creates two bottlenecks:      Point clouds do not naturally come with triangle connectivity.    Vertex-level interaction becomes expensive as point resolution and object count grow.    RigidFormer’s answer is to treat rigid bodies as coherent objects. A rigid body may have many observed points, but its motion is low-dimensional. So the model should reason over object-level state and only use local point features where contact geometry matters.  Inputs and Outputs  The core dynamics model is:  \\[x_{t+1} = f_\\theta(x_{t-1}, x_t, \\Delta t)\\]  Each scene has \\(M\\) rigid objects. Object \\(i\\) is represented by a point set:  \\[x_t^{(i)} \\in \\mathbb{R}^{N_v^{(i)} \\times d}\\]  For each point, the paper builds a 12-dimensional feature vector from:      nearest-neighbor displacement to another object or the ground,    per-step position increment \\(x_t - x_{t-1}\\), used as a discrete velocity proxy,    offset from a reference shape/frame,    physical parameters \\([m, \\mu, \\epsilon]\\): mass, friction, and restitution.    The model also receives the integration step size \\(\\Delta t\\). In the articulated-body extension, it can additionally receive high-level control commands such as target speed, target movement direction, and target facing direction.  The output is the next state of every object as full-resolution point positions:  input:  object point clouds at t-1 and t, physics/contact features, step size, optional controlsoutput: object point clouds at t+1    Internally, the output path is more structured than direct point regression:  points -> object tokens -> object interaction Transformer       -> anchor queries -> anchor accelerations       -> Verlet integration -> candidate anchor positions       -> Kabsch rigid alignment -> rigid transform       -> all object points at t+1    This is important. The model predicts enough motion to determine a rigid transform, then applies that transform to all points. The final object cannot arbitrarily deform because the projection enforces rigid-body structure.  Method  1. Object-Level Interaction  RigidFormer first uses a PointNet-style encoder to compress each object’s point cloud into one object token. These object tokens, plus 16 learned register tokens, enter a Transformer decoder. Because there is no sequence-index positional embedding over objects, the object interaction stage is designed to be permutation-equivariant: reordering objects should reorder the outputs, not change the physics.  The Transformer is conditioned on the temporal step size through FiLM. The conditioning code uses \\((s, s^2)\\), mirroring the first-order and second-order terms that appear in motion integration. This lets a single model operate at different effective time resolutions.  2. Anchor-Based State Advance  Instead of predicting every point’s next position, RigidFormer chooses a small number of FPS anchors per object, with \\(N_a = 4\\) as the default. Each anchor attends to object tokens and receives local point features through Anchor-Vertex Pooling. This pooling aggregates nearby vertex features with a learnable distance kernel, giving anchors contact-local context without dense vertex-level attention.  The network predicts per-anchor acceleration. Candidate anchor positions are advanced with Verlet integration:  \\[\\hat{q}{t+1}^{(i,k)} =a_t^{(i,k)} \\Delta t^2 + 2q_t^{(i,k)} - q{t-1}^{(i,k)}\\]  The paper then aligns reference anchors to predicted anchors using Kabsch alignment and applies the resulting \\(SE(3)\\) transform to the full object point set.  3. Anchor-Based RoPE  The paper introduces Anchor-based Rotary Positional Embedding to inject 3D geometry into attention. The idea is to encode object geometry through sparse anchor positions rather than through a single centroid or all vertices. Mean-pooling anchor rotary descriptors makes the embedding invariant to anchor reindexing while still carrying shape extent and world-frame position information.  This is a small but meaningful design decision: a rigid object’s geometry matters for contact, but anchor identity should not become an arbitrary index dependency.  Results  On MOVi-A, MOVi-B, and MOVi-Sphere, RigidFormer matches or outperforms strong learned simulator baselines while using point inputs rather than mesh connectivity. The most relevant comparison is with HopNet, a strong prior rigid-dynamics baseline. On MOVi-B at 100 frames, the paper reports an improvement from 0.176 m / 17.91 deg to 0.161 m / 15.33 deg under the matched step-size-1 setting.  The step-size experiments are especially interesting. Larger step sizes reduce long-horizon autoregressive error because the model makes fewer rollout calls over the same physical horizon. On MOVi-B at 100 frames, the reported errors are:                    Step size        Position RMSE        Orientation RMSE                            1        0.161 m        15.33 deg                    5        0.136 m        13.55 deg                    10        0.115 m        10.85 deg              This is not just a numerical trick. It says the learned simulator can expose sparse long-horizon futures cheaply, which is useful for planning when the planner does not need every high-frequency contact frame.  The runtime comparison is also central to the paper’s claim:                    Method        ms/step        FPS                            HopNet        4228.7        0.2                    FIGNet        336.0        3.0                    RigidFormer        41.9        23.9              The paper also shows scalability on WreckingBall scenes with up to 217 objects and a preliminary extension to command-conditioned articulated bodies, where body parts are treated as interacting object-level components.  What I Like  The paper has a clean structural bias. Rigid objects should move rigidly; object interactions should be object-level; local contact still needs local geometry. RigidFormer maps those intuitions into architecture:      object tokens for global interaction,    anchors for low-dimensional state advance,    local anchor-vertex pooling for contact cues,    rigid projection for stability,    step-size conditioning for controllable rollout.    This makes the model feel less like a generic Transformer pasted onto physics and more like a learned simulator with the right pressure points exposed...."
    },{
      "title": "KEEP: A KV-Cache-Centric Memory Management System for Efficient Embodied Planning",
      "url": "/posts/2026/05/keep-paper-notes/",
      "date": "2026-05-13",
      "tags": ["Embodied Planning","KV Cache","Memory Systems","LLM Inference","Robotics","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  KEEP studies a very practical bottleneck in LLM-based embodied planning: memory is useful, but feeding long textual memory into the model at every planning step makes the agent slow. The planner may only need to output one action such as “find the potato” or “open the microwave”, yet before it can do that, the LLM repeatedly pays the prefill cost for a long prompt containing object states, robot state, past actions, and few-shot task examples.  The paper’s answer is to treat memory as KV cache rather than repeatedly as raw text. But embodied memory is awkward for naive caching because the world changes after actions. If the robot picks up an object, opens a drawer, places a plate in the sink, or turns on a faucet, part of the memory becomes stale and part of the cache is no longer valid. KEEP is therefore not simply a KV cache reuse paper. It is a paper about how to make KV reuse respect the structure and update pattern of embodied memory.  The system has three main pieces:      Static-Dynamic Memory Construction: stable memory is cached in larger semantic groups, while frequently changing memory is cached at finer segment granularity.    Multi-hop Memory Re-computation: the system recomputes KV for memories that matter to the current query, including memories that are only indirectly relevant through other memories.    Layer-balanced Memory Loading: KV loading from CPU memory to GPU is scheduled across layers to avoid idle bubbles caused by uneven recomputation ratios.    On ALFRED, KEEP keeps success rate close to full recomputation while substantially lowering TTFT. Compared with CacheBlend, the paper reports 4.13% success-rate improvement and 1.90x TTFT reduction on Qwen-2.5-32B INT4.  Paper Info  The paper is “KEEP: A KV-Cache-Centric Memory Management System for Efficient Embodied Planning” by Zebin Yang, Tong Xie, Baotong Lu, Shaoshan Liu, Bo Yu, and Meng Li. The arXiv version is 2602.23592. The paper is also listed by Microsoft Research, and the code is available at PKU-SEC-Lab/KEEP_Embodied_Memory.  The local codebase I read contains two implementations:      KEEP_transformers, a simpler Transformers implementation mainly for accuracy-oriented experiments.    KEEP_vllm, a modified vLLM implementation used for latency experiments and system-level KV loading/recomputation.    Why This Problem Exists  A language-based embodied planner usually runs in a loop. It observes the environment, updates memory, builds a prompt, asks the LLM to choose the next action, executes that action, then repeats. At first glance this looks like ordinary prompting. But the cost profile is different from a chat application.  The generated answer is short. In ALFRED-style planning, the output might be one skill selected from a constrained set: find an object, pick it up, put it down, slice something, open a receptacle, close it, toggle an appliance, and so on. The prompt, however, can be long. It may contain task rules, demonstrations, object memories, the current state of discovered objects, and the task instruction. Over many planning steps, this creates a strange imbalance: the model spends most of its time reading memory, not producing action.  This is why TTFT matters. If every step must prefill thousands of memory tokens before generating a tiny action, the agent becomes sluggish. In a real robot loop, that is not a cosmetic issue. Slow planning changes the feel of interaction, limits how often the agent can replan, and makes long-horizon tasks more expensive.  The tempting fix is KV caching. Once the model has processed a memory segment, why not store its key-value states and reuse them later? That is exactly the right instinct, but embodied memory immediately complicates it.  Suppose the memory says:  {\"object\": \"milk\", \"state\": \"cold\", \"position\": \"on the table\"}{\"object\": \"table\", \"position\": \"in the kitchen\"}    After the agent picks up the milk, the milk record changes. Depending on the representation, the table record may change too, because it no longer holds the milk. A prefix cache would only reuse the exact unchanged prefix, so a small edit can invalidate a long suffix. Fixed-size KV blocks are less brittle, but their boundaries are mechanical rather than semantic. A block might mix a stable task-history example with a volatile object state, or split two related object memories apart.  KEEP is built around this mismatch. Embodied memory is structured, semantically meaningful, and updated unevenly. A good cache strategy should be structured too.  The Memory Design Space  The paper places KEEP against several broad memory paradigms.  Parametric memory stores task experience in model weights through fine-tuning. It can be fast at inference time, but it is expensive to update and risks forgetting or overfitting.  Text memory keeps memory as prompt text. This is simple, transparent, and training-free, which is why it is common in embodied-agent prototypes. But the more memory you retrieve, the larger the prefill cost becomes.  Latent memory compresses history into learned or summarized states. This can be efficient, but compression may discard details that later become important. In embodied tasks, a tiny detail such as where a key was last seen may become the whole difference between success and failure.  KV-centric memory, KEEP’s choice, stores memory as the LLM’s own KV states. It is training-free like text memory, but aims to avoid repeated prefill. The challenge is scalability and correctness: KV tensors are large, may live partly in CPU RAM, and become invalid when their corresponding memory changes.  So the core contribution of KEEP is not merely “store KV”. The contribution is a management policy for deciding what granularity to cache, what to recompute, and when to load it.  Idea 1: Not All Memory Ages at the Same Speed  The first KEEP idea is Static-Dynamic Memory Construction. The intuition is almost mundane, which is part of why I like it: some memories change all the time, and some barely change at all.  Object-state memory is often dynamic. A potato can be whole, sliced, heated, moved, placed in a sink, or held by the robot. A drawer can be opened or closed. A mug can move from a countertop to a table. These records are exactly the ones that make embodied memory useful, but they are also the ones that break naive cache reuse.  Task-history examples are different. A few-shot example saying “Human: Put a cooked potato slice in the sink. Robot: find a knife, pick up the knife, …” does not change after the robot executes the current task. It is semantically rich and stable. Caching it as many tiny independent blocks would lose useful internal attention for no good reason.  KEEP therefore clusters memory using a sentence encoder, and then classifies memory groups by recent update frequency. A group is static if all its segments have stayed unchanged for the recent threshold window, set to t = 10 in the paper. Static groups are cached and retrieved as groups, preserving cross-attention inside the group. Dynamic groups are cached at individual segment granularity, so one updated object does not invalidate a large semantic neighborhood.  In other words, KEEP asks each memory group a practical question: are you old enough to be cached together, or active enough that I should isolate your changes?  This directly addresses a weakness of fixed-size block reuse. Fixed token blocks know nothing about object identity, task examples, or update frequency. KEEP uses the fact that embodied memory already has a semantic structure.  Idea 2: Importance Can Be Indirect  The second idea is Multi-hop Memory Re-computation. This part answers a subtle question: if we cached memory segments independently, how do we recover the cross-attention that the model would have used under full prefill?  One answer is to recompute a small fraction of tokens. Prior KV recomputation systems often choose positions heuristically, or choose tokens based on local discrepancy. KEEP argues that embodied planning needs a more query-aware and context-aware criterion.  Consider the instruction:  Open the door.    The directly relevant memory might be:  {\"object\": \"door\", \"state\": \"locked\"}    But the correct next action may depend on another memory:  {\"object\": \"key\", \"position\": \"on the table\"}    And that memory may itself depend on:  {\"object\": \"table\", \"position\": \"in the kitchen\"}    The table memory is not necessarily the most query-similar memory if we only compare it to “open the door”. But through the chain door to key to table, it becomes essential. This is the paper’s central reason for doing importance propagation.  At selected layers, KEEP starts from query-to-memory attention and assigns initial importance scores. It then selects a top ratio of memories, looks at attention from those selected memories to other memories, updates importance scores, and..."
    },{
      "title": "From Kepler to Newton: Explainable AI for Science",
      "url": "/posts/2026/05/from-kepler-to-newton-paper-notes/",
      "date": "2026-05-13",
      "tags": ["Explainable AI","AI for Science","Symbolic Regression","Scientific Discovery","Physics","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  “From Kepler to Newton: Explainable AI for Science” uses a historically elegant test case to argue for Explainable AI in scientific discovery. The authors ask: if modern AI had access only to Tycho Brahe and Kepler’s Mars observations, could it rediscover pieces of Kepler’s planetary laws and Newton’s inverse-square gravitation? Their answer is yes, but with an important caveat. Black-box neural networks can fit and augment sparse observations, symbolic regression can expose compact equations, but humans still need to interpret variables, invent concepts such as acceleration and force, and prevent the equations from being over-generalized.  Paper Info  The paper is “From Kepler to Newton: Explainable AI for Science” by Zelong Li, Jianchao Ji, and Yongfeng Zhang from Rutgers University. It appeared at the 2nd AI4Science Workshop at ICML 2022 and is available as arXiv:2111.12210. I reviewed the arXiv PDF and kept a local copy at assets/papers/2111.12210.pdf.  1. Motivation  The paper starts from the classic scientific loop:  observation -> question -> hypothesis -> prediction -> experiment and test  This is the hypothetico-deductive paradigm. It has served science well, but modern science increasingly has too much data for researchers to manually inspect every possible pattern. The authors frame AI as a way to help with this bottleneck, especially in the step where a scientist proposes useful hypotheses from observations.  Their proposed alternative is an Explainable AI-based, hypothesis-free discovery loop:  observation -> question -> model learning -> model interpretation -> prediction -> experiment and test  The key substitution is not simply “use AI.” It is more specific: use black-box models for accurate prediction and data augmentation, then use white-box explanation methods to translate the learned model into human-readable equations or rules. In the authors’ framing, AI can generate candidate hypotheses, but human scientists still decide what those hypotheses mean.  2. Why Kepler and Newton?  The historical analogy is clean:      Tycho Brahe represents observation: careful astronomical measurements.    Johannes Kepler represents model learning: discovering regularities in planetary motion.    Isaac Newton represents explanation: explaining those regularities through force, acceleration, and gravitation.    The paper asks what this pipeline would look like if part of Kepler and Newton’s work were assisted by modern AI. To keep the setting historically constrained, the experiments use Mars observation data from Kepler’s Astronomia Nova, not modern high-precision planetary data. That choice matters because the paper wants to test whether AI can rediscover insight from the kind of limited but high-quality data available around the scientific revolution.  3. Model Learning: Black-Box Prediction and Data Augmentation  The first model is deliberately simple: a three-layer MLP with hidden size 100. The authors use the same neural-network structure across experiments instead of hand-designing a physics-aware architecture.  For Kepler’s first law, the network learns a function:\\[r = NN(\\theta)\\]  where (r) is the Sun-Mars distance and (\\theta) is Mars’ angular position in heliocentric ecliptic coordinates. The original dataset has only 28 observations, so the neural network is used both for smooth fitting and for generating 1,000 augmented samples.  This is the “black-box AI” part of the paper. It can make useful predictions, and it can turn sparse historical data into a denser function-like object. But by itself, the MLP does not tell us why Mars moves as it does. A nested set of learned weights is not the same as a scientific explanation.  4. Model Interpretation: Symbolic Regression and Kepler’s First Law  The explanation step uses symbolic regression, implemented with TuringBot. The goal is to translate the black-box function (r = NN(\\theta)) into an explicit expression (r = f(\\theta)).  Using cosine, addition, multiplication, and division as base operations, symbolic regression finds a compact expression:\\[r = \\frac{1.51033}{1 + 0.0927177 \\cos(\\theta + 0.544536)}\\]  This matches the polar form of an ellipse:\\[r = \\frac{l}{1 + \\epsilon \\cos(\\theta)}\\]  So the AI pipeline recovers Kepler’s first law in an interpretable form: Mars follows an elliptical orbit with the Sun at a focus. Even better, the coefficient (0.0927177) can be read as Mars’ eccentricity. The paper reports that this is within 0.1% of Kepler’s own value and about 0.7% from the modern value, reasonable given that the experiment intentionally uses 400-year-old data.  The phase shift (\\theta + 0.544536) also has a physical interpretation. It suggests that Mars reaches perihelion around August, consistent with historical records of close Mars oppositions.  5. From Kepler Toward Newton  The second experiment adds time. The authors train another MLP:\\[\\theta = NN(t)\\]  where (t) is time normalized into one Mars orbital period. This is useful because the exact inverse relationship from time to angle is hard to express directly; the paper notes that the known orbital mechanics equation gives (t) as a function of (\\theta), but (\\theta) as a function of (t) is not easy to write in closed form.  Because the neural model is smooth, it can still support analysis. The authors estimate angular velocity (\\omega) by finite differences and combine it with (r), then ask symbolic regression to search for relations among (r), (r^2), (r^3), (\\omega), (\\omega^2), and (\\omega^3).  The compact discovered rule is:\\[r^3 \\omega^2 = 0.000298491 \\ \\mathrm{AU^3 day^{-2}}\\]  Modern physics recognizes the same structure as:\\[r^3 \\omega^2 = GM\\]  The reported relative error is about 0.8% compared with the modern value in AU and days. This is the paper’s strongest example of the model-learning plus model-interpretation workflow: neural networks make the time-angle relationship usable, symbolic regression identifies a simple invariant, and that invariant points toward a real physical law.  6. The Human Role: Meaning Is Not Automatic  The most interesting part of the paper is not just that symbolic regression finds equations. It is the authors’ insistence that equations are not automatically science.  For example, symbolic regression can output:\\[r^3 \\omega^2 = c\\]  But to reach Newtonian gravitation, a human still has to recognize that (a = r\\omega^2) is centripetal acceleration, then reorganize the expression as:\\[a \\propto \\frac{1}{r^2}\\]  and finally connect force to acceleration through (F = ma), yielding:\\[F \\propto \\frac{1}{r^2}\\]  This is the conceptual leap. AI can narrow the hypothesis search space and expose useful symbolic structure, but the meaning of “acceleration,” “force,” and “gravity” is assigned through human scientific understanding.  The paper also warns against over-reading the result. The Mars-only relationship (r^3\\omega^2=c) is not itself Kepler’s third law, because Kepler’s third law is a cross-planet statement about orbital period and mean distance. Still, it points in the right direction: if one considers (\\bar{\\omega}=2\\pi/T), the expression suggests a relation between (\\bar{r}^3) and (T^2), which could guide the search once data from more planets is included.  7. Strengths      The historical case study is unusually readable. It gives the paper a clear scientific-discovery pipeline instead of only a machine-learning benchmark.    The division between prediction and explanation is sharp. The authors do not pretend that a high-accuracy MLP is a scientific theory.    Symbolic regression is used in a natural role: translating a fitted model into candidate laws.    The paper is careful about the human scientist’s role. It treats AI-generated equations as hypotheses and search-space reducers, not as self-sufficient knowledge.    8. Limitations      The experiment is more a demonstration than a full discovery engine. The search space is still strongly shaped by the authors’ chosen variables and base functions.    The physical domain is clean and low-dimensional. Real AI-for-science settings may involve noisy, high-dimensional, partially observed systems where symbolic regression is much harder.    The paper depends on human interpretation at key points, which is philosophically honest but also means the AI pipeline is not autonomous.    The singularity discussion is provocative but less technically grounded than the Kepler/Newton experiments. The most durable contribution is the concrete XAI-for-science workflow.    Takeaways  This paper is valuable because it makes a precise distinction: prediction is not explanation, and explanation is not yet meaning. Black-box models can be powerful scientific instruments, especially for fitting sparse data and creating differentiable approximations. Symbolic regression can turn those instruments into candid..."
    },{
      "title": "Morphologically Symmetric Reinforcement Learning for Ambidextrous Bimanual Manipulation",
      "url": "/posts/2026/05/symdex-paper-notes/",
      "date": "2026-05-12",
      "tags": ["Robotics","Reinforcement Learning","Bimanual Manipulation","Equivariance","Sim-to-Real","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  SYMDEX asks a clean question: if a bimanual robot is physically symmetric, why should one arm have to relearn what the other arm already discovered? The paper turns bilateral morphology into an RL prior for ambidextrous manipulation. It decomposes a complex bimanual task into per-hand subtasks, trains each subtask policy with a symmetry-equivariant actor and symmetry-invariant critic, then distills those specialist policies into a global ambidextrous policy.  The key design is not merely data augmentation. SYMDEX encodes the robot’s reflection group directly into the policy class:  \\[g \\triangleright_A \\pi(o) = \\pi(g \\triangleright_O o)\\]  In practice, this means a reflected scene should produce a correspondingly reflected action. On six Isaac Lab bimanual dexterous tasks, SYMDEX reaches more than 80% success and outperforms PPO baselines. The authors also report zero-shot real-world transfer on box-lift and table-clean, with curriculum learning playing a major role.  Paper Info      Title: Morphologically Symmetric Reinforcement Learning for Ambidextrous Bimanual Manipulation    Authors: Zechu Li, Yufeng Jin, Daniel Ordonez Apraez, Claudio Semini, Puze Liu, Georgia Chalvatzaki    Venue: CoRL 2025    arXiv: 2505.05287    Project page: supersglzc.github.io/projects/symdex    Codebase read here: PyTorch / Isaac Lab implementation of SYMDEX with ESCNN and MorphoSymm    1. Motivation  Humans can mirror many gross manipulation skills between left and right hands, but fine dexterity often develops a dominant side. Robots do not need this kind of handedness. If the hardware is bilaterally symmetric, a robot should be able to choose whichever arm is better placed for the current scene.  The problem is that bimanual RL is hard in exactly the places where this symmetry would help:      the observation and action spaces are high-dimensional,    both arms contribute to one task-level outcome,    reward shaping becomes messy when one arm succeeds and the other fails,    ambidexterity turns a fixed two-arm controller into a task-assignment problem.    SYMDEX addresses this by making the learning problem smaller and more structured: learn one policy per subtask, enforce morphology-aware equivariance, then distill the behavior into a single deployable policy.  2. Method  The paper formulates bimanual manipulation as a multi-task, multi-agent POMDP. Each robot arm is an agent, and each manipulation role is a subtask. In the running example, one arm holds a bowl while the other operates an egg beater. Under a left-right reflection, the roles should swap.  For a symmetry group \\(G\\), the paper assumes the POMDP dynamics, rewards, and initial-state distribution are invariant under group actions. This gives the usual policy equivariance and value invariance conditions:  \\[g \\triangleright_A \\pi^(\\sigma(s)) = \\pi^(\\sigma(g \\triangleright_S s))\\]  \\[V^(\\sigma(s)) = V^(\\sigma(g \\triangleright_S s))\\]  SYMDEX uses this structure in three steps.  Subtask Decomposition  Instead of training one monolithic 44-DoF bimanual policy, SYMDEX trains 22-DoF single-arm policies. Each policy sees the assigned arm state and task-specific object state. This reduces action dimensionality and gives each policy a cleaner reward signal.  Equivariant PPO  Each subtask actor is a \\(G\\)-equivariant neural network, while each critic is \\(G\\)-invariant. The actor should transform actions consistently when observations are reflected; the critic should assign the same value to symmetric states.  The intuition is simple: a left hand reaching in a mirrored workspace should behave like a transformed version of the right hand reaching in the original workspace.  Global Policy Distillation  After the subtask policies are trained, they generate a dataset of state-action pairs. A student policy is then trained to imitate the combined behavior. This student is also equivariant, but unlike the subtask policies, it observes the global non-privileged state and learns the task-arm assignment implicitly.  This is the deployable policy: one global ambidextrous controller, trained from specialist teachers.  3. Curriculum for Sim-to-Real  The paper uses a curriculum with two practical pieces:      Randomization curriculum: begin with scene-level symmetry randomization, then gradually introduce object pose and physical-parameter variation.    Safety curriculum: introduce collision and energy penalties later, after the policy has learned useful task behavior.    This matters. In real-world evaluation, an equivariant Gaussian policy without curriculum drops sharply, while the curriculum-trained version transfers much better.  4. Experiments  SYMDEX is evaluated on six simulated Isaac Lab tasks:                    Task        Main challenge                            Box-lift        coordinated symmetric lifting                    Table-clean        two-arm sweeping / object handling                    Drawer-insert        asymmetric object and drawer roles                    Threading        coordinated precise insertion                    Bowl-stir        one arm stabilizes, the other manipulates                    Handover        role-specific grasp and transfer              The paper compares against five PPO-style baselines: monolithic equivariant PPO, independent PPO, equivariant independent PPO, a centralized-critic variant, and a symmetry-augmentation variant.  The headline simulation result is that SYMDEX learns all six tasks and exceeds 80% success, while the baselines fail especially when the two arms must perform different roles. This supports two claims at once: task decomposition helps credit assignment, and architectural equivariance is stronger than only augmenting data.  For distillation, the paper compares:                    Student        Box        Table        Drawer        Threading        Bowl        Handover                            Gaussian policy        0.83        0.74        0.69        0.62        0.75        0.54                    Equivariant Gaussian policy        0.89        0.83        0.87        0.63        0.87        0.86                    Equivariant Diffusion policy        0.91        0.84        0.87        0.60        0.88        0.68              Both equivariant students improve over the vanilla Gaussian student. Interestingly, the Gaussian equivariant student is more robust than the diffusion variant in the real world, which the authors attribute to the homogeneous teacher-generated dataset and imperfect state estimation at deployment time.  5. Codebase Reading  The repository is a compact Isaac Lab project. The public entry points are straightforward:  train.py       # train SYMDEX with Hydra configs and W&B loggingvisualize.py   # load saved actors and execute policies in simulationrandom_actions.pysymdex/cfg/symdex/env/tasks/symdex/algo/symdex/utils/    The README exposes six tasks:  insertDrawer, boxLift, pickObject, stirBowl, threading, handover    The default training command is:  python train.py task=insertDrawer save_model=True    The most important implementation pieces are:  Symmetry Configuration  symdex/cfg/task/base.yaml defines the reflection group:  group_label: C2symmetric_envs: Truepermutation_Q_js: ...reflection_Q_js: ...permutation_student_Q_js: ...reflection_student_Q_js: ...    For the single-arm policy, the joint representation keeps the 22-DoF order and applies joint-specific sign flips. For the student/global policy, permutation_student_Q_js swaps the two 22-DoF halves, while reflection_student_Q_js applies the corresponding signs.  Equivariant Networks  symdex/utils/symmetry.py builds the ESCNN group and registers representations for joint space, tangent joint space, Euclidean vectors, pseudo-vectors, and flattened rotations. symdex/algo/network/emlp.py then uses those field types to build equivariant MLPs.  The implementation is particularly nice because it handles both actor and critic cases:      if the output representation is non-trivial, the EMLP is equivariant;    if the output is trivial, the EMLP pools invariant features and behaves as an invariant function.    That matches the paper’s actor/critic split almost directly.  PPO Agent  symdex/algo/eqs.py defines AgentSYMDEX. It creates two actor-critic pairs:  actor, criticactor_left, critic_left    When same_policy is enabled, these can share parameters. Otherwise they are optimized separately, which mirrors the paper’s dedicated subtask-policy setup.  During rollout, the agent:      reads the environment’s symmetry_tracker,    slices each subtask observation through SymmetryManager.get_multi_agent_obs,    samples actions from the two actors,    combines or swaps actions through get_execute_action,    splits detailed rewards back into subtask rewards through get_multi_agent_rew,    runs PPO updates for the right and left buffers.    This is the code-level vers..."
    },{
      "title": "HumDex: Humanoid Dexterous Manipulation Made Easy",
      "url": "/posts/2026/05/humdex-paper-notes/",
      "date": "2026-05-12",
      "tags": ["Humanoid Robotics","Dexterous Manipulation","Teleoperation","Imitation Learning","Human Data","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  HumDex is a portable teleoperation and imitation-learning system for humanoid whole-body dexterous manipulation. Its core idea is practical: replace infrastructure-heavy or occlusion-prone tracking with inertial full-body and hand tracking, learn a lightweight hand retargeter for 20-DoF dexterous hands, and use abundant human demonstrations as a pretraining source before fine-tuning on a small amount of robot data. The result is a system that can collect better demonstrations faster, solve tasks that vision-based teleoperation struggles with, and improve policy generalization to new object positions, categories, and backgrounds.  Paper Info  The paper is “HumDex: Humanoid Dexterous Manipulation Made Easy” by Liang Heng, Yihe Tang, Jiajun Xu, Henghui Bao, Di Huang, and Yue Wang, from USC Physical Superintelligence Lab and WorldEngine AI. It is available as arXiv:2603.12260, with the code released at physical-superintelligence-lab/HumDex. The local codebase I reviewed includes the teleoperation pipeline, Wuji hand retargeting/training utilities, ACT policy learning scripts, and documentation for real/human data collection.  1. Problem and Motivation  Humanoid dexterous manipulation has a data problem. Imitation learning can learn impressive long-horizon manipulation behaviors, but collecting high-quality demonstrations on a humanoid with dexterous hands is slow, brittle, and hardware-dependent. Optical motion capture and exoskeleton systems can be accurate but require dedicated infrastructure. VR or vision-based systems are more portable but suffer from self-occlusion, especially when the operator grasps tools or performs fine finger motions.  HumDex attacks this bottleneck from two directions. First, it builds a portable whole-body dexterous teleoperation stack based on IMU tracking, so operators can move naturally without keeping hands inside a headset camera’s field of view. Second, it treats human demonstrations as a cheap source of diversity: the robot policy first learns broad visual and motion priors from human data, then adapts to the robot embodiment with robot teleoperation data.  2. System Overview  HumDex combines three layers:      Wearable tracking. The system supports inertial full-body tracking, including a commercial 15-node Vdmocap/VIRDYN-style setup and a low-cost SlimeVR-based alternative. For hands, it supports inertial gloves such as Vdhand and Manus.    Whole-body control. Human body motion is retargeted through a pelvis-centric General Motion Retargeting formulation, then streamed into low-level humanoid controllers such as TWIST2 or SONIC. This keeps locomotion and balance handled by robust existing controllers while providing high-level targets.    Dexterous hands. Each Wuji hand has 20 actuated DoFs. Instead of controlling hands as binary open/close grippers, HumDex maps five fingertip positions to full 20-DoF joint targets.    The implementation mirrors this modular design. The repo exposes a unified teleoperation entry point:  bash scripts/teleop.sh --policy twist2 --body slimevr --hand vdhandbash scripts/teleop.sh --policy sonic --body vdmocap --hand manus    That code-level detail matters: the paper is not just proposing a concept, but a composable stack where body source, hand source, and low-level controller can be swapped through configuration.  3. Learning-Based Hand Retargeting  Dexterous hand retargeting is one of the paper’s most useful engineering choices. The inertial glove provides five fingertip positions, represented as a 15D vector in the glove wrist frame. HumDex trains a small MLP:\\[f_\\theta: \\mathbb{R}^{15} \\rightarrow \\mathbb{R}^{20}\\]  to predict the 20-DoF Wuji hand joint vector. The training objective is simple supervised regression:\\[\\min_\\theta \\mathbb{E}_{(p,q)\\sim D}\\left[\\|f_\\theta(p)-q\\|_2^2\\right]\\]  The labels come from an offline optimization-based retargeting process, but runtime inference is neural and constant-time. In the appendix, the paper describes a finger-wise MLP with five sub-networks, each mapping one fingertip’s 3D position to four finger joints. The calibration cost is modest: about 20k frames, or less than 20 minutes of recording.  This design is attractive because it turns a per-frame constrained optimization problem into a fast learned mapping, while still using optimization to produce the initial supervised targets. In task tests, glove plus learned retargeting performs especially well on dexterity-heavy subtasks such as scanner triggering and doll grasping.  4. Two-Stage Imitation Learning  The policy backbone is ACT with a ResNet-18 visual encoder. Observations include egocentric RGB from a RealSense camera plus proprioceptive state; actions include whole-body targets and bimanual hand targets.  The central difficulty is that human demonstrations do not have true robot proprioception. HumDex approximates the missing robot state with the previous action, based on the observation that robot actions correspond closely to next-step states. Then it trains sequentially:      Human pretraining. Train on diverse human demonstrations to learn visual invariances and broad motion priors.    Robot fine-tuning. Fine-tune on robot teleoperation data to adapt the policy to the Unitree G1 plus Wuji hand embodiment.    This sequential design is important. The paper reports that naively mixing human and robot data fails to converge, likely because similar visual states map to conflicting human-style and robot-style actions. Pretrain-then-finetune avoids that conflict: human data teaches diversity first; robot data teaches embodiment-specific execution second.  The repo’s data tools match this story. Human data preprocessing explicitly approximates proprioception with previous-frame action, and act/imitate_episodes.py supports sequential training with multiple datasets.  5. Experiments and Main Results  The evaluated tasks are deliberately hard for a humanoid with hands:      Scan & Pack: hold a scanner, pull its trigger, scan a toy, pack it into a bag, and hand the bag over.    Hang Towel: coordinate both hands to thread a towel through a hanger and return the hanger.    Open Door: press a real door handle while walking forward.    Place Basket on Shelf: squat, pick up a basket, stand, rotate, and place it.    Pick Bread: grasp a deformable-like object and place it into a basket.    Compared with a vision-based teleoperation baseline, HumDex improves the common-task data collection time from 59.8 minutes to 44.3 minutes for 60 episodes, a 26% efficiency gain. It also improves teleoperation success from 74.6% to 91.7%, and policies trained on its demonstrations improve from 57.5% to 80.0% success on the shared task set. The baseline cannot complete Scan & Pack because scanner grasping occludes the hand; HumDex succeeds because inertial gloves do not depend on visual hand visibility.  For generalization on Pick Bread, the robot-data-only policy performs well in the seen setting but drops sharply under distribution shift:                    Policy        Seen        Unseen Position        Unseen Object        Unseen Background                            Robot data only        29/30        12/30        10/30        9/30                    HumDex two-stage        30/30        21/30        20/30        25/30              The strongest gain is background generalization, where human pretraining improves from 9/30 to 25/30. This supports the paper’s main claim: diverse human data is valuable not because it can be replayed directly on the robot, but because it teaches robust perception and high-level action priors.  6. Codebase Notes  The repository is unusually implementation-facing for a paper release. The reviewed code and docs expose several practical pieces:      deploy_real/config/teleop.yaml centralizes runtime, network, retargeting, adapter, and policy settings.    scripts/teleop.sh provides the unified selector interface for controller, body tracker, and hand tracker.    deploy_real/adapters/ separates body sources such as Vdmocap, SlimeVR, and Xsens from hand sources such as Vdhand and Manus.    wuji_policy/training/ contains the learned hand policy stack, including dataset, model, trainer, loss, and export logic.    act/convert_to_hdf5.py, act/scripts/convert_human_data.py, and act/imitate_episodes.py support policy learning from robot and human datasets.    This is the most encouraging part of HumDex as a research artifact: the paper’s abstractions appear as runnable interfaces rather than only diagrams.  7. Strengths and Limitations  Strengths. HumDex is strong because it solves a real systems bottleneck, not just a modeling subproblem. The IMU-first design directly targets the occlusion failure mode of vision-based teleoperation. The learned hand retargeter is simple, fast, and calibratable. The two-stage learning setup is also a clean answer to a subtl..."
    },{
      "title": "EquiBim: Learning Symmetry-Equivariant Policy for Bimanual Manipulation",
      "url": "/posts/2026/05/equibim-paper-notes/",
      "date": "2026-05-12",
      "tags": ["Robotics","Bimanual Manipulation","Imitation Learning","Equivariance","LeRobot","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  EquiBim is a simple but useful idea for bimanual imitation learning: if a task is left-right symmetric, then the policy should behave consistently when the observation is mirrored and the two arms are swapped. Instead of building a special equivariant neural architecture, the paper adds a prediction-level regularization term:  \\[L_{sym} = |\\pi(S(O)) - S(\\pi(O))|_2^2\\]  This makes EquiBim model-agnostic. It can be attached to image policies, point-cloud policies, joint-space actions, or end-effector actions, as long as the symmetry transform \\(S\\) is defined for both observations and actions. In simulation on RoboTwin and real-world experiments on a dual LeRobot SO101 setup, the method improves average success and robustness under mirrored or shifted object distributions.  From the codebase side, the repository implements the idea inside LeRobot’s ACT policy: mirror the observation, run the same policy again, mirror the original prediction, and penalize the mismatch.  Paper Info      Title: EquiBim: Learning Symmetry-Equivariant Policy for Bimanual Manipulation    Authors: Zhiyuan Zhang, Aditya Mohan, Seungho Han, Wan Shou, Dongyi Wang, Yu She    Affiliations: Purdue University, University of Arkansas    arXiv: 2603.08541    Project page: zhangzhiyuanzhang.github.io/equibim-website    Codebase: a LeRobot-based implementation for bimanual SO101 manipulation and ACT symmetry loss    1. Motivation  Bimanual robots have a structural prior that many learned policies only use implicitly: the left and right arms are often physically symmetric, and many tasks remain equivalent if we mirror the workspace and exchange the two arms.  Standard behavior cloning does not enforce this. If the dataset has more clean demonstrations on one side than the other, or if the test object appears in a mirrored pose, a policy can produce inconsistent left/right behavior even when the mirrored strategy should be valid. This is especially visible in dual-arm manipulation because coordination errors are not just visual mistakes; they become timing, grasping, and role-assignment failures.  EquiBim’s claim is that this symmetry should be an explicit training signal. The nice part is that the signal lives at the policy-output level, so it does not require redesigning the backbone.  2. Method  The policy is trained by behavior cloning. Given an observation history \\(O\\), the policy predicts a future action sequence:  \\[\\pi: O \\rightarrow A\\]  EquiBim defines a symmetry transformation \\(S\\) over both observation and action spaces. For a bimanual setup, \\(S\\) corresponds to a left-right reflection plus an exchange of the two arms.  The desired equivariance is:  \\[\\pi(S(O)) \\approx S(\\pi(O))\\]  So the symmetry loss is:  \\[L_{sym} = |\\pi(S(O)) - S(\\pi(O))|_2^2\\]  The full training objective keeps the ordinary imitation loss and adds this consistency term. Importantly, both branches use the same policy parameters. The model is not asked to learn a new task; it is asked to be self-consistent under a physically meaningful transformation.  3. Symmetry Across Modalities  The paper handles several observation/action combinations:                    Component        Symmetry transform                            RGB image        horizontal flip                    Point cloud        transform to image/camera-aligned frame, reflect along lateral axis, transform back                    End-effector pose        reflect position and orientation consistently in the control frame                    Joint action        swap left/right arms and apply joint-specific sign flips from the robot kinematics              This is the main reason the method is practical. The regularizer is the same, while the implementation of \\(S\\) changes with the sensor and action representation.  4. Results  RoboTwin Simulation  The paper evaluates eight symmetric bimanual tasks in RoboTwin: Beat Block Hammer, Click Alarmclock, Handover Block, Move Can Pot, Pick Dual Bottles, Place Empty Cup, Stamp Seal, and Press Stapler.  Average success rates improve across all tested observation/action settings:                    Backbone / Setting        Baseline        + EquiBim        Gain                            DP, Image + Joint        34.1        43.6        +9.5                    DP, Image + EE        37.3        40.0        +2.7                    DP3, Point Cloud + Joint        73.5        77.9        +4.4                    DP3, Point Cloud + EE        74.5        77.8        +3.3              The biggest gain is in the weakest geometric setting, image observations plus joint-space actions. That makes intuitive sense: images do not explicitly encode 3D structure, and joint actions are less directly spatial than end-effector poses. The symmetry loss supplies a missing structural prior.  The paper also reports that some tasks can drop under EquiBim, especially when the optimal strategy contains useful role asymmetry. Handover Block and Pick Dual Bottles are examples where timing, contact, or grasp ordering can make the mirrored policy less universally correct. This is an important caveat: symmetry regularization helps when task-level symmetry dominates, but it can fight the data when the task only looks symmetric geometrically.  Real-World SO101 Experiments  The real-world setup uses two LeRobot SO101 arms with a centered Logitech C920x camera. The top-down camera arrangement makes horizontal image flips line up naturally with the workspace’s left-right direction.  The paper evaluates:      Banana Handover    Drumstick Hook Hanging    Toy Chicken Hook Hanging    With 50 demonstrations per task and 10 evaluation trials per object, ACT + EquiBim improves robustness, especially under distribution shifts:                    Task / Distribution        ACT        ACT + EquiBim                            Banana, training distribution        3/10        6/10                    Banana, shifted distribution        0/10        5/10                    Drumstick, shifted distribution        1/10        4/10                    Toy Chicken, shifted distribution        4/10        6/10              The banana result is the clearest: when the object orientation and side placement are mirrored relative to training, vanilla ACT fails completely, while the equivariant version keeps half the trials successful.  5. Codebase Reading  The repository is built on LeRobot and adds a practical bimanual SO101 workflow:  bimanual_teleop.pybimanual_teleop_camera.pybimanual_data_collection.pytrain_bimanual.shbimanual_capture_home_pose.pybimanual_inference.py    The symmetry switch is exposed in ACT config:  use_sym_loss: bool = Falseeq_loss_weight: float = 0.1    The main implementation lives in src/lerobot/policies/act/modeling_act.py.  The code defines a 6-dimensional per-arm sign vector:  JOINT_SIGN = torch.tensor([-1, +1, +1, +1, -1, +1])    Then mirror_state and mirror_action split the 12-dimensional bimanual vector into left and right halves, apply the sign transform, and concatenate the swapped result:  [left_arm, right_arm] -> [signed_right_arm, signed_left_arm]    For images, the implementation simply flips along the width dimension:  torch.flip(img, dims=[-1])    During training, ACT first computes the ordinary L1 action loss. If use_sym_loss is enabled, it builds a mirrored batch, predicts actions on that mirrored input, and adds:  eq_loss = mse(mirror_action(actions_hat.detach()), actions_hat_sym)loss = l1_loss + eq_loss_weight * eq_loss    The detach() is a small but meaningful implementation choice: the original prediction acts like the target for the mirrored branch, so the symmetry term regularizes the mirrored pass without letting both sides chase each other in the same backward path.  6. Strengths and Limitations  Strengths. EquiBim is easy to add to existing imitation learning systems, and the inductive bias is physically meaningful. The method also gives a concrete recipe for using stronger demonstrations on one side to regularize weaker demonstrations on the other side. For low-cost bimanual platforms where data is limited, this is a very practical advantage.  Limitations. The method depends on the symmetry transform being correct. If the camera is not centered, the robot mounting is not symmetric, the object interaction is role-specific, or the task has real left/right asymmetry, the loss can penalize useful behavior. The paper is honest about this through the per-task drops in simulation. Also, the real-world evaluation is promising but still small: 10 trials per object and three task families.  7. Takeaways  EquiBim is a good reminder that not every robotics improvement needs a larger model. Sometimes the right move is to encode a physical prior that the robot already gives us for free.  For bimanual learning, bilateral symmetry is one of those priors. The paper’s contribution is to turn it into a model-..."
    },{
      "title": "LDA-1B: Scaling Latent Dynamics Action Model via Universal Embodied Data Ingestion",
      "url": "/posts/2026/04/lda-1b-paper-notes/",
      "date": "2026-04-29",
      "tags": ["Robotics","World Models","Latent Dynamics","DINO","Embodied AI","Robot Learning","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  LDA-1B is a 1.6B-parameter robot foundation model that learns policy, forward dynamics, inverse dynamics, and visual forecasting in a shared DINO latent space. Instead of predicting future pixels, it predicts future structured visual features extracted by a frozen DINOv3 encoder, which lets the model focus more on object structure, contact-relevant regions, and action-induced state changes.  The paper’s central claim is not that DINO itself is a physics engine. Rather, DINO provides a cleaner and more semantic visual state space, making it easier for a large action-conditioned dynamics model to learn interaction physics from heterogeneous embodied data. LDA-1B scales this recipe to 30k+ hours of robot and human interaction data and reports strong results across simulation, real-world gripper manipulation, dexterous manipulation, and long-horizon tasks.  Paper Info      Title: LDA-1B: Scaling Latent Dynamics Action Model via Universal Embodied Data Ingestion    Authors: Jiangran Lyu*, Kai Liu*, Xuheng Zhang*, Haoran Liao, Yusen Feng, Wenxuan Zhu, Tingrui Shen, Jiayi Chen, Jiazhao Zhang, Yifei Dong, Wenbo Cui, Senmao Qi, Shuo Wang, Yixin Zheng, Mi Yan, Xuesong Shi, Haoran Li, Dongbin Zhao, Ming-Yu Liu, Zhizheng Zhang, Li Yi, Yizhou Wang, He Wang    Affiliations: Peking University, Galbot, CASIA, BAAI, Tsinghua University, Sun Yat-sen University, NVIDIA    Date: 2026-02-12    arXiv: 2602.12215    Project page: pku-epic.github.io/LDA    1. Motivation  Most recent robot foundation models are still heavily behavior-cloning-centric: collect high-quality demonstrations, then train a policy to imitate expert actions. This works, but it creates a bottleneck:      High-quality robot data is expensive.    Many large embodied datasets are heterogeneous, noisy, or actionless.    Pure BC discards low-quality trajectories and actionless videos, even though they may contain useful dynamics knowledge.    Pixel-space world models waste capacity on appearance details such as lighting, texture, and background clutter.    The paper argues that robot pretraining should use data according to its quality and supervision type:                    Data Type        Main Use                            High-quality robot/human demonstrations        Policy + dynamics + visual forecasting                    Lower-quality trajectories        Dynamics + visual forecasting, not direct policy imitation                    Actionless human videos        Visual forecasting and scene-transition priors              This is what the authors call Universal Embodied Data Ingestion.  2. Core Idea  LDA-1B follows a Unified World Model style formulation. Given current observation \\(o_t\\), future observations \\(o_{t+1:t+k}\\), action chunk \\(a_{t+1:t+k}\\), and language instruction \\(\\ell\\), it jointly learns:      Policy: \\(p(a_{t+1:t+k} \\mid o_t, \\ell)\\)    Forward dynamics: \\(p(o_{t+1:t+k} \\mid o_t, a_{t+1:t+k}, \\ell)\\)    Inverse dynamics: \\(p(a_{t+1:t+k} \\mid o_{t:t+k}, \\ell)\\)    Visual planning / forecasting: \\(p(o_{t+1:t+k} \\mid o_t, \\ell)\\)    The important twist is that future visual states are represented as DINO latent features, not pixels or VAE image latents.  RGB observation  -> frozen DINOv3-ViT-s encoder  -> structured visual latent tokens  -> MM-DiT predicts future DINO latents and/or action chunks    During pretraining, the DINO encoder and VLM are frozen. The trainable part is the MM-DiT plus action encoder/decoder.  3. Why DINO Latent Helps  The paper makes a strong case that representation quality is a scaling bottleneck for world models. Pixel-space or VAE-space prediction entangles many things:      object geometry    texture    background    illumination    camera viewpoint    task-relevant contact dynamics    This makes dynamics learning harder because the model must spend capacity predicting visual details that may not matter for control.  DINO features are different. They tend to preserve semantic and spatial structure while suppressing irrelevant low-level appearance variation. For robot manipulation, this means the latent state is closer to the things we care about:      where the object is    whether the object has moved    which surface is contact-relevant    whether a tool, hand, or gripper is aligned with the object    whether the future state is coherent under the action    My read: DINO latent is not physics by itself; it is a better coordinate system for learning physics-like dynamics. The physics comes from training the temporal/action model on lots of interaction sequences.  4. Model Architecture  LDA uses a Multi-Modal Diffusion Transformer (MM-DiT). It jointly denoises action chunks and future DINO latent tokens.  The model receives:      current observation and language encoded by a VLM    current / past DINO visual features    noisy future action chunks    noisy future DINO latent tokens    task embeddings indicating policy, forward dynamics, inverse dynamics, or visual forecasting    diffusion timestep embeddings    Each MM-DiT block uses multi-modal self-attention over action and visual tokens, with modality-specific projections and output heads. Language tokens are injected through cross-attention.  Key model details from the appendix:                    Component        Value                            VLM        Qwen3-VL-4B-Instruct                    Observation encoder        DINOv3-ViT-s                    Hidden size        1536                    Layers        16                    Attention heads        32                    Image shape        224 x 224 x 3                    DINO latent shape        14 x 14 x 384                    Action chunk        16                    Pretraining batch        32 x 48                    Pretraining hardware        48 NVIDIA H800 GPUs                    Pretraining iterations        400k                    Compute cost        4,608 GPU hours              5. EI-30k Dataset  The paper builds EI-30k, an embodied interaction dataset with more than 30,000 hours of human and robot trajectories.                    Category        Hours                            Real-world robot data        8.03k                    Simulated robot data        8.6k                    Ego human data with actions        7.2k                    Actionless ego human videos        10k                    Total        30k+              The dataset is standardized into the LeRobot format and includes aligned hand-centric action representations:      robot 6-DoF end-effector pose plus gripper width or dexterous hand joints    human 6-DoF wrist pose plus MANO hand parameters    camera extrinsics retained to decouple hand motion from egocentric camera motion    quality labels so lower-quality trajectories can be used for dynamics without forcing policy imitation    This dataset engineering is one of the paper’s quiet but important contributions. The model improvement is not just from architecture; it also comes from making heterogeneous data trainable in one pipeline.  6. Main Results  RoboCasa-GR1 Simulation  On RoboCasa-GR1, the paper evaluates 24 tabletop rearrangement and articulated-object manipulation tasks. The key ablation compares VAE latent state representations with DINO latent representations.                    Method        Visual Rep.        MM-DiT        VLM        Success Rate                            GR00T-N1.6        -        -        Cosmos        47.6                    StarVLA        -        -        Qwen3-VL        47.8                    GR00T-EI30k        -        -        Qwen3-VL        51.3                    UWM-0.1B        VAE        no        -        14.2                    UWM-1B        VAE        no        Qwen3-VL        19.3                    UWM + MM-DiT        VAE        yes        Qwen3-VL        20.0                    LDA + DiT        DINO        no        Qwen3-VL        48.9                    LDA-0.5B        DINO        yes        Qwen3-VL        50.7                    LDA-1B        DINO        yes        Qwen3-VL        55.4              The striking number is the jump from 20.0 to 55.4 when replacing VAE-style pixel latents with DINO representations under the LDA setup. The authors use this to argue that semantically structured latent spaces are critical for scalable dynamics learning.  Real-World Gripper Manipulation  LDA-1B is evaluated on Galbot G1 with a two-finger gripper. It outperforms GR00T-N1.6 and \\(\\pi_{0.5}\\) across pick-and-place, contact-rich, fine manipulation, and long-horizon tasks.  Examples:      Pick and place: 80-90% success    Flip box: 60% success vs. 40% / 20% baselines    Wipe board: 72% success vs. 44% / 52% baselines    Clean rubbish: 35% success while both baselines fail at 0%    The long-horizon and contact-rich tasks are especially relevant: they require temporal consistency, contact reasoning, and recovery from intermediate error..."
    },{
      "title": "VITRA: Scalable VLA Pretraining from Real-Life Human Videos",
      "url": "/posts/2026/04/vitra-paper-notes/",
      "date": "2026-04-28",
      "tags": ["VLA","Robot Learning","Dexterous Manipulation","Human Videos","PaliGemma","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  VITRA explores a simple but powerful idea: use ordinary egocentric human activity videos as scalable pretraining data for dexterous Vision-Language-Action models. The authors treat the human hand as a dexterous end-effector, reconstruct 3D hand motion with MANO-based labels, segment long videos into atomic hand actions, caption each segment with language, and pretrain a PaliGemma-based VLA with a diffusion action head. Robot data is then used mainly for adaptation: after human-video pretraining, the model is fine-tuned on a much smaller set of real robot trajectories, with the robot hand mapped into the human/MANO action space.  Paper Info  The paper is “Scalable Vision-Language-Action Model Pretraining for Robotic Manipulation with Real-Life Human Activity Videos” by Qixiu Li, Yu Deng, Yaobo Liang, Lin Luo, Lei Zhou, Chengtang Yao, Lingqi Zeng, Zhiyuan Feng, Huizhi Liang, Sicheng Xu, Yizhong Zhang, Xi Chen, Hao Chen, Lily Sun, Dong Chen, Jiaolong Yang, and Baining Guo from Tsinghua University and Microsoft Research Asia. It is listed on the project page as ICRA 2026. The paper, project, code, data, and models are linked from microsoft.github.io/VITRA, with the arXiv PDF at 2510.21571.  The Core Problem  Modern VLA models need action data, but robot action data is expensive, slow to collect, and often narrow in objects, scenes, and skills. This is even more severe for dexterous hands, where large-scale robot datasets are scarce. In contrast, the web and existing egocentric video datasets contain many real human manipulation behaviors, but they are unsegmented, uncalibrated, noisy, and missing action labels.  VITRA asks whether these raw human videos can be converted into the same kind of supervision used by robotic VLA models: image, language instruction, state, and future action chunks. Their answer is yes, with a pipeline that aligns human video data to robot VLA data at two levels: task granularity and action labels.  Turning Raw Human Videos into VLA Data  The data construction pipeline has three stages.  First, VITRA estimates 3D motion. It detects whether the camera is static or moving, estimates intrinsics, undistorts fisheye or wide-angle videos into a pinhole model, reconstructs per-frame 3D hand motion with HaWoR, and represents hand pose with the MANO parametric hand model. Camera poses are estimated with a modified MegaSAM pipeline using MoGe-2 depth priors, allowing the system to transform camera-space hand motion into world-space trajectories.  Second, it segments long videos into atomic actions. Instead of using fixed time windows or existing dataset annotations, VITRA cuts at local minima of wrist speed in world space. This is a nicely pragmatic idea: human hand motions often slow down at action boundaries, and because the speed is computed from reconstructed 3D wrist trajectories, the segmentation is tied to action dynamics rather than only image appearance.  Third, it labels each action with language. For every segment, the pipeline samples frames, overlays projected 3D hand trajectories, and asks GPT-4.1 to describe the specified hand action in imperative form. The trajectory overlay is important because it gives the captioning model temporal and geometric hints about what the hand is actually doing.  The resulting dataset contains about 1M atomic VLA episodes and 26M frames, sourced from Ego4D, EPIC-KITCHENS, EgoExo4D, and Something-Something-V2. Importantly, the original human annotations from those datasets are not used, because they do not match the desired robotic action granularity.  Model and Action Space  The VLA model uses PaliGemma-2 3B as the vision-language backbone and a Diffusion Transformer action expert for action prediction. The VLM receives the image, language instruction, a camera FoV token, and a learnable cognition token. The cognition token becomes the conditioning feature for the diffusion action head.  The model predicts a chunk of future hand actions. In the paper, the hand action is:\\[a_t = [\\Delta t_l, \\Delta r_l, \\theta_h^l, \\Delta t_r, \\Delta r_r, \\theta_h^r] \\in \\mathbb{R}^{102}\\]  Here, each hand has 51 dimensions: 3D relative wrist translation, 3D relative wrist rotation, and 45 MANO finger pose angles from 15 joints times 3 Euler angles. The released code pads this into a unified 192-D action space, where left hand occupies 0:51, right hand occupies 51:102, and 102:192 is currently padding. The state is padded to 212-D. In the code, the active state mostly uses the same left/right hand kinematic slots, while the final 20 dimensions are reserved for MANO beta shape parameters but are not currently used.  This padding is best understood as a fixed interface with masks. The diffusion head always sees the same action dimensionality, but the action mask says which hand and which dimensions are valid. That makes single-hand, dual-hand, human, and robot data easier to route through one model.  Robot Fine-Tuning and XHand Mapping  My reading of the training flow is: VITRA first learns broad manipulation priors from human data, then uses robot data for deployment adaptation. This is not “robot data only afterthought” exactly, but robot data is not the main source of scale; it is used after pretraining to adapt the policy to real robot embodiment and execution.  For the real robot experiments, the paper uses a Realman arm with a 12-DoF XHand and a RealSense head camera. The robot data is aligned to the human-hand action space: camera-space end-effector pose gives the 6D wrist action, while robot hand joints are mapped to the closest MANO/human-hand joint dimensions.  The released code makes this concrete. XHand raw state/action is 36-D: 18 dimensions per hand, consisting of 6D wrist pose plus 12 hand joints. The function transfer_xhand_to_human inserts these XHand dimensions into selected channels of the human/MANO-style action space. During inference, transfer_human_to_xhand extracts the same mapped channels back into XHand commands. So the mapping is not an IK solver or full pose retargeting method; it is a hard-coded sparse index/sign correspondence between selected XHand joints and selected MANO action dimensions.  This also explains the relation between human pretraining and robot inference. Human pretraining supervises MANO-style finger action. Robot fine-tuning teaches the model which subset of those human-hand channels matter for XHand, and inference reads only those channels back out for the robot hand.  Results  On unseen human hand action prediction, VITRA outperforms baselines trained on lab data, original human annotations, and a concurrent hand-VLA baseline. The ablations are intuitive: trajectory-aware augmentation helps, causal action denoising helps, wrist-speed segmentation beats fixed-interval segmentation, and trajectory overlays improve GPT captioning.  The real-robot results are the most important part. After fine-tuning on 1.2K teleoperated robot trajectories, VITRA reaches 71.0% average success on seen tasks and 64.6% on unseen object/background/category settings, substantially higher than the compared baselines including VPP, π0, no VLA pretraining, latent-action pretraining, and OXE pretraining. The paper also shows a positive scaling trend: larger and more diverse human-video pretraining improves both human hand prediction and downstream robot success.  Codebase Notes  The released repo is useful for understanding the data representation and model interface, but it does not appear to include the complete production-scale raw-video-to-VITRA-1M processing pipeline. It includes dataset documentation, undistortion scripts, metadata formats, loaders, a hand reconstruction wrapper that calls MoGe/HaWoR/MANO, model code, and the robot XHand alignment functions.  For the MANO part, the released metadata stores hand_pose as (T, 15, 3, 3) local MANO joint rotations, plus wrist orientation, translation, MANO beta, and 21 hand joints. In the training code, the default action type is angle-based, so the finger action is MANO joint pose, not raw robot joint pose. XHand enters later through the sparse mapping described above.  Takeaway  VITRA is compelling because it treats human video not as a vague source of visual representation, but as explicit action supervision. The key move is to force alignment: segment human videos into robot-like atomic tasks, reconstruct MANO-based 3D hand actions, caption them in robot-instruction style, and train a VLA action head in a unified action space. The approach is imperfect because monocular hand and camera reconstruction is noisy, and the released repo does not expose every dataset-construction component. But the direction feels important: scalable human activity video can become a serious pretraining source for dexterous robot manipulation, with small robot datasets used to bridge embodiment differences.  本文支持通过顶部导航栏的语言切换按钮在 Engli..."
    },{
      "title": "UniDex: A Robot Foundation Suite for Universal Dexterous Hand Control from Egocentric Human Videos",
      "url": "/posts/2026/04/unidex-paper-notes/",
      "date": "2026-04-10",
      "tags": ["Dexterous Manipulation","Vision-Language-Action","Human Video Learning","Cross-Embodiment","Foundation Model","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  UniDex is a complete foundation suite for universal dexterous hand control, built from egocentric human videos rather than expensive robot teleoperation data. It introduces three tightly coupled components: (1) UniDex-Dataset — 50K+ trajectories across 8 different robot hands (6–24 DoFs) retargeted from human video data; (2) FAAS (Function–Actuator–Aligned Space) — a unified action representation that maps functionally similar joints across different hands to shared coordinates, enabling cross-hand transfer; and (3) UniDex-VLA — a 3D vision-language-action policy pretrained on this dataset. On five challenging real-world tool-use tasks, UniDex-VLA achieves 81% average task progress (vs. 38% for π₀), demonstrates zero-shot cross-hand skill transfer, and shows that human video data can partially substitute for robot demonstrations at a ~2:1 exchange rate. Accepted at CVPR 2026.  Paper Info      Title: UniDex: A Robot Foundation Suite for Universal Dexterous Hand Control from Egocentric Human Videos    Authors: Gu Zhang†, Qicheng Xu, Haozhe Zhang, Jianhan Ma, Long He, Yiming Bao*, and 13 additional contributors    Affiliations: Tsinghua University, Shanghai Qizhi Institute, Sun Yat-sen University, UNC Chapel Hill    Venue: CVPR 2026    arXiv: 2603.22264    Project page: unidex-ai.github.io    Code: github.com/unidex-ai/UniDex    1. Problem and Motivation  Dexterous manipulation with multi-fingered hands is hard — and building foundation models for it is even harder than for grippers, for three reasons:      Data scarcity: Dexterous hand teleoperation is expensive and doesn’t scale. Most existing robot foundation datasets are gripper-centric.    Embodiment heterogeneity: Dexterous hands vary wildly in DoFs (6–24), morphology, kinematics, and appearance. A policy trained on one hand doesn’t transfer to another.    High dimensionality: Controlling 20+ joints simultaneously demands expressive action spaces and effective learning algorithms.    The key insight: dexterous robot hands are designed to mimic human hands, and humans naturally generate abundant manipulation data in daily life. Egocentric human videos are cheaper, more diverse, and easier to scale than robot teleoperation. The challenge is bridging the kinematic and visual gaps between human and robot hands.  2. Method  2.1 UniDex-Dataset: From Human Videos to Robot Trajectories  Data Sources: Four egocentric human-manipulation datasets — H2O, HOI4D, HOT3D, and TACO — providing diverse daily manipulation activities.  Human-to-Robot Transformation Pipeline:      Visual alignment: Compute pointclouds from RGB-D, mask out human hands (using WiLoR + SAM2), replace with retargeted robot hand meshes, and reproject to single-view pointcloud    Kinematic retargeting (human-in-the-loop):              Extract \\(m\\) fingertip targets from human hand pose: \\(X^\\star = [x_1^\\star, \\ldots, x_m^\\star] \\in \\mathbb{R}^{3 \\times m}\\)        Introduce a 6-DoF dummy base offset \\(T_{\\text{offset}}\\) for global alignment        Solve fingertip IK: \\(x_i(q; T_{\\text{offset}}) = \\text{Trans}(T_{\\text{world}}^{\\text{dummy}} \\cdot T_{\\text{offset}} \\cdot T_i(q)) \\in \\mathbb{R}^3\\)        Automatic stage: IK solver minimizes fingertip error with joint limits/damping        Interactive stage: Human adjusts \\(T_{\\text{offset}}\\) via GUI sliders until contacts look plausible              Result: 9M paired image–pointcloud–action frames, 50K+ trajectories, 8 robot hand platforms (Inspire, Leap, Shadow, Allegro, Ability, Oymotion, Xhand, Wuji), covering 6–24 active DoFs.                    Dataset        Trajectories        Hands        Language        Varied Scenes        Pointcloud                            UniDex-Dataset        52K        8        ✓        ✓        ✓                    ActionNet        30K        2        ✓        ✗        low-quality                    RoboMind        19K        1        ✓        ✗        ✗                    RealDex        2K        2        ✓        ✗        ✓              2.2 FAAS: Function–Actuator–Aligned Space  The core idea: despite different DoFs and kinematics, all dexterous hands implement the same small set of functional primitives — thumb–index pinch, finger curling around handles, lateral ab-/adduction for stabilization.  FAAS maps each actuator to a shared index based on its functional role, not its URDF position. This creates a function-centric control interface shared across embodiments:      82-dimensional vector: 18 dims for wrist pose (9d pose × 2 hands = absolute + relative), 64 dims for joint commands (32 slots per hand)    21 base actuator slots shared across all hands; remaining slots for hand-specific DoFs    Functionally similar joints (e.g., thumb flexion on Allegro vs. Inspire) get the same FAAS index    This is elegant because it’s purely a mapping — no learned alignment, no post-processing. Just group actuators by what they do.  2.3 UniDex-VLA: 3D Vision-Language-Action Policy  Architecture follows π₀ with key modifications for 3D dexterous control:      3D pointcloud encoder: Replace SigLIP (2D) with Uni3D — a vanilla ViT pretrained to align pointcloud features with image–text features    Backbone: Gemma (from PaliGemma), fusing pointcloud features with text and proprioception    Action head: Flow matching with forward-Euler integration at inference    Observation: \\(o_t = [P_t, \\ell_t, q_t]\\) — colored pointcloud, language instruction, proprioceptive state (all in FAAS)    Action: \\(H\\)-step action chunk \\(A_t = [a_t, \\ldots, a_{t+H-1}]\\) in FAAS    Training: Pretrain on UniDex-Dataset, then finetune with 50 task demonstrations per task.  2.4 UniDex-Cap: Human-Robot Data Co-training  A practical portable capture setup:      Apple Vision Pro (hand/head pose estimation) + Intel RealSense L515 (RGB-D) + 3D-printed mount    Time-synchronized, calibrated to shared coordinate frame    Captured human data → transformation pipeline → robot trajectories for co-training    Key finding: ~2 human demos can substitute for 1 robot demo, and human demos are ~5.2× faster to collect. This means significant cost reduction for scaling dexterous data.  3. Experiments and Main Results  Hardware      7-DoF Franka Panda arm + three dexterous end-effectors: Inspire (6 active, 12 full DoFs), Wuji (20 active DoFs), Oymotion (6 active, 11 full DoFs)    Intel RealSense L515 for egocentric RGB-D    Only 50 demonstrations per task for fine-tuning    Five Real-World Tool-Use Tasks      Make Coffee (Inspire): Grasp kettle → lift to dripper → pour water    Sweep Objects (Inspire): Grasp sweeper → sweep objects into dustpan    Water Flowers (Wuji): Grasp spray bottle → press trigger with thumb    Cut Bags (Wuji): Insert fingers into scissors → cut bags    Use Mouse (Wuji): Place fingers on mouse → drag file → click    Main Results (Average Task Progress, 20 trials/task)                    Model        Make Coffee        Sweep        Water Flowers        Cut Bags        Use Mouse        Average                            DP        32.5        37.5        50.0        27.5        20.0        29.0 ± 19.9%                    DP3        35.0        50.0        40.0        12.5        20.0        35.0 ± 17.1%                    π₀        60.0        55.0        85.0        15.0        60.0        38.0 ± 7.4%                    UniDex-VLA (No Pretrain)        60.0        82.5        50.0        32.5        30.0        32.5 ± 18.5%                    UniDex-VLA        87.5        82.5        85.0        90.0        60.0        81.0 ± 12.1%              UniDex-VLA achieves 81% average task progress — more than doubling π₀ (38%) and all other baselines.  Generalization Results  Spatial generalization: With DemoGen augmentation, UniDex-VLA approaches near-perfect success across out-of-distribution object placements.  Object generalization: Replacing the original black kettle with a smaller purple kettle of different shape → UniDex-VLA achieves 80% (vs. 15% for π₀), showing robust tool understanding.  Cross-hand transfer (zero-shot):                    Hand        π₀        UniDex-VLA (No Pretrain)        UniDex-VLA                            Wuji        0%        0%        40%                    Oymotion        10%        5%        60%              A policy trained only on Inspire Hand transfers zero-shot to Wuji and Oymotion — this is enabled by FAAS. Baselines completely fail.  Human-Robot Co-training  The co-training heatmap (Fig. 13) reveals:      With 0 robot demos, adding human demos alone doesn’t work (all zeros)    With even 10 robot demos + human demos, performance scales steadily    The “high-performance” boundary has slope ≈ 2, meaning ~2 human demos ≈ 1 robot demo    Human demos are ~5.2× faster to collect → substantial cost savings    4. Strengths      Complete suite, not just a model: Dataset + action space + policy + capture system — each component is independentl..."
    },{
      "title": "PointVLA: Injecting the 3D World into Vision-Language-Action Models",
      "url": "/posts/2026/04/pointvla-paper-notes/",
      "date": "2026-04-10",
      "tags": ["Robotics","Vision-Language-Action","3D Point Cloud","Imitation Learning","Bimanual Manipulation","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  Current VLA models are pre-trained on massive 2D vision-language data, but their reliance on RGB images limits spatial reasoning for real-world manipulation. Retraining from scratch with 3D data is prohibitively expensive. PointVLA proposes a lightweight solution: freeze the pre-trained VLA, inject point cloud features into its action expert via a modular block, and use a skip-block analysis to find which blocks in the action expert are least critical — injecting 3D features only there to minimize disruption. The result is a model that gains 3D spatial understanding (height adaptability, real-vs-photo discrimination, few-shot multi-tasking) while preserving the full benefit of large-scale 2D pre-training, with only 5 additional lightweight injection blocks to train.  Paper Info  The paper is “PointVLA: Injecting the 3D World into Vision-Language-Action Models,” by Chengmeng Li, Yichen Zhu, Junjie Wen, Yan Peng, Yaxin Peng, and Feifei Feng from Midea Group, Shanghai University, and East China Normal University. Accepted at IEEE RA-L 2025. The project page is at pointvla.github.io and the paper at arXiv:2503.07511.  1. Problem and Motivation  VLA models like OpenVLA, \\(\\pi_0\\), and DexVLA have shown impressive capabilities by leveraging pre-trained vision-language models as backbones, then training action experts to translate visual-linguistic understanding into robot actions. Their strength comes from billions of parameters pre-trained on internet-scale 2D data.  But they only see in 2D. This creates real failure modes: a VLA cannot distinguish a photograph of an object from the real thing (both look identical in RGB from the right angle), cannot adapt when an object is placed at a different height than in training, and generally lacks the depth perception needed for precise 3D manipulation.  The naive fix — retrain the whole foundation model with 3D data — is impractical. 3D robotic datasets are orders of magnitude smaller than 2D vision-language corpora. Retraining would also discard the valuable 2D representations. An alternative approach like 3DVLA processes 3D tokens through the LLM backbone, but current VLMs exhibit limited 3D comprehension when fine-tuned on small 3D datasets due to the domain gap between 2D pixels and 3D structures.  PointVLA takes a different path: treat 3D point clouds as a complementary conditioning signal rather than a primary input modality, injecting them into the action expert rather than the vision-language backbone.  2. Method  PointVLA builds on DexVLA (Qwen2-VL as the 2B-parameter VLM backbone + ScaleDP as the 1B-parameter diffusion policy action expert). The key insight is to keep the VLM entirely intact and inject 3D information only into the action expert, where spatial reasoning most directly affects motor behavior.  2.1 Point Cloud Encoder  Rather than using a pre-trained 3D visual encoder (which the authors found hinders generalization to new environments, consistent with findings in DP3 and iDP3), PointVLA adopts a simplified hierarchical convolutional architecture. Upper layers extract low-level features, lower layers learn high-level scene representations, with max pooling between layers to reduce point cloud density. Feature embeddings from each convolutional block are concatenated into a unified multi-level 3D representation.  2.2 Point Cloud Injector  The injector has three components:      Channel alignment: Transform the 128-dimensional point cloud embedding to match the action expert’s 1280-dimensional channel size    Action embedding bottleneck: Compress the potentially large action embedding (from chunk-based prediction) to align with the point cloud embedding    Block-wise injection: For each selected block in the action expert, an MLP adapter processes the point cloud embedding, followed by a zero-initialized linear layer that adds the 3D features to the block’s output    The zero initialization is important — it means the injected features start as zero, so the model initially behaves identically to the vanilla VLA. The 3D signal is gradually learned during fine-tuning without disrupting existing representations.  2.3 Skip-Block Analysis: Where to Inject  Not all blocks in the action expert are equally important. Injecting 3D features everywhere would be both expensive and disruptive. The authors perform a systematic skip-block analysis on DexVLA’s 32-block action expert using a shirt-folding task:  Single-block skipping: The first 11 blocks are critical — skipping any one causes significant performance drops. From block 11 onward, skipping a single block is acceptable, indicating these later blocks contribute less after training.  Multi-block skipping: Starting from block 11, up to 5 consecutive blocks can be skipped before the model fails. Skipping 6 or more causes instant performance collapse.  Based on this analysis, PointVLA injects 3D features into 5 blocks (blocks 12, 13, 16, and two others in the less-critical zone). All modules in the vanilla action expert are frozen except the final layers that fit the embodiment’s output. Only the 5 injection blocks are trained — making the approach highly parameter-efficient.  3. Experiments  3.1 Setup  Two real-world bimanual platforms:      Bimanual UR5e: Two UR5e arms with Robotiq grippers, three cameras (two wrist RealSense D435i + one top), 14-dim action space, 15Hz. RealSense L515 for point clouds.    Bimanual AgileX: Two 6-DoF AgileX arms with wrist cameras and base camera, 14-dim action space, 30Hz. Same L515 for point clouds.    Baselines: OpenVLA, Diffusion Policy (DP), 3D Diffusion Policy (DP3), ScaleDP-1B, Octo, and DexVLA. Since PointVLA is built on DexVLA, DexVLA serves as the direct ablation — same model without point cloud injection.  3.2 Few-Shot Multi-Tasking (AgileX)  Four tasks with only 20 demonstrations each (80 total): ChargePhone, WipePlate, PlaceBread, TransportFruit. These test both independent and coordinated bimanual movements.  PointVLA outperforms all baselines. Notably, Diffusion Policy fails on most tasks — with only 20 demos per task, the action representation space becomes entangled. Even scaling up (ScaleDP-1B) doesn’t help much. DexVLA shows strong few-shot capability but PointVLA consistently improves on it, demonstrating that point cloud integration enables more sample-efficient learning.  3.3 Long-Horizon Packing (UR5e)  A challenging conveyor belt task: pick up two laundry detergent bottles from a moving belt and pack them into a box, then seal it (5 sequential subtasks). The assembly line is in motion, the embodiment differs from pre-training data, and the task is long-horizon.  Results (average completion length out of 5 subtasks):      OpenVLA: 0.36, DP: 0.36, ScaleDP-1B: 0.72    DexVLA: 1.72    PointVLA: 2.36    PointVLA surpasses DexVLA by 0.64 in average completion length — a substantial margin on a task where all other baselines essentially fail after the first 1–2 steps.  3.4 Real-vs-Photo Discrimination  A striking experiment: replace a real laundry detergent bottle on the conveyor belt with its photograph displayed on a screen. From the egocentric top camera, the photo closely resembles the real object. All 2D-based models (OpenVLA, DP, ScaleDP, DexVLA) attempt to grasp the non-existent object, with DexVLA entering a repetitive grasping loop. PointVLA is the only model that correctly recognizes no real object exists on the belt, achieving 3/3 success while all baselines score 0/3.  This is perhaps the most compelling demonstration of why 3D understanding matters for safety — a purely 2D model can be trivially “deceived” by a printed image.  3.5 Height Adaptability  Training data uses a 3mm foam layer under the bread; at test time, this is replaced with a 52mm layer. All 2D baselines (OpenVLA, DP, ScaleDP, DexVLA) fail — they push down to the trained height and miss the object. PointVLA succeeds 5/5 by perceiving the actual 3D height and adjusting accordingly.  3.6 Simulation (RoboTwin)  On the RoboTwin benchmark (14-DoF mobile bimanual platform, 16 diverse tasks), PointVLA achieves the highest average success rate across all tasks with both 20 and 50 demonstrations. Interestingly, for pure 3D methods like DP3, adding RGB input can actually hurt performance, while PointVLA’s approach of conditionally integrating 3D as a complement to 2D avoids this problem.  4. Strengths and Limitations  Strengths. The paper’s core contribution is conceptually clean: rather than choosing between 2D pre-training and 3D understanding, PointVLA gets both by treating point clouds as a complementary signal injected into carefully selected locations. The skip-block analysis is a principled and reusable technique — it provides a general recipe for identifying where to inject new modalities into pre-trained models. The real-vs-photo and height adaptability experiments are not just ablations; they expo..."
    },{
      "title": "HandX: Scaling Bimanual Motion and Interaction Generation",
      "url": "/posts/2026/04/handx-paper-notes/",
      "date": "2026-04-10",
      "tags": ["Hand Motion","Motion Generation","Bimanual Interaction","Diffusion Models","Scaling Laws","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  This paper introduces HandX, a unified foundation for bimanual hand motion generation that spans data, annotation, and evaluation. The authors consolidate existing datasets and collect new motion-capture data emphasizing contact-rich two-hand interactions, then propose a decoupled annotation pipeline that extracts kinematic features and uses LLM reasoning to produce fine-grained text descriptions. They benchmark both diffusion and autoregressive models across multiple scales and observe clear scaling trends: jointly increasing model capacity and data size consistently improves text-motion alignment and hand-contact quality.  Paper Info  The paper is “HandX: Scaling Bimanual Motion and Interaction Generation,” by Zimu Zhang, Yucheng Zhang, Xiyan Xu, Ziyin Wang, Sirui Xu, Kai Zhou, Bing Zhou, Chuan Guo, Jian Wang, Yu-Xiong Wang, and Liang-Yan Gui from University of Illinois Urbana-Champaign, Specs Inc., and Snap Inc. It appears at CVPR 2026. The project page is at handx-project.github.io and code is available at github.com/handx-project/HandX.  1. Problem and Motivation  Human motion synthesis has made impressive progress, but hand motion — especially bimanual interaction — remains underexplored. Most whole-body motion models treat hands as rigid end-effectors and miss the fine-grained cues that matter: finger articulation, contact timing, and inter-hand coordination. Meanwhile, hand-centric datasets tend to focus on object interaction, use coarse annotations, or lack high-fidelity bimanual sequences altogether.  The bottleneck is threefold. First, existing datasets either lack hand detail (Motion-X, InterAct) or are limited to object-centric settings with categorical labels (ARCTIC, H2O). Second, mismatched skeletons, frame rates, and annotation protocols across sources make it hard to unify data. Third, standard evaluation metrics like FID and R-Precision do not capture hand-specific qualities like contact fidelity or bimanual coordination.  HandX is designed to address all three. It provides a large-scale, contact-rich bimanual motion dataset with fine-grained multi-level text annotations, and introduces hand-focused metrics to evaluate generation quality.  2. Dataset  HandX is built in two steps.  Aggregating existing data. The authors consolidate multiple open-source datasets with bimanual motion (HOT3D, ARCTIC, GigaHands, H2O, HoloAssist), converting them to a unified skeletal representation and coordinate system. An intensity-aware filter removes static or near-static segments that would cause generative models to freeze.  Capturing new data. Using a 36-camera OptiTrack optical motion-capture system, they record dexterous two-hand interactions with 25 reflective markers per hand, capturing fine-grained articulation of the wrist, palm, fingers, and fingertips. The hand skeleton is reconstructed by estimating joint centers and enforcing anatomical constraints on bone lengths, with per-frame refinement for kinematic consistency.  The final dataset comprises 54.2 hours of high-quality bimanual motion, 5.9 million frames, and 490K text descriptions. Compared to prior datasets, HandX stands out for its contact richness, motion intensity, and fine-grained language annotations organized in a triplet structure (left hand, right hand, inter-hand relation).  3. Annotation Pipeline  Manually annotating this much bimanual motion is prohibitively expensive. The authors propose a two-stage automatic pipeline.  Stage 1: Kinematic Feature Extraction. They compute a set of kinematic descriptors at each frame — finger flexion, finger-palm distances, inter-hand spatial relationships — then segment the temporal evolution into discrete events (e.g., touch, slide, release). These events are organized into a structured JSON format that LLMs can readily parse.  Stage 2: LLM-based Description Generation. Given the JSON-formatted kinematic features, a carefully designed prompt guides the LLM to generate descriptions following three principles: (a) explicitly describe left hand, right hand, and their inter-hand relationships; (b) report critical motion events like contact, separation, and hyperextension; (c) incorporate temporal context to preserve the progression of events. The LLM generates five levels of detail, from concise summaries to comprehensive descriptions covering subtle changes and speed variations.  This decoupled approach ensures that annotations are grounded in actual motion dynamics rather than hallucinated from visual appearance, and the multi-level structure supports both fine- and coarse-grained generation tasks.  4. Generation Models  The paper benchmarks two representative paradigms.  4.1 Diffusion Model  The motion representation concatenates 3D joint coordinates with a compact rotation scalar per joint (exploiting the limited rotational degrees of freedom of hand joints). An MLP-based encoder projects each frame into a \\(D\\)-dimensional embedding. The three text prompts (\\(T_L\\), \\(T_R\\), \\(T_I\\) for left, right, and interaction) are encoded separately by T5, each with a learnable CLS token to prevent left-right confusion. The text embeddings are cross-attended with the motion embeddings and fused through residual connections:\\[\\tilde{z} = z'_t + \\sum_{k \\in \\{L,R,I\\}} \\text{CrossAttention}(z'_t, \\mathfrak{T}_k)\\]  An MLP decoder maps the fused representation back to motion: \\(\\tilde{x} = G(\\tilde{z}) \\in \\mathbb{R}^{F \\times 2J \\times 4}\\).  A key design insight is that at inference time, the diffusion model supports diverse generation tasks through partial denoising — blending known constraints with the current sample at each denoising step. This enables motion in-betweening, keyframe control, wrist trajectory conditioning, hand-reaction synthesis, and long-horizon generation, all from a single model.  4.2 Autoregressive Model  The AR model uses Finite Scalar Quantization (FSQ) for tokenization, which offers better codebook utilization and scaling behavior than VQ-VAE. It adopts a local motion representation (wrist-relative positions and velocities) to improve codebook utilization. A text-prefix autoregressive model then predicts the next motion token conditioned on preceding tokens and the T5-encoded text prefix:\\[\\mathcal{L} = -\\sum_{k=1}^{n} \\log p(\\hat{y}^k \\mid y^{<k}, \\mathfrak{T})\\]  The tokenizer uses 1D convolutional blocks with a temporal downsampling factor of 2, and the autoregressive model is explored with varying Transformer layers (8, 12, 16) and codebook sizes (512 to 4,096).  5. Metrics  Beyond standard FID, Diversity, R-Precision, and MM Distance, the paper introduces contact-focused metrics: contact precision (\\(C_\\text{prec}\\)), recall (\\(C_\\text{rec}\\)), and F1 (\\(C_\\text{F1}\\)). These evaluate whether the generated sequence reproduces contact events at the corresponding frames in the ground truth, with a 2 cm contact threshold.  This is a meaningful addition. Standard metrics can look good even when contact timing and inter-hand coordination are poor, which is exactly the failure mode you care about in bimanual generation.  6. Experiments and Main Results  Scaling Trends  Both model families show clear positive scaling trends. For diffusion models, scaling either model depth or training data consistently improves R-Precision and contact-related scores. The 12-layer model achieves the best overall performance, but further scaling to a 16-layer ultra-large variant (6.7× more parameters) causes performance to drop across all metrics — a clear saturation point.  For autoregressive models, increasing codebook size alone does not reliably help. Performance only improves when codebook size and model capacity are scaled jointly, suggesting finer discrete representations need sufficient autoregressive capacity to be useful.  Under a fixed 5% data budget, the authors observe an approximately log-linear relationship between Top-3 R-Precision and FLOPs, with a correlation coefficient of 0.96:\\[R_\\text{prec} = 0.4391 \\times \\log_{10}(\\text{FLOPS}) - 3.8707\\]  Qualitative Results  The qualitative comparisons are telling. Models trained on the full dataset generate more expressive motion with better text alignment than those on 5% or 20% subsets. Larger models produce motion better aligned with text and exhibit improved bimanual contact. The generated sequences successfully capture complex contact events specified in the text prompt — finger-to-finger touches, temporal coordination, and hand-hand spatial relationships.  The framework also demonstrates versatile generation: text-to-motion, motion in-betweening, trajectory control, keyframe guidance, hand-reaction synthesis, and long-horizon generation, all from the same model via the partial denoising mechanism.  7. Code and Implementation  The codebase is well-structured, with separate modules for diffusion, autoregressive, evaluation, and IsaacGym-based simula..."
    },{
      "title": "Goal-VLA: Image-Generative VLMs as Object-Centric World Models Empowering Zero-shot Robot Manipulation",
      "url": "/posts/2026/04/goal-vla-paper-notes/",
      "date": "2026-04-10",
      "tags": ["Vision-Language-Action","Zero-shot Manipulation","World Model","Goal-conditioned","Spatial Grounding","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  Goal-VLA proposes a zero-shot robotic manipulation framework that uses image-generative VLMs as object-centric world models. Instead of training end-to-end VLAs on expensive paired action data, Goal-VLA generates a goal image depicting the desired task outcome, extracts a precise 3D object pose from it via feature matching and point cloud registration, and then uses a training-free low-level policy to execute the manipulation. A novel Reflection-through-Synthesis mechanism iteratively validates and refines the generated goal image before execution. The result: 59.9% average success in RLBench simulation (vs. 26% for the best baseline MOKA) and 60% in real-world tasks — all zero-shot, with no task-specific fine-tuning.  Paper Info      Title: Goal-VLA: Image-Generative VLMs as Object-Centric World Models Empowering Zero-shot Robot Manipulation    Authors: Haonan Chen, Jingxiang Guo, Bangjun Wang, Tianrui Zhang, Xuchuan Huang, Boren Zheng, Yiwen Hou, Chenrui Tie, Jiajun Deng, Lin Shao†    Affiliations: National University of Singapore, HKU, Peking University, Tsinghua University    arXiv: 2506.23919    Project page: nus-lins-lab.github.io/goalvlaweb    1. Problem and Motivation  Generalization in robotic manipulation is hard. Two dominant VLA paradigms both have critical weaknesses:                    Paradigm        Intermediate Repr.        Training-Free?        Precise 3D Grounding?                            End-to-end VLAs (OpenVLA, π₀)        N/A        ✗        ✗                    Hierarchical VLAs (MOKA, VoxPoser, SUSIE)        Keypoints / Value maps / Subgoal images        Partially        ✗                    Goal-VLA (this paper)        Object state (goal image → 3D pose)        ✓        ✓              The key insight: object state representation is the golden interface that naturally separates high-level semantic reasoning from low-level spatial control. End-to-end models need massive action data. Hierarchical models either use sparse representations (keypoints — not enough spatial detail) or dense ones (subgoal images — but then need trained low-level policies to interpret them). VLMs excel at semantic reasoning but struggle with precise spatial reasoning.  Goal-VLA resolves this by letting the VLM do what it’s good at (semantic goal generation) and offloading spatial grounding to a dedicated geometric module.  2. Method  The framework has three stages:  2.1 Goal State Reasoning (High-Level)  Given an RGB-D observation \\(O = (I, D)\\) and language instruction \\(L\\):      Prompt Enhancement: Feed \\(I\\) and \\(L\\) into a text-output VLM (Gemini 2.5 Pro) to produce a richer, more descriptive prompt \\(L_e\\).    Goal Image Generation: An image-generative VLM (Gemini 2.5 Flash-image) generates a candidate goal image \\(I’_{\\text{cand}}\\).    Reflection-through-Synthesis Loop (the key novelty):              Synthesize: Segment the target object from the candidate goal using Grounded SAM, overlay it onto the original scene with partial transparency        Reflect: A Reflector VLM evaluates whether the synthesized image is semantically correct and physically feasible        Refine: If rejected, the Reflector generates corrective feedback for the next generation attempt        Repeat until validated or max iterations reached              This is clever — the synthesis step grounds the reflection by showing the goal object in the context of the original scene, making errors much easier to spot (e.g., the VLM moving the pan along with the tomato).  2.2 Spatial Grounding  Once a valid goal image \\(I’\\) is obtained, convert the semantic goal into a precise 3D transformation:  Semantic Matching: Use Geo-Aware features to find pixel correspondences between initial image \\(I\\) and goal image \\(I’\\):  \\((x’, y’) = \\arg\\max_{(p,q)} \\frac{f_{(x,y)} \\cdot f’_{(p,q)}}{|f_{(x,y)}| |f’_{(p,q)}|}\\)  This is necessary because the generated goal image is semantically correct but may not preserve instance-level appearance — so traditional optical flow fails.  Point Cloud Registration: Lift 2D to 3D using depth, align depth scales via least-squares regression on background pixels:  \\(D[(M \\cup M’)^c] = s_1 \\cdot D’[(M \\cup M’)^c] + b\\)  Then solve for a similarity transformation using the Umeyama algorithm:  \\(s_2 \\cdot P’ = RP + t\\)  where \\(R \\in SO(3)\\), \\(t \\in \\mathbb{R}^3\\), and \\(s_2\\) accounts for scale differences.  2.3 Low-Level Policy      Contact Module: Sample-based method to find feasible contact poses on the object’s point cloud (surface normals → collision filtering → geometric scoring)    Goal Pose: Apply the computed \\((R, t)\\) transformation to the contact pose    Motion Planning: Standard sample-based planner to execute the trajectory    The entire low-level policy is training-free — no action data needed.  3. Experiments and Main Results  Simulation (RLBench, 8 tasks, 100 trials each)                    Method        Paradigm        Avg Success                            OpenVLA        End-to-end        0.2%                    π₀        End-to-end        0.0%                    SUSIE        Hierarchical        0.0%                    VoxPoser        Hierarchical        5.8%                    MolmoAct        End-to-end        11.3%                    MOKA        Hierarchical        26.0%                    Goal-VLA        Hierarchical        59.9%              End-to-end models completely fail in zero-shot settings — they’re brittle without in-domain action data. Goal-VLA more than doubles the best baseline.  Real World (4 tasks, 10 trials each)                    Method        Tomato        Sweeping        Duck        Bottle        Avg                            OpenVLA        0/10        0/10        0/10        0/10        0%                    MOKA        5/10        1/10        3/10        0/10        22.5%                    MolmoAct        5/10        0/10        6/10        0/10        27.5%                    Goal-VLA        9/10        4/10        7/10        4/10        60%              Ablation Study                    Configuration        Avg Success                            Base (no enhancement, no reflection)        40.0%                    + Reflector only        51.2%                    + Input Enhancement only        67.5%                    + Both (max 1 reflection)        83.8%                    + Both (max 3 reflections)        88.8%              Input Enhancement contributes the most (+27.5pp), Reflection adds +11.2pp, and they’re complementary. More reflection iterations help further.  4. Strengths      Truly zero-shot: No task-specific fine-tuning, no paired action data — the whole pipeline uses off-the-shelf foundation models    Object-centric abstraction: By focusing on object state rather than agent-centric representations, the framework is inherently cross-embodiment    Reflection-through-Synthesis is well-designed: The synthesis overlay trick makes VLM self-evaluation much more reliable by providing in-context visual comparison    Strong empirical results: 2.3× the best baseline in simulation, 2.2× in real world    Minimal assumptions: Only requires a single RGB-D view and language instruction — no pre-scanned maps, object meshes, or task-specific priors    5. Limitations      Depth estimation bottleneck: Most real-world failures trace back to inaccurate depth in the spatial grounding module, especially for precision-demanding tasks (weighing duck, bottle stand-up)    High-level reasoning failures: Table sweeping requires sophisticated semantic understanding that the VLM sometimes gets wrong    Rigid-body assumption: The framework assumes rigid object transformations — deformable objects or articulated manipulations would need extensions    Latency: Multiple VLM calls (prompt enhancement → image generation → reflection loop) means this is not a real-time system    Limited task complexity: All evaluated tasks are single-step pick-and-place style — long-horizon multi-step manipulation is not addressed    Dependency on commercial VLMs: Uses Gemini 2.5 Pro/Flash, which may not always be available or affordable    6. Takeaways      Object state as the interface between high-level and low-level is a powerful design choice — it cleanly separates what VLMs are good at (semantics) from what geometric methods are good at (spatial precision)    Image-generative VLMs as world models is a natural and underexplored direction — instead of training robot-specific world models, just use the VLM’s ability to imagine future states    Reflection-through-Synthesis is a generally useful technique: synthesize the proposed change in context before evaluating it, rather than evaluating the generated output in isolation    The massive gap between end-to-end VLAs (0-0.2%) and Goal-VLA (59.9%) in zero-shot settings is striking — it suggests that current end-to-end VLAs have essentially no zero-shot capability outside th..."
    },{
      "title": "Fast-dVLA: Accelerating Discrete Diffusion VLA to Real-Time Performance",
      "url": "/posts/2026/04/fast-dvla-paper-notes/",
      "date": "2026-04-10",
      "tags": ["VLA","Discrete Diffusion","Robot Learning","Inference Acceleration","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  Fast-dVLA is the first method to accelerate discrete diffusion VLA (dVLA) models to real-time 30 Hz control. The key insight: despite using bidirectional attention, dVLAs implicitly follow a left-to-right block-wise decoding pattern. Fast-dVLA exploits this by combining block-wise causal attention (enabling KV cache reuse) with diffusion forcing (enabling inter-block parallelism), achieving 2.8×–4.1× speedup while maintaining SOTA-level success rates across CALVIN, LIBERO, and SimplerEnv benchmarks, and demonstrating real-time performance on a real bimanual robot.  Paper Info      Title: Fast-dVLA: Accelerating Discrete Diffusion VLA to Real-Time Performance    Authors: Wenxuan Song*, Jiayi Chen*, Shuai Chen*, Jingbo Wang, Pengxiang Ding, Han Zhao, Yikai Qin, Xinhu Zheng, Donglin Wang, Yan Wang†, Haoang Li†    Affiliation: HKUST (Guangzhou), ShanghaiTech University, Tsinghua University, Westlake University, Zhejiang University    Date: 2026-03-31 (v1), 2026-04-07 (v3)    Venue: arXiv preprint    arXiv: 2603.25661    Project Page: Fast-dVLA    1. Problem and Motivation  Discrete diffusion VLAs (dVLAs) like Dream-VLA and DD-VLA are promising alternatives to flow-matching VLAs — they offer better multimodal alignment and preserve pretrained VLM knowledge. However, they are far too slow for real-time robotic control (~30 Hz):                    Paradigm        Forwards per Sequence        Forward Speed        Inference Speed                            Autoregressive VLA        High        Fast        Slow                    Discrete Diffusion VLA (dVLA)        Low        Slow (no KV cache)        Slow                    Block Diffusion VLA        Moderate        Fast        Moderate                    Fast-dVLA (ours)        Low        Fast        Fast              The root cause: dVLAs use bidirectional attention, which means KV representations change every denoising iteration — preventing KV cache reuse and making each forward pass expensive.  Key observation: Despite bidirectional attention, dVLAs exhibit an implicit block-wise autoregressive decoding tendency — earlier action blocks are decoded before later ones. This is because (1) the backbone is initialized from an AR VLM, retaining AR characteristics, and (2) actions have inherent temporal dependencies.  2. Method  Fast-dVLA has three core components:  2.1 Block-Wise Causal Attention for KV Cache Reuse  Replace bidirectional attention with block-wise causal attention: each block can only attend to prefix tokens and tokens within the current block. Once a block is fully decoded, its KV states are frozen and can be cached for all subsequent blocks — just like standard AR decoding but at the block level.  2.2 Diffusion Forcing for Inter-Block Parallelism  Instead of waiting for block i to finish before starting block i+1 (as in standard block diffusion), Fast-dVLA assigns monotonically increasing noise levels to blocks:  \\(t_1 < t_2 < \\cdots < t_N\\)  Earlier blocks have lower noise (closer to clean) while later blocks are more heavily masked. The model factorizes the denoising as:                    \\(p_\\theta(Y_0        Y_{t_{1:N}}) = \\prod_{i=1}^{N} p_\\theta(Y^0_{B_i}        Y^{t_1}{B_1}, \\ldots, Y^{t_i}{B_i})\\)              This allows concurrent denoising across blocks — earlier blocks finish first, and their cached KV states accelerate later blocks.  2.3 Asymmetric Distillation for Efficient Training  Rather than training from scratch, Fast-dVLA distills from a finetuned bidirectional dVLA teacher:                    \\(\\mathcal{L}{AD} = \\mathbb{E}\\left[\\sum{i=1}^{N} D_{KL}\\left(p_\\theta(\\cdot        \\text{causal view}) | p_{\\phi^-}(\\cdot        \\text{global view})\\right)\\right]\\)              The teacher sees all blocks (bidirectional), while the student only sees causally preceding blocks. This is asymmetric — the student learns to approximate the teacher’s richer context with restricted attention. Converges in only 1/10 of the steps needed for training from scratch.  2.4 Pipelined Parallel Decoding  At inference, blocks operate in a dynamic pipeline with two states:      Semi-activated: block is introduced when the preceding block’s completion ratio exceeds τ_add; only high-confidence tokens are decoded    Fully-activated: transitions when predecessor exceeds τ_act; at least 1/n remaining tokens decoded per step    This mechanism balances speed and reliability while preserving temporal causality.  3. Experiments and Main Results  Paradigm Comparison (LIBERO)                    Method        Avg SR        Speed (tokens/s)        Speedup                            Dream-VLA        0.856        98.8        1.0×                    + Fast-dLLM        0.828        183.2        1.9×                    + Block Diffusion        0.858        181.7        1.8×                    + Fast-dVLA        0.870        313.1        3.2×                    DD-VLA        0.963        152.1        1.5×                    + Fast-dLLM        0.935        312.5        3.2×                    + Block Diffusion        0.967        322.1        3.3×                    + Fast-dVLA        0.966        402.7        4.1×              Fast-dVLA achieves the best speed–performance trade-off, even slightly improving success rates in some cases.  CALVIN Long-Horizon (UD-VLA)  Fast-dVLA achieves 2.8× speedup on UD-VLA (625-token sequences) while maintaining competitive performance (Avg Len 4.54 vs 4.64 baseline), ranking among the top methods on CALVIN ABCD→D.  SimplerEnv (Dream-VLA)  Fast-dVLA achieves 59.3% average success rate at 366.4 tokens/s, outperforming:      Flow-matching methods (π0: 27.1%, GR00T-N1: 36.5%)    AR methods (π0-FAST: 32.1%)    Other dVLA acceleration methods    Real-World Results  On a bimanual AgileX platform with 3 tasks:      Achieves consistent 30 Hz execution frequency — the first dVLA to reach real-time control    Nearly doubles efficiency over Dream-VLA on conveyor belt picking    Maintains competitive success rates on semantic manipulation tasks    Training Efficiency  Asymmetric distillation converges in ~2,000 steps — only 1/5 of the original finetuning budget, and 1/10 of training from scratch.  4. Ablation Highlights      Block size: Using multiples of action dimensionality (e.g., 7 for 7-DoF actions) preserves intrinsic temporal dependencies and yields the best speed/performance trade-off    Confidence threshold (τ_conf): 0.5 balances 2.8× acceleration with only 2% performance drop    Thresholds τ_add and τ_act: Set to 2/7 and 4/7 respectively for the pipelined decoding schedule    5. Strengths and Limitations  Strengths:      First real-time dVLA — reaches 30 Hz on physical robots    Clean, principled design: block-wise attention + diffusion forcing is a natural fit for the observed decoding pattern    Highly efficient training via asymmetric distillation (1/10 steps)    Generalizes across multiple dVLA architectures (Dream-VLA, DD-VLA, UD-VLA)    Maintains or improves performance while dramatically accelerating inference    Limitations:      Requires a pretrained bidirectional dVLA as teacher — not a standalone training method    Block size must align with action dimensionality — less flexible for variable-length outputs    The implicit AR tendency is observed empirically but not theoretically guaranteed for all architectures    Real-world experiments limited to a single bimanual platform    6. Takeaways      dVLAs have a hidden AR structure — even with bidirectional attention, the decoding order is block-wise left-to-right. This is a useful inductive bias, not a bug.    Block-wise attention + diffusion forcing is a powerful combination: KV cache reuse from AR-style attention + inter-block parallelism from diffusion forcing = real-time discrete diffusion.    Asymmetric distillation is remarkably efficient — converting a bidirectional dVLA to a fast block-wise one costs only ~2k training steps.    dVLAs can now compete with flow-matching VLAs on speed while retaining their advantages in multimodal alignment and unified generation — a significant step toward practical deployment.    本文支持通过顶部导航栏的语言切换按钮在 English / 中文 之间切换。  概要  Fast-dVLA 是首个将离散扩散 VLA（dVLA）模型加速至实时 30 Hz 控制的方法。核心发现：尽管使用了双向注意力，dVLA 隐式地遵循从左到右的逐块解码模式。Fast-dVLA 利用这一特性，结合逐块因果注意力（实现 KV 缓存复用）和扩散强制（实现块间并行），在 CALVIN、LIBERO 和 SimplerEnv 基准上实现了 2.8×–4.1× 加速，同时保持 SOTA 级成功率，并在真实双臂机器人上展示了实时性能。  论文信息      标题：Fast-dVLA: Accelerating Discrete Diffusion VLA to Real-Time Performance    作者：Wenxuan Song*、Jiayi Chen*、Shuai Chen* 等    机构：香港科技大学（广州）、上海科技大学、清华大学、西湖大学、浙江大学    日期：2026-03-31（v1），2026-04-07（v3）    arXiv：2603.25661    项目主页：Fast-dVLA    1. 问题与动机  离散扩散 VLA（dVLA），如 Dream-VLA 和 DD-VLA，是流匹配 VLA 的有力替代方案——它们具有更好的多模态对齐能力，且能更好地保留预训练 VLM 的知识。然而，它们的推理速度远不能满足实时机器人控制的需求（通常约 30 Hz）：                    范式        每序列前向次数        单次前向速度        推理速度                            自回归 VLA        高        快        慢                ..."
    },{
      "title": "EVA: Aligning Video World Models with Executable Robot Actions via Inverse Dynamics Rewards",
      "url": "/posts/2026/04/eva-paper-notes/",
      "date": "2026-04-10",
      "tags": ["Video World Models","Robotics","Reinforcement Learning","Alignment","Inverse Dynamics","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  EVA (Executable Video Alignment) identifies and addresses the executability gap in video world models for robotics — where visually coherent generated rollouts can still produce infeasible robot actions. The key idea: train an inverse dynamics model (IDM) on real robot data, then repurpose it as a reward model to post-train the video generator via GRPO. The reward penalizes non-smooth motions (high acceleration/jerk) and out-of-bound actions, directly aligning generated videos with physical executability. On the RoboTwin benchmark and a real bimanual robot, EVA improves kinematic plausibility by +20.9% and boosts real-world task success from 52% to 64% (seen) and 42% to 60% (OOD).  Paper Info      Title: Aligning Video World Models with Executable Robot Actions via Inverse Dynamics Rewards    Authors: Ruixiang Wang, Qingming Liu, Yueci Deng, Guiliang Liu, Zhen Liu, Kui Jia    Affiliation: The Chinese University of Hong Kong, Shenzhen; DexForce Technology Co., Ltd.    Date: 2026-03-24    Venue: arXiv preprint    arXiv: 2603.17808    Project Page: eva-project-page.github.io    1. Problem and Motivation  Video generative models are increasingly used as world models for robotic manipulation in a decoupled paradigm:      A video world model generates a future visual rollout conditioned on the current observation and language instruction    An inverse dynamics model (IDM) converts the generated frames into executable robot actions    The problem: current video world models are optimized for visual realism but lack executability constraints. Even visually coherent rollouts can contain:      Morphological deformations — arm stretching or melting    Joint ambiguity — unclear articulation states    Temporal discontinuities — abrupt jumps between frames    When decoded by an IDM, these artifacts translate into unstable control signals: high-frequency jitter, abrupt joint jumps, or out-of-bounds commands. The authors call this the executability gap.  While this gap can be mitigated at inference time (e.g., rejection sampling), such approaches are inefficient given the high cost of video generation. EVA instead addresses this at training time.  2. Method: Executable Video Alignment  2.1 Inverse Dynamics Model (IDM)  The IDM predicts robot actions from short temporal windows of visual observations:  \\(\\mathcal{L}_{\\text{IDM}} = \\mathbb{E}\\left[\\sum_t |f_\\phi(I_{t-k:t+k}) - a_t^{gt}|_2^2\\right]\\)  Architecture: convolutional backbone → spatial softmax → MLP. The spatial softmax produces keypoint-like 2D coordinates per channel, which proves more stable than global pooling when decoding actions from generated (potentially artifact-laden) rollouts.  2.2 IDM-based Executability Reward  Given a generated video \\(V\\), the frozen IDM predicts joint commands \\(A = {a_t}_{t=1}^T\\). The reward evaluates the action sequence along two axes:  Smoothness penalties — Huber penalties on acceleration and jerk (finite differences of IDM-decoded actions):  \\(P_\\alpha = \\mathbb{E}_t[\\text{Huber}(\\alpha_t; \\delta_\\alpha)], \\quad P_j = \\mathbb{E}_t[\\text{Huber}(j_t; \\delta_j)]\\)  Embodiment limit penalties — penalize violations of velocity and acceleration bounds:                    \\(P_{\\text{vel}} = \\mathbb{E}_t|\\max(        v_t        - v_{\\max}, 0)|_2^2, \\quad P_{\\text{acc}} = \\mathbb{E}_t|\\max(        \\alpha_t        - a_{\\max}, 0)|_2^2\\)              The total penalty is combined into a bounded reward:  \\(R(V) = \\left(\\frac{1 + P(A)}{P_0}\\right)^{-\\gamma}\\)  where \\(P_0\\) is estimated from rollouts of the pretrained model, and \\(\\gamma\\) controls decay rate.  Key insight: the reward remains informative even when generated videos contain severe visual artifacts, because such artifacts typically translate into unstable or out-of-bound actions — providing a strong penalty signal.  2.3 RL Post-Training via GRPO  EVA uses Flow-GRPO (Group Relative Policy Optimization adapted for flow-matching models) to fine-tune the video generator:      Sample \\(G=8\\) rollouts per prompt from a stochastic SDE derived from the flow model    Score each rollout with the IDM-based reward    Compute group-relative advantages: \\(\\hat{A}_i = (R_i - \\mu_R) / (\\sigma_R + \\epsilon)\\)    Optimize using clipped policy gradient with KL regularization against the reference model    The IDM is kept frozen during GRPO fine-tuning — it serves purely as a reward model.  3. Experiments and Main Results  Base Model      Wan2.1-14B DiT backbone with diffusion forcing    Initialized from the Large Video Planner (LVP) checkpoint    SFT on embodiment-specific data → then GRPO with IDM reward    LoRA rank 32, 8× A800 GPUs, batch size 32    Visual Rollout Quality (Human Evaluation, 210 prompts)                    Method        Kinematic        Interaction        Instruction        Perfect                            Vidar (Wan2.2)        67.6%        66.7%        87.6%        62.9%                    EVA (w/o RL)        70.5%        83.3%        90.5%        68.1%                    EVA (with RL)        91.4%        86.2%        89.5%        83.8%              EVA improves kinematic plausibility by +20.9% and perfect execution by +15.7% over the SFT-only baseline.  Simulation (RoboTwin 2.0, 21 bimanual tasks)                    Method        Average Success                            ACT        29.0%                    Diffusion Policy        29.5%                    RDT        37.1%                    π₀        45.7%                    EVA (w/o RL)        46.2%                    EVA (with RL)        52.6%              EVA outperforms strong VLA baselines (π₀) while using a single multi-task policy across all 21 tasks (baselines are per-task).  Real-World Deployment (Agilex CobotMagic bimanual platform)                    Method        Seen (5 tasks)        OOD (5 tasks)                            ACT        42.0%        N/A                    π₀        51.0%        11.0%                    Vidar        44.0%        34.0%                    GE-Act        43.0%        3.0%                    EVA (w/o RL)        52.0%        42.0%                    EVA (with RL)        64.0%        60.0%              Particularly strong OOD generalization: 60% success on novel tasks, far exceeding π₀ (11%) and GE-Act (3%).  IDM Validation  The IDM achieves 89.52% success rate when decoding ground-truth video demonstrations on RoboTwin, confirming it is a reliable reward model for the alignment phase.  4. Strengths      Elegant problem formulation: the executability gap is a real and underexplored issue — using the IDM as both action decoder and reward model is a clean, dual-purpose design    Dense reward from action space: the reward remains informative even for badly artifact-laden videos, since visual artifacts reliably produce kinematic violations    Strong OOD generalization: the video world model paradigm + alignment produces the best generalization to novel tasks among all tested methods    Practical simplicity: only requires an IDM trained on real robot data + domain knowledge about joint limits — no learned value function or human feedback needed    5. Limitations      No contact dynamics modeling: the reward focuses on kinematic smoothness but does not model forces, friction, or torques — critical for precision contact-rich tasks    Computational cost: diffusion-based video generation is expensive, limiting applicability to high-frequency reactive control    Open-loop execution: the receding-horizon approach helps, but true closed-loop control with video world models remains challenging    IDM as bottleneck: the 89.52% IDM accuracy means ~10% of failures may stem from the IDM itself, not the video generator    6. Takeaways      Executability as an alignment target: shifting from visual/semantic fidelity to physical feasibility is a promising direction for robotic world models. The action space provides a natural, dense signal for alignment.    IDM dual use: training an IDM for action decoding and then repurposing it as a reward model is efficient — no additional reward model training needed.    Video world models generalize better: the decoupled paradigm (video planner + IDM) consistently shows stronger OOD performance than end-to-end VLA policies, likely due to leveraging internet-scale video priors.    RL post-training works for video generation: the success of Flow-GRPO for aligning video world models mirrors the RLHF paradigm in LLMs, suggesting this could become standard practice for robotic video generation.    摘要  EVA（Executable Video Alignment，可执行视频对齐） 识别并解决了机器人视频世界模型中的可执行性差距——即视觉上连贯的生成视频仍然可能产生不可行的机器人动作。核心思路：在真实机器人数据上训练一个逆动力学模型（IDM），然后将其复用为奖励模型，通过 GRPO 对视频生成器进行后训练。奖励函数惩罚非平滑运动（高加速度/冲击）和超出范围的动作，直接将生成视频与物理可执行性对齐。在 RoboTwin 基准和真实双臂机器人上，EVA 将运动学合理性提升了 +20.9%，真实世界任务成功率从 52% 提升至 64%（已见任务），从 42% 提升至 60%（分布外任务）。  论文信息      标题: Aligning Video World Models with Executable Robot Actions via Inverse Dynamics Rew..."
    },{
      "title": "EgoVerse: An Egocentric Human Dataset for Robot Learning from Around the World",
      "url": "/posts/2026/04/egoverse-paper-notes/",
      "date": "2026-04-10",
      "tags": ["Robotics","Egocentric Vision","Human-to-Robot Transfer","Imitation Learning","Dataset","Scaling Laws","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  This paper introduces EgoVerse, a collaborative, continuously growing ecosystem for collecting, processing, and learning from egocentric human demonstrations for robot manipulation. The dataset currently contains 1,362 hours of human data across 1,965 tasks, 240 scenes, and 2,087 demonstrators, contributed by a consortium of academic labs and industry partners. Beyond the dataset itself, the paper presents the first large-scale, cross-lab, cross-embodiment study of human-to-robot transfer, finding that co-training with human data consistently improves robot performance, but that domain-aligned data is essential to anchor effective scaling, and that scene diversity matters more than raw data volume for generalization under limited budgets.  Paper Info  The paper is “EgoVerse: An Egocentric Human Dataset for Robot Learning from Around the World,” led by Ryan Punamiya and Simar Kareer at Georgia Institute of Technology, with a large multi-institution team spanning Stanford, UC San Diego, ETH Zürich, MIT, Meta Reality Labs, Mecka AI, and Scale AI. Academic PIs include Marc Pollefeys, Robert Katzschmann, Xiaolong Wang, Shuran Song, Judy Hoffman, and Danfei Xu. The project page is at egoverse.ai and code at github.com/GaTech-RL2/EgoVerse.  1. Problem and Motivation  Robot learning increasingly depends on large, diverse data. But collecting robot demonstrations is expensive: it requires physical hardware, expert teleoperation, and controlled setups. Expanding robot datasets in scale and diversity remains slow and difficult to sustain.  Egocentric human data offers a compelling alternative. Humans naturally perform manipulation tasks in diverse environments every day, generating behavioral data at a scale infeasible for robots. Human data also provides a unifying abstraction — researchers can focus on curating diverse experience data while deferring embodiment decisions downstream.  But two major challenges remain. First, effective human-to-robot transfer is still an open problem, with unresolved questions about the embodiment gap and scaling behavior. Second, most existing human datasets are static, one-off releases collected for a specific study, making them hard to extend and fragment across institutions.  EgoVerse addresses both: it provides an ever-growing dataset ecosystem with standardized collection and annotation protocols, paired with a systematic consortium-scale study of when and how human data actually helps robot learning.  2. The EgoVerse Dataset  The dataset has two complementary components.  EgoVerse-A (Academic)  Collected under carefully controlled and standardized protocols across participating labs, designed for reproducible studies. Academic partners use Project Aria glasses (75g head-worn devices with wide-FoV RGB + two monochrome scene cameras for SLAM and hand tracking) as the standardized capture platform.  Data is organized around dataset units — each following a common instruction format with ~5 minutes of recording yielding 5–10 demonstrations per task. Six flagship tasks are shared across all labs:      object-in-container: pick, place, dump, repeat (single-arm)    cup-on-saucer: reorient a cup and place on saucer (bimanual)    bag-grocery: open bag, load 1–3 items (bimanual, long-horizon)    fold-clothes: three-fold a T-shirt (bimanual)    scoop-granular: scoop and transfer granular material (single-arm)    sort-utensils: pick and sort into containers (single-arm)    Diversity is structured along three axes: task (the flagship tasks), scenario (8–12 scenes per task, 1–10 dataset units per scene), and demonstrator (1–8 per lab).  EgoVerse-I (Industry)  The largest action-labeled egocentric human dataset, comprising nearly 1,400 hours across ~2,000 tasks, 240 scenes, and 2,087 demonstrators. Collected using custom wearable sensor platforms with stereo fisheye RGB cameras. Focuses on scale, diversity, and annotation richness — including fine-grained (1–2s) language descriptions, active-hand indicators, and manipulation flags. Categories span logistics (15.4%), cooking (13.7%), cleaning (11.6%), laundry (10.9%), and more.  Annotations  For each frame, EgoVerse estimates 3D hand poses (21 keypoints per hand in camera frame) paired with calibrated 6-DoF head pose from visual-inertial SLAM. Academic partners use Project Aria’s Machine Perception Service; industry datasets combine partner SLAM, model-based pose estimation, and post-processing.  EgoDB  A cloud-based data management system supporting continuous ingestion from all sources. Data flows to S3-backed storage, gets processed nightly into a unified training-ready format, and is registered in a centralized SQL database. Users can sync filtered subsets via configuration files for local training.  3. The EgoVerse Study: Human-to-Robot Transfer  This is where the paper becomes more than a dataset release. The authors conduct a consortium-scale evaluation of human-to-robot transfer that is reproducible by design — experiments are replicated across multiple independent labs, tasks, and robot embodiments.  Robot Platforms  Three distinctive robots are used:      Robot A: Two 6-DoF ARX5 arms with parallel jaw grippers, upright mount, Aria glasses + wrist RealSense cameras    Robot B: Two ARX5 arms on custom 3D-printed shoulder structure for human-like workspace, Aria glasses + wrist webcams    Robot C: Unitree G1 with 7-DoF arms and 6-DoF Dexterous Inspire Hands, ZED 2 stereo camera    Action Representations  A careful design decision: human hand poses in the moving camera frame are projected into camera-centered stable reference frames, constructing actions as future hand trajectories relative to the current device frame. This gives a common representation that can serve as proxies for robot end-effector motion across embodiments.\\[a^H_{t:t+k} = \\left[ \\left(T_t^{\\text{device}}\\right)^{-1} \\cdot T_{t+i}^{\\text{device}} \\cdot p_{t+i}^H \\right]_{i=1}^k\\]  Policy Architecture  An encoder-decoder architecture with modality-specific stems. Image observations go through a ResNet-18 backbone; proprioceptive inputs through an MLP. A shared vision stem processes egocentric RGB from both human and robot embodiments. A shared transformer encoder \\(f_\\phi\\) fuses multi-modal tokens via learned query attention, and a flow matching action decoder \\(\\pi_\\theta\\) (multi-block transformer decoder trained with conditional flow matching loss) generates actions.  The co-training loss is straightforward:\\[\\mathcal{L}_{\\text{BC-cotrain}}(\\phi, \\theta) = \\mathbb{E}_{(o,a) \\sim \\mathcal{D}_H \\cup \\mathcal{D}_R} [\\mathcal{L}_{\\text{BC}}(\\pi_\\theta(f_\\phi(o)), a)]\\]  In practice, each training step computes flow matching loss on a mini-batch of both human and robot samples.  Evaluation  Four flagship tasks evaluated on all three robots, with 20 in-domain (ID) and 20 out-of-domain (OOD) rollouts per task. Performance measured using task-specific subtask metrics and reported as a normalized score.  4. Key Findings  Finding 1: Co-training with human data consistently improves robot performance  Joint training with EgoVerse-A data improves both in-domain performance and out-of-domain generalization across robots. OOD improvements reach up to 30%. This is the first time this effect is validated under a standardized, cross-lab setup spanning multiple robots.  Finding 2: Domain-aligned data is essential to anchor scaling  This is the most nuanced and important finding. Scaling benefits depend critically on the availability of aligned human-robot data — human and robot data that share task semantics and scene context. Neither 8 hours of diverse EgoVerse-A data nor domain-aligned human data alone drives significant performance gains. But when domain-aligned data is included as part of training, positive scaling emerges: just 2 hours of domain-aligned data facilitates transfer from 2 hours of diverse EgoVerse-A data, a trend that scales further as diverse data increases to 8 hours.  In other words, aligned data acts as an anchor that teaches the policy how to bridge the embodiment gap, and only then can diverse human data contribute additional knowledge.  Finding 3: Different forms of diversity contribute unevenly  Under controlled conditions (the Controlled-Diversity Subset with 16 demonstrators × 16 scenes):      Demonstrator diversity consistently improves generalization to unseen demonstrators. UMAP visualizations show increased feature overlap between training and validation demonstrators as diversity grows.    Scene diversity improves generalization to unseen scenes, with the strongest gains under limited data budgets. Beyond a certain data quantity, adding more data in existing scenes yields diminishing returns, while expanding scene coverage continues to help.    When jointly scaling both, scene diversity improves under both demonstrator budgets, while the marginal benefit of additiona..."
    },{
      "title": "Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Better",
      "url": "/posts/2026/03/knowledge-insulation-vla/",
      "date": "2026-03-30",
      "tags": ["Vision-Language-Action","Robot Learning","Knowledge Transfer","Flow Matching","VLM Fine-tuning","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  This paper from Physical Intelligence identifies a critical problem in training continuous-action VLAs: naively adding a flow-matching action expert to a pre-trained VLM backbone degrades both training speed and knowledge transfer, because gradients from the randomly initialized action expert corrupt the backbone’s pre-trained representations. The proposed fix — knowledge insulation — is elegant: train the VLM backbone with discrete (FAST-tokenized) actions via next-token prediction, while simultaneously training a smaller action expert with flow matching on continuous actions, but stop the gradient from the action expert back into the backbone. This yields a model that trains as fast as π₀-FAST, runs fast at inference (via the small action expert), follows language instructions better, and generalizes more effectively — all by preserving the VLM’s pre-trained knowledge.  Paper Info      Title: Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Better    Authors: Danny Driess, Jost Tobias Springenberg, Brian Ichter, Lili Yu, Adrian Li-Bell, Karl Pertsch, Allen Z. Ren, Homer Walke, Quan Vuong, Lucy Xiaoyang Shi, Sergey Levine    Affiliation: Physical Intelligence    Date: 2025 (preprint, under review)    Project page: pi.website/research/knowledge_insulation    1. Motivation  Vision-language-action (VLA) models promise to bring web-scale VLM knowledge to robot control. But there’s a tension:                    Approach        Inference Speed        Action Quality        VLM Knowledge Retention                            Autoregressive VLAs (e.g., RT-2, π₀-FAST)        Slow (~750ms/chunk)        Discrete, lossy        Good (next-token prediction)                    Continuous-action VLAs (e.g., π₀)        Fast (10 Hz)        High-fidelity, smooth        Poor (gradient interference)              The core dilemma: adding a continuous action expert (diffusion/flow matching head) to a VLM introduces randomly initialized parameters whose gradients damage the pre-trained backbone, hurting language following and generalization. Simply freezing the backbone doesn’t work either — VLM representations alone are insufficient for robotics without fine-tuning.  2. Method: Knowledge Insulation  The recipe has three key ingredients:  2.1 Joint Discrete/Continuous Action Prediction  Train the model to predict both discrete and continuous actions simultaneously:                    \\(\\mathcal{L}{\\text{CO-VLA}}(\\theta) = \\mathbb{E}\\left[-\\sum{j} M_j^{\\ell} \\log p_\\theta(\\hat{\\ell}_{j+1}        x_{1:j}) + \\alpha M^{\\text{act}} | \\omega - a_{1:H} - f_\\theta^a(a_{1:H}^{\\tau,\\omega}) |^2 \\right]\\)                  The VLM backbone is trained on discrete action tokens (FAST tokenization) via standard next-token prediction — this provides a clean learning signal    A separate action expert (300M parameter transformer) is trained with flow matching on continuous action chunks    At inference time, only the smaller action expert is used → fast continuous control    2.2 Stop-Gradient (Knowledge Insulation)  The critical innovation: stop the gradient flow from the action expert back to the VLM backbone. The action expert can read backbone features (via cross-attention), but its gradients don’t write back into them:  \\(P_{ab} = \\text{softmax}\\left(Q_a(X_a) \\cdot \\text{sg}(K_b(X_b))^T + A\\right)\\)  where sg is the stop-gradient operator. Value embeddings from the backbone are similarly detached. This means:      The backbone learns only from the clean autoregressive (discrete action + language) loss    The action expert learns to use backbone features without corrupting them    2.3 VLM Data Co-training  Co-train the model on general VLM tasks (image captioning, VQA, object localization) alongside robot data. This further preserves pre-trained knowledge and improves language following and generalization to novel objects.  3. Architecture Details      VLM Backbone: PaliGemma (2B language model, 3B total), initialized from pre-trained weights    Action Expert: 300M parameter transformer with separate Q/K/V projections    Action Representation: FAST tokenization for discrete actions (training signal for backbone), flow matching for continuous actions (used at inference)    State Representation: Text state or continuous state both work well; special token state is worse    Embeddings interact via self-attention with a carefully designed mask — information flows unidirectionally from VLM to action expert    4. Experiments and Main Results  Real-World Tasks  Evaluated on complex, long-horizon manipulation tasks across multiple robot embodiments:                    Task        Robot Type        Key Finding                            Items in drawer        Static single-arm        Ours significantly outperforms all baselines (p<0.001 vs most)                    Table bussing        Static single-arm        Ours best performance + fast inference; π₀-FAST 2× slower                    T-shirt folding        Static bimanual        Ours matches or exceeds π₀-FAST (p=0.765)                    Mobile manipulation (4 tasks)        Mobile bimanual        Ours w/ VLM data clearly best              Key Quantitative Results      vs π₀: The proposed method significantly outperforms π₀ on language following and task performance. π₀ struggles because its action expert gradients degrade the backbone    vs π₀-FAST: Comparable task performance, but 2× faster wall-clock time (π₀-FAST requires slow autoregressive decoding at ~750ms per chunk)    vs joint-training (no stop-grad): Stop-gradient consistently improves language following; without it, the backbone gets corrupted similarly to π₀    Training speed: Converges as fast as π₀-FAST, while π₀ requires 7.5× more training steps for similar performance    Simulation Benchmarks                    Benchmark        LIBERO-90        LIBERO-Spatial                            π₀        85.2        96.8                    π₀-FAST        60.2        96.8                    OpenVLA-OFT        94.5        97.6                    Ours (generalist)        96.0        98.0              State-of-the-art on LIBERO-90 and LIBERO-Spatial.  DROID Benchmark  Score of 0.55 ± 0.09 vs π₀ at 0.49 ± 0.09 and π₀-FAST at 0.45 ± 0.09.  Language Following  Stopping the gradient flow from the action expert is an effective way to improve language following. Co-training on VLM data further enhances this. The model pays more attention to language inputs rather than just overfitting to visual patterns.  Generalization to Novel Objects  Co-training on VLM data is particularly important for OOD generalization — the model transfers semantic knowledge from captioning/VQA tasks to robotic manipulation of unseen objects.  5. Ablation Highlights      Freezing backbone: 0% performance — VLM representations alone aren’t enough for robotics    HybridVLA (allows AR tokens to attend to flow-matching inputs): Significantly worse than the proposed masking strategy    Naive tokenization vs FAST: FAST provides a better representation learning signal, though naive tokenization still works    Without VLM data co-training: Slightly worse task completion, significantly worse language following on joint-training    6. Strengths      Clean, principled solution to a well-identified problem (gradient interference from action experts)    Comprehensive experimental evaluation across diverse real-world tasks and embodiments    Achieves the best of both worlds: fast training (like FAST), fast inference (like π₀), strong language following and generalization    The three ingredients (joint training, stop-gradient, VLM co-training) are independently ablated    7. Limitations      Training with both discrete and continuous outputs increases computational cost by ~20% (offset by faster convergence)    Language following, while improved, is still not perfect — training data distribution still causes occasional language instruction ignoring    Evaluation limited to the π₀/PaliGemma architecture family    8. Takeaways      Gradient interference is real and severe: Randomly initialized action experts can badly damage pre-trained VLM representations. This is a fundamental issue for any VLA that adds continuous action heads to pre-trained backbones.    Stop-gradient is a simple but powerful fix: By insulating the backbone from action expert gradients, you preserve pre-trained knowledge while still allowing the action expert to leverage backbone features.    Discrete tokens as a representation learning signal: Even if you want continuous actions at inference, training the backbone with discrete action tokens provides a cleaner, more compatible learning signal.    VLM co-training matters: Mixing in general VLM tasks during VLA training is not just regularization — it actively helps language following and semantic generalization.    Architecture design matters as much as ..."
    },{
      "title": "Do World Action Models Generalize Better than VLAs? A Robustness Study",
      "url": "/posts/2026/03/wam-vla-robustness-paper-notes/",
      "date": "2026-03-24",
      "tags": ["Robotics","World Models","Vision-Language-Action","Robustness","Benchmarking","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR      WAMs are strongest on visual perturbations. LingBot-VA reaches 74.2% on RoboTwin 2.0-Plus, and Cosmos-Policy reaches 82.2% on LIBERO-Plus.    But WAMs do not universally dominate. On LIBERO-Plus, pi0.5 reaches 85.7%, beating the best WAM overall.    The likely advantage of WAMs is that video pretraining gives them stronger spatiotemporal priors, so downstream policy training does not need to learn dynamics from scratch as aggressively.    The biggest practical downside is speed: the evaluated WAMs are about 4.8x to 83.0x slower than pi0.5 per action chunk.    Paper Info      Title: Do World Action Models Generalize Better than VLAs? A Robustness Study    Authors: Zhanguang Zhang, Zhiyuan Li, Behnam Rahmati, Rui Heng Yang, Yintao Ma, Amir Rasouli, Sajjad Pakdamansavoji, Yangzheng Wu, Lingfeng Zhang, Tongtong Cao, Feng Wen, Xingyue Quan, Yingxue Zhang    Affiliation: Huawei Technologies; University of Toronto    Date: 2026-03-23    Venue: arXiv preprint    arXiv: 2603.22078    1. Question and Setup  The paper asks a simple but important question: do world action models (WAMs) actually generalize more robustly than vision-language-action models (VLAs)?  To test this, the authors evaluate open-source or publicly released policies on two perturbation-heavy benchmarks:      LIBERO-Plus: single-arm Franka manipulation    RoboTwin 2.0-Plus: a new dual-arm benchmark built on RoboTwin 2.0    Both benchmarks perturb the policy along seven axes:      Camera    Robot initial state    Language    Light    Background    Noise    Layout    The model pool spans three families:      VLAs: pi0, pi0.5, OpenVLA-OFT, X-VLA, UniVLA, RIPT-VLA    Hybrid approaches: MOTUS, VLA-JEPA    WAMs: GE-Act, Cosmos-Policy, LingBot-VA    One caveat matters: DreamZero is discussed in the taxonomy, but excluded from quantitative evaluation because of dataset mismatch and very high training/inference cost.  2. What distinguishes WAMs from VLAs?  The paper frames the difference cleanly:      VLA: predict the next action directly from the current history, p_theta(a_t | h_t)    WAM: jointly predict future state and action, p_phi(h_{t+1}, a_t | h_t), or first predict future state and then generate action, p_phi(h_{t+1} | h_t) * g_psi(a_t | h_t, h_{t+1})    So the main distinction is not just “language model backbone” versus “video model backbone”. It is also:      direct action prediction versus    future-state-aware action generation    Because WAM backbones are pretrained for video generation, the authors argue they inherit stronger spatiotemporal priors from web-scale video pretraining.  3. Main Results  Overall outcome                    Benchmark        Best WAM        Total        Best overall model        Total        Main read                            RoboTwin 2.0-Plus        LingBot-VA        74.2        LingBot-VA        74.2        WAMs lead clearly on this dual-arm robustness benchmark                    LIBERO-Plus        Cosmos-Policy        82.2        pi0.5        85.7        A strong VLA beats the best WAM overall              This is why I would read the paper’s title question as “often yes, but not always.”  Where WAMs look strongest      On RoboTwin 2.0-Plus, LingBot-VA is especially strong under light (89.0), background (91.3), noise (80.9), and layout (87.9) perturbations.    On LIBERO-Plus, Cosmos-Policy and GE-Act also show strong robustness under light, noise, and layout perturbations.    Hybrid methods such as MOTUS and VLA-JEPA also improve robustness, which suggests that even partial integration of video/dynamics learning helps.    Where WAMs still struggle      Camera viewpoint changes remain hard. On RoboTwin 2.0-Plus, LingBot-VA drops to 28.9 under camera perturbation.    Robot initial-state perturbations are another weak spot. LingBot-VA gets 36.2 on RoboTwin, and Cosmos-Policy trails pi0.5 on LIBERO-Plus robot perturbations (63.3 vs 77.5).    So the current evidence says: WAMs are especially good at visual robustness, but not automatically better under geometry or embodiment shifts.  4. Why WAMs help  The paper’s explanation is intuitive:      video backbones are pretrained on temporally rich internet-scale videos    that pretraining teaches fine-grained visual dynamics    downstream policy training can therefore focus more on action generation, instead of learning dynamics from scratch    Table 2 is one of the most interesting parts of the paper. It highlights how different the training pipelines are:      Cosmos-Policy uses only 185 task trajectories for task-specific finetuning    pi0.5 relies on a much broader recipe: robot data, web captioning/VQA/grounding data, high-level planning data, and post-training stages    My read: the paper supports a training-efficiency story more strongly than a pure architecture-only story. WAMs seem to buy robustness more cheaply, while strong VLAs can catch up or even surpass them when backed by a richer data pipeline.  5. Runtime and Practical Limits  The paper is also very clear about the main downside: WAM inference is slow.                    Model        Inference time per chunk                            pi0.5        63 ms                    X-VLA        195 ms                    GE-Act        300 ms                    Cosmos-Policy        390 ms                    LingBot-VA (real-world setting)        480 ms                    MOTUS        1175 ms                    LingBot-VA (RoboTwin setting)        5230 ms              The authors attribute much of this gap to the state denoising process inside WAMs. Even the faster WAMs in this study are at least 4.8x slower than pi0.5.  Other limitations worth keeping in mind:      RoboTwin 2.0-Plus is an in-house benchmark, so external replication will take time    DreamZero is not included in the quantitative benchmark comparison    the compared methods do not use matched data pipelines, so this is not a perfectly controlled apples-to-apples study    6. Takeaways      This paper supports “WAMs are a strong robustness prior” more than “WAMs always generalize better than VLAs.”    WAMs look especially attractive for visual perturbations such as light, noise, and cluttered layouts.    Strong VLAs can still win overall when they are trained with sufficiently rich and diverse data, as pi0.5 shows on LIBERO-Plus.    Hybrid methods matter because they show that importing temporal/video structure into VLA pipelines already helps a lot.    Inference efficiency is the biggest deployment bottleneck for WAMs today.    本文支持通过顶部导航栏的语言切换按钮在 English / 中文 之间切换。  TL;DR      WAMs 在视觉扰动上最有优势。 LingBot-VA 在 RoboTwin 2.0-Plus 上达到 74.2%，Cosmos-Policy 在 LIBERO-Plus 上达到 82.2%。    但 WAMs 并不是在所有场景都占优。 在 LIBERO-Plus 上，pi0.5 取得 85.7%，总体上超过了最强 WAM。    WAMs 的核心优势很可能在于：视频预训练给了它们更强的时空先验，因此下游策略训练不必像 VLA 那样更激进地从零学习世界动力学。    最大的现实代价是 推理速度：论文中评估的 WAM 每个 action chunk 大约比 pi0.5 慢 4.8 倍到 83.0 倍。    论文信息      标题: Do World Action Models Generalize Better than VLAs? A Robustness Study    作者: Zhanguang Zhang, Zhiyuan Li, Behnam Rahmati, Rui Heng Yang, Yintao Ma, Amir Rasouli, Sajjad Pakdamansavoji, Yangzheng Wu, Lingfeng Zhang, Tongtong Cao, Feng Wen, Xingyue Quan, Yingxue Zhang    机构: Huawei Technologies；University of Toronto    日期: 2026-03-23    发表形式: arXiv preprint    arXiv: 2603.22078    1. 论文问题与实验设置  这篇论文在问一个很直接但很关键的问题：世界动作模型（WAMs）是否真的比视觉-语言-动作模型（VLAs）更鲁棒、更会泛化？  作者在两个高扰动基准上评测公开模型：      LIBERO-Plus：单臂 Franka 操作任务    RoboTwin 2.0-Plus：作者基于 RoboTwin 2.0 构建的双臂新基准    两个基准都沿着七个维度对策略进行扰动：      相机    机器人初始状态    语言    光照    背景    噪声    布局    模型大致分为三类：      VLAs: pi0, pi0.5, OpenVLA-OFT, X-VLA, UniVLA, RIPT-VLA    混合方法: MOTUS, VLA-JEPA    WAMs: GE-Act, Cosmos-Policy, LingBot-VA    有一个重要限制需要注意：DreamZero 虽然出现在 WAM 分类表里，但由于数据集不匹配以及训练/推理成本很高，被排除在定量评测之外。  2. WAM 和 VLA 的区别到底是什么？  论文把两者的区别总结得很清楚：      VLA: 直接根据当前历史预测动作，p_theta(a_t | h_t)    WAM: 联合预测未来状态和动作，p_phi(h_{t+1}, a_t | h_t)，或者先预测未来状态再生成动作，p_phi(h_{t+1} | h_t) * g_psi(a_t | h_t, h_{t+1})    所以它们的区别不只是“语言模型骨干”和“视频模型骨干”的差异，更是直接动作预测与基于未来状态的动作生成之间的差异。  由于 WAM 的骨干是视频生成模型，作者认为它们从大规模网络视频预训练中继承了更强的时空先验。  3. 主要结果  整体结论                    基准        最强 WAM        总分        全表最强模型        总分        结论                            RoboTwin 2.0-Plus        LingBot-VA        74.2        LingBot-VA        74.2        在这个双臂鲁棒性基准上，WAM 明显领先                    LIBERO-Plus        Cosmos-Policy        82.2        pi0.5        85.7        强 VLA 在总分上超过了最强 WAM              因此，我会把论文标题里的问题理解成：很多时候是，但不是永远如此。  WAM 最强的地方      在 RoboTwin 2.0-Plus 上，LingBot-VA 在 光照 (89.0)、背景 (91.3)、噪声 (80.9)、布局 (87.9) 扰动下表现尤其强。    在 LIBERO-Plus 上，Cosmos-Policy 和 GE-Act 在 光照、噪声、布局 等视觉扰动上也表现很稳。    MOTUS 和 VLA-JEPA 这样的混合方法同样带来了鲁棒性提升，这说明即便只是部分引入视频/动力学学习，也会有明显帮助。    WAM 仍然较弱的地方      相机视角变化仍然很难。在 RoboTwin 2.0-Plus 上，LingBot-VA 在相机扰动下掉到 28.9。    机器人初始状态扰动也是弱项。LingBot-VA 在 RoboTwin 上只有 36.2，而 Cosmos-Policy 在 LIBERO-Plus 的机器人状态扰动上也明显落后于 pi0.5（63.3 对 77.5）。    所以当前证据更像是在说：WAM 在视觉鲁棒性上特别强，但在几何变化或具身变化下并不..."
    },{
      "title": "RL Token: Bootstrapping Online RL with Vision-Language-Action Models",
      "url": "/posts/2026/03/rlt-paper-notes/",
      "date": "2026-03-24",
      "tags": ["Robotics","Reinforcement Learning","Vision-Language-Action","Online RL","Sample Efficiency","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  This paper tackles a very practical problem in robotics: pretrained vision-language-action models can already do a surprising amount, but they often break down in the last few millimeters of a task, exactly where precision and speed matter most. The authors propose RL Token (RLT), a lightweight way to fine-tune a pretrained VLA with online reinforcement learning in the real world without trying to RL-train the whole giant model.  The key move is to make the frozen VLA expose a compact RL token, then train a small actor-critic on top of that token while keeping the policy anchored to the VLA’s suggested action chunk. On four real-robot precision tasks, this leads to large improvements within minutes to a few hours of robot practice, including up to 3x speedup in the hardest phase of the task and substantial gains in success rate.  Paper Info  The paper is “RL Token: Bootstrapping Online RL with Vision-Language-Action Models,” by Charles Xu, Jost Tobias Springenberg, Michael Equi, Ali Amin, Adnan Esmail, Sergey Levine, and Liyiming Ke from Physical Intelligence. The PDF points to the project page at pi.website/research/rlt.  1. Motivation  The starting point is easy to sympathize with. Modern VLAs can already solve a broad set of manipulation tasks from demonstrations, but their performance on a specific task is capped by the quality of the demonstration data. That becomes painfully obvious on tasks like screw insertion, zip tie fastening, or cable insertion, where tiny alignment errors lead to hesitation, repeated probing, or outright failure.  Reinforcement learning is the obvious tool for pushing beyond that ceiling, because RL can optimize exactly the part of the behavior that matters most for success. The problem is that real-world robot RL is operating under a harsh data budget. You usually do not get millions of trials. You get minutes or hours of robot time, a sparse success signal, and a limited tolerance for breakage, wear, and operator overhead.  That tension defines the paper. Full-scale RL fine-tuning of a giant VLA is too expensive and sample-inefficient, but throwing away the VLA and training a small policy from scratch also throws away the representation and behavioral prior that made the VLA useful in the first place. RLT tries to sit between those two extremes.  2. Core Idea  The method builds a compact RL interface on top of a pretrained π0.6 VLA. Rather than directly updating the whole model online, the authors first train the VLA to expose an RL token, a compressed representation extracted from its internal embeddings. This token is meant to preserve task-relevant information while being small enough that a lightweight actor and critic can actually learn from it online.  Concretely, they append a learned special token to the VLA’s internal embedding sequence, run a small encoder over that sequence, and use the resulting token as a bottleneck. A decoder is then trained to reconstruct the original VLA embeddings from this compact representation. The reconstruction objective is what forces the RL token to stay informative rather than becoming an arbitrary low-dimensional projection.  Once that representation is trained, the VLA and token extractor are frozen. Online RL then happens only in a small actor-critic head. The critic estimates chunk-level value, and the actor does not generate behavior from scratch. Instead, it receives both the RL token and a reference action chunk sampled from the VLA. This changes the nature of RL from open-ended search to local refinement around a competent prior.  The actor objective reflects that design directly:\\[\\mathcal{L}_{\\pi}(\\theta) = \\mathbb{E}\\left[-Q_{\\psi}(x, a_{1:C}) + \\beta \\lVert a_{1:C} - \\tilde{a}_{1:C}\\rVert_2^2 \\right]\\]  Here, \\(a_{1:C}\\) is the actor’s chunked action, \\(\\tilde{a}_{1:C}\\) is the VLA’s sampled reference chunk, and the regularization term keeps the RL policy near the base VLA unless the critic has a good reason to move away.  That anchoring term is important because the paper is not trying to rediscover robot behavior from scratch. It is trying to take a good-but-imperfect VLA and improve the small, high-precision parts where demonstrations are weakest.  3. Why the Method Is Structured This Way  Several choices in the paper are there to make online RL realistic rather than elegant.  First, the method uses action chunks rather than single-step actions. The base VLA predicts a 50-step chunk, while the RL policy operates on a shorter chunk of length 10. This shortens the effective credit-assignment horizon and makes sparse-reward temporal-difference learning much more plausible on real robots running at 50 Hz.  Second, the actor is conditioned on the VLA’s sampled reference chunk. That matters for two reasons. It preserves mode information from the VLA’s multimodal action distribution, and it means the RL policy is editing a promising behavior rather than inventing one from scratch. The paper argues, convincingly in my view, that this is one reason online learning can move so quickly.  Third, the authors add reference action dropout during training. Without it, the actor could simply copy the VLA proposal and never learn to improve it. By randomly zeroing out the reference chunk for some updates, they force the actor to maintain an independent pathway while still benefiting from the VLA prior whenever it is available.  Finally, the overall system is built around a practical intervention loop. The replay buffer contains VLA rollouts, online RL rollouts, and optional human interventions. A human supervisor also provides sparse success or failure labels. This is not yet a fully autonomous RL pipeline; it is a deliberately pragmatic one.  4. Experimental Setup  The experiments focus on four real-world manipulation tasks that all have a narrow, precision-critical bottleneck: screw installation, zip tie fastening, Ethernet insertion, and charger insertion. These are exactly the sorts of tasks where a generalist VLA can usually get close, but the hardest part still requires millimeter or sub-millimeter execution.  The authors evaluate the method in two regimes. In the critical-phase evaluation, the episode starts right before the most precision-sensitive part of the task. This isolates the part of the behavior that RL is supposed to improve. In the full-task evaluation, the robot starts from its home position and must arrive at the critical phase through the normal base-policy execution, which is harder because upstream variation now matters.  The base VLA is fine-tuned on 1 to 10 hours of demonstrations for each task, and then RLT is trained online for roughly 400 to 1000 episodes depending on difficulty. In actual robot time, the online data budget ranges from around 15 minutes to 5 hours, which is exactly the regime where a method like this needs to work to be practically interesting.  5. Main Results  The headline result is that RLT improves both success rate and throughput, where throughput measures successful completions per 10-minute interval and therefore captures speed as well as reliability. The gains are largest on the hard, contact-rich parts of the task.  In the controlled critical-phase setting, the paper reports up to 3x faster execution on the hardest portion of the task. On the challenging screw task, the paper highlights a jump in success rate from 20% to 65%. In the harder full-task setting, where upstream errors compound, RLT still improves overall success by about 40% on screw installation and 60% on zip tie fastening.  One of the most striking qualitative results comes from the Ethernet insertion task. The base VLA tends to probe, back off, readjust, and try again. The final RLT policy instead approaches more decisively and inserts in a fluid motion. The paper reports a median episode length of 228 timesteps for the base policy, 146 for expert teleoperation, and 66 for the RLT policy on the critical insertion phase. That is an unusually concrete example of online RL discovering a strategy that is not just more successful, but genuinely faster than the demonstrations it started from.  6. Comparison to Baselines  The baseline comparison helps clarify what is doing the real work here. The paper compares RLT against HIL-SERL, Probe-Learn-Distill, DSRL, and DAgger. The strongest pattern is that methods operating on single-step actions struggle badly in this setting. At 50 Hz with sparse rewards, the credit-assignment horizon simply becomes too long.  DAgger and DSRL remain more competitive, especially on the easier Ethernet task, but they do not deliver the same throughput improvement. That makes intuitive sense. DAgger is still imitation learning, so it is constrained by the speed and style of the human interventions. DSRL stays closer to the frozen VLA’s action manifold, which makes it stable but also limits how far it can move towar..."
    },{
      "title": "Discovering state-of-the-art reinforcement learning algorithms",
      "url": "/posts/2026/03/discorl-paper-notes/",
      "date": "2026-03-24",
      "tags": ["Reinforcement Learning","Meta-Learning","RL Algorithms","Automated Discovery","DiscoRL","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  DiscoRL turns RL algorithm design itself into a meta-learning problem. Instead of hand-writing TD targets, policy losses, or auxiliary objectives, the paper learns a meta-network that emits update targets for an agent’s policy and internal predictions.  This is not “meta-RL” in the usual fast-adaptation sense. The thing being meta-learned is the learning rule itself.  The headline result is strong:      Disco57 reaches IQM 13.86 on Atari, outperforming prior hand-designed RL rules on this benchmark    The discovered rule also transfers to unseen benchmarks such as ProcGen, Crafter, and NetHack    A broader discovery run, Disco103, gets even stronger by training on a more diverse set of environments    The big message is that RL algorithm discovery can scale with environment diversity and compute, much like model pretraining scales with data.  Paper Info      Title: Discovering state-of-the-art reinforcement learning algorithms    Authors: Junhyuk Oh, Gregory Farquhar, Iurii Kemaev, Dan A. Calian, Matteo Hessel, Luisa Zintgraf, Satinder Singh, Hado van Hasselt, David Silver    Affiliation: Google DeepMind    Venue: Nature, Vol. 648, 11 December 2025 issue    Published online: 2025-10-22    DOI: 10.1038/s41586-025-09761-x    Code: google-deepmind/disco_rl    1. Motivation  Most RL progress still comes from humans inventing better update rules: TD learning, Q-learning, PPO, auxiliary losses, distributional targets, and so on. Previous “automatic discovery” work usually searched a much narrower space:      tune a few hyperparameters    learn a scalar objective    meta-train in toy environments    This paper asks a more ambitious question:      Can we directly discover the RL rule that updates the agent, using only the cumulative experience of many agents interacting with complex environments?    That framing is what makes the paper interesting. It is not just learning a better policy. It is trying to learn how policies and predictions should be learned.  2. Core Idea  2.1 Agent outputs  The agent produces more than a policy:      a policy \\pi(s)    an observation-conditioned prediction vector y(s)    an action-conditioned prediction vector z(s, a)    an action-value head q(s, a)    an auxiliary policy prediction p(s, a)    The important part is that y and z do not have predefined semantics. They are slots in which the meta-learning process can invent useful internal predictions.  2.2 Meta-network as the discovered RL rule  A backward LSTM-based meta-network reads short trajectory segments containing:      agent outputs over time    rewards    episode termination signals    It then emits targets for the current policy and predictions:      \\hat{\\pi}    \\hat{y}    \\hat{z}    The agent is updated to move toward those targets. In simplified form:\\[\\mathcal{L}_{\\theta}=\\mathbb{E}\\left[D(\\hat{\\pi}, \\pi_{\\theta})+ D(\\hat{y}, y_{\\theta})+ D(\\hat{z}, z_{\\theta})+ \\mathcal{L}_{\\text{aux}}\\right]\\]  where the paper uses KL-style distances and auxiliary targets for the value and auxiliary-policy heads.  2.3 Why this is more expressive than tuning a loss scalar  The meta-network outputs targets, not just one scalar loss. That matters because it naturally includes:      bootstrapping from future predictions    joint policy-and-prediction updates    the possibility of inventing new prediction semantics beyond value functions    The authors argue this search space is strictly more expressive than only meta-learning a scalar objective.  2.4 Meta-optimization  The meta-parameters \\eta are improved so that agents trained under the discovered rule get higher return:\\[\\nabla_{\\eta} J(\\eta)\\approx\\mathbb{E}\\left[(\\nabla_{\\eta} \\theta)(\\nabla_{\\theta} J(\\theta))\\right]\\]  In practice, they backpropagate through a window of agent updates and use a large population of agents across many environments. This is what lets the “algorithm designer” receive gradient signal from actual downstream learning performance.  3. What DiscoRL Seems to Discover  One of the most interesting parts of the paper is that the discovered predictions are not just hidden value functions with different names.  The analysis suggests:      the discovered predictions often spike before salient events, such as large rewards    they encode information about future policy entropy    they attend to observation regions that are different from what the policy or value function focuses on    future predictions strongly affect current targets, showing an emergent bootstrapping mechanism    The takeaway is that DiscoRL seems to invent internal predictive signals that complement the usual policy/value decomposition rather than simply rediscovering it.  4. Main Empirical Results  4.1 Atari  The strongest single claim is on Atari:      Disco57 is discovered from the 57 Atari games themselves    it reaches IQM 13.86    it outperforms prior published hand-designed RL rules on Atari, including strong model-based and actor-critic baselines    The paper also emphasizes improved wall-clock efficiency relative to MuZero in this setting.  4.2 Generalization to unseen benchmarks  Even though Disco57 is discovered on Atari, it transfers to environments it never saw during discovery:      ProcGen: beats existing published methods in the paper’s comparison    Crafter: competitive performance    NetHack Challenge: reaches third place on the NeurIPS 2021 challenge leaderboard without domain-specific shaping or handcrafted subtasks    This is one of the paper’s most convincing results. A discovered rule is only interesting if it generalizes beyond the exact benchmarks used to create it.  4.3 More diverse discovery environments help  The second discovered rule, Disco103, uses a broader discovery set:      Atari    ProcGen    DMLab-30    Compared with Disco57, it achieves:      similar Atari performance    better scores on every other seen and unseen benchmark reported in the main figure    human-level performance on Crafter    near-MuZero state-of-the-art performance on Sokoban    This supports the paper’s key scaling claim: stronger and more diverse discovery environments produce a better RL rule.  4.4 Discovery efficiency and compute  The paper frames discovery as surprisingly efficient relative to manual algorithm design:      the best Atari rule emerges within roughly 600 million steps per game    this is about three full experiments per Atari game    But the absolute compute is still large:      Disco57: 128 agents, 1,024 TPUv3 cores for 64 hours    Disco103: 206 agents, 2,048 TPUv3 cores for 60 hours    So the story is not “cheap discovery”. It is “expensive, but now plausibly worth scaling”.  5. Why the Paper Matters      It expands the discovery space beyond hyperparameter tuning or minor loss shaping    It shows real benchmark competitiveness, not just toy-environment meta-learning    It provides evidence for novel internal predictions, which is more interesting than merely matching known RL rules    It turns algorithm design into a scaling problem over environments and compute    For me, that last point is the most important. The paper suggests that at least part of RL algorithm research may become a systems-and-scaling problem, not only a human-theory problem.  6. Limitations and Open Questions      Very high discovery cost: the method is still far from cheap or easy to reproduce    Handcrafted scaffolding remains: the agent architecture, auxiliary heads, KL-style update form, and meta-optimization recipe are all still designed by humans    Benchmark domain bias: the strongest evidence is on discrete-action game-like environments; the paper does not yet prove the same effect for continuous control or robotics    Interpretability remains partial: we know the discovered predictions matter, but we still do not have a clean semantic theory for what each learned signal represents    Commercial framing: the paper notes pending patent applications and Google ownership, which is worth keeping in mind for long-term openness    7. Takeaways      Learning rules themselves are now viable targets for scaling. This paper makes RL algorithm discovery feel more like pretraining than like manual feature engineering.    Searching over target-generating update rules is powerful. It is richer than tuning coefficients on a fixed PPO-style or TD-style loss.    Complex discovery environments matter. Toy meta-training is not enough if the goal is to invent algorithms that work in hard settings.    New predictive semantics may be a core ingredient of stronger RL. DiscoRL’s hidden predictions seem to carry information that standard policy/value heads miss.    This is a milestone, not the endpoint. The paper shows that machine-discovered RL rules can beat strong human-designed baselines, but it does not yet mean human algorithm design is obsolete.    References      [Paper] Nature DOI: 10.1038/s41586-025-09761-x    [Code] gi..."
    },{
      "title": "DreamZero: World Action Models are Zero-shot Policies",
      "url": "/posts/2026/03/dreamzero-paper-notes/",
      "date": "2026-03-21",
      "tags": ["Robotics","World Models","Video Diffusion","Robot Foundation Models","Cross-Embodiment","Zero-shot Generalization","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  DreamZero is a 14B World Action Model (WAM) built on a pre-trained video diffusion backbone (Wan2.1-I2V-14B). It jointly predicts future video frames and robot actions from image observations and language instructions. Key results:      2x+ improvement in zero-shot generalization to unseen tasks/environments over SOTA VLAs (GR00T N1.6, pi0.5)    Learns effectively from diverse, non-repetitive teleoperation data (~500 hrs on AgiBot G1)    Cross-embodiment transfer: 42%+ relative improvement on unseen tasks from just 10-20 min of video-only demos from other robots or humans    Few-shot embodiment adaptation: transfers to a new robot with only 30 minutes of play data    Real-time control at 7Hz via a 38x inference speedup through system, implementation, and model-level optimizations    Paper Info      Title: World Action Models are Zero-shot Policies    Authors: Seonghyeon Ye, Yunhao Ge, Kaiyuan Zheng, Shenyuan Gao, Sihyun Yu, George Kurian, et al.    Affiliation: NVIDIA    Date: 2026-02-19    arXiv: 2602.15922    Project page: dreamzero0.github.io    1. Motivation  Vision-Language-Action models (VLAs) inherit semantic priors from VLMs — they know what to do — but lack spatiotemporal priors for how to execute novel physical motions. A VLA can do “move coke can to Taylor Swift” (if it knows the move skill) but fails at “untie the shoelace” if that skill was never in the training data. VLAs are also pre-trained on static image-text data, which limits their understanding of physical dynamics.  The core hypothesis: video diffusion models encode rich spatiotemporal priors about how the physical world evolves. By building a robot policy on top of a video model — jointly predicting video and actions — we can achieve much better generalization to novel tasks and environments.  2. What is a World Action Model (WAM)?  A WAM has the input-output structure:      [Current Image + Language Instruction] → [Future Video + Robot Actions]    Unlike VLAs that directly map observations to actions, WAMs first “imagine” what successful execution looks like (video), then extract actions aligned with that imagined future. The key insight is that joint video-action prediction decomposes into:                    \\(\\pi(o_{l:l+H}, a_{l:l+H}        o_{0:l}, c, q_l) = \\underbrace{\\pi(o_{l:l+H}        o_{0:l}, c, q_l)}{\\text{video prediction}} \\cdot \\underbrace{\\pi(a{l:l+H}        o_{0:l+H}, q_l)}_{\\text{inverse dynamics}}\\)              But instead of using two separate models, DreamZero trains a single end-to-end model with a joint denoising objective.  3. Architecture  DreamZero is built on Wan2.1-I2V-14B, a 14B image-to-video diffusion transformer:      Inputs: visual context (VAE-encoded), language instruction (text encoder), proprioceptive state (state encoder)    Backbone: autoregressive DiT with flow matching    Outputs: future video frames (VAE decoder) + continuous actions (action decoder)    Training objective: flow matching with teacher-forcing chunk-wise video denoising    Why autoregressive (not bidirectional)?                    Property        Autoregressive        Bidirectional                            Inference speed        Fast (KV caching)        Slower (fixed-length)                    Frame rate        Preserves native FPS        Requires subsampling → misalignment                    Motion smoothness        Smoother (temporal backprop)        Similar task progress but jerkier                    Error accumulation        Mitigated via ground-truth observation replacement in KV cache        N/A              Closed-loop inference  A critical advantage of WAMs in closed-loop control: after executing each action chunk, ground-truth observations replace generated frames in the KV cache. This eliminates the compounding error problem inherent to autoregressive video generation.  4. Real-time Execution (38x Speedup)  Naive inference: ~5.7s per action chunk on a single GPU. Target: <200ms for smooth 7Hz control.                    Optimization Level        Technique        Cumulative Speedup (GB200)                            System        CFG parallelism (2 GPUs)        1.8x                    System        DiT caching (reuse attention)        5.4x                    Implementation        Torch compile + CUDA graphs        10.9x                    Implementation        Kernel & scheduler optimizations        14.8x                    Implementation        NVFP4 quantization        16.6x                    Model        DreamZero-Flash (1-step denoising)        38x              DreamZero-Flash  Key insight: at inference with very few denoising steps, the video tokens remain noisy while we need clean actions. Standard training couples video and action noise levels → train-test mismatch.  Solution: decouple noise schedules. Bias video timesteps toward high-noise states via Beta(7, 1), while keeping action timesteps uniform. This trains the model to predict clean actions from noisy visual context, directly matching the single-step inference regime.  Result: 1-step inference at 150ms with only ~9% performance drop vs. 4-step baseline.  5. Key Experimental Results  Data and setup      AgiBot G1: ~500 hrs diverse teleoperation across 22 environments, ~7.2K episodes, ~42 subtasks/episode    DROID-Franka: public heterogeneous dataset    Baselines: GR00T N1.6, pi0.5 (both from-scratch and pretrained variants)    Q1: Learning from diverse data (seen tasks, unseen environments)                    Model        Initialization        AgiBot Avg Task Progress                            GR00T N1.6        From scratch        0.6%                    GR00T N1.6        Pretrained        8.4%                    pi0.5        From scratch        0%                    pi0.5        Pretrained        27.4%                    DreamZero        From scratch        62.2%              VLAs trained from scratch achieve near-zero on diverse data. DreamZero achieves 2x+ over the best pretrained VLA.  Q2: Zero-shot generalization to unseen tasks  On 10 tasks entirely absent from training (untying shoelaces, ironing, painting, shaking hands, etc.):      DreamZero: 39.5% average task progress    Best pretrained VLA: 16.3%    From-scratch VLAs: <1%    Qualitatively: VLAs overfit to dominant training behaviors (e.g., always try pick-and-place). DreamZero performs visual planning and executes novel motions.  Q3: Post-training  After fine-tuning on task-specific data (shirt folding, fruit packing, table bussing):      DreamZero: 90.5% average task progress    Best pretrained VLA: 53.3%    Environment generalization is retained after post-training.    Q4: Cross-embodiment transfer  Using only 10-20 min of video-only data (no actions) from another robot (YAM) or humans:                    Method        Unseen Task Progress                            DreamZero (baseline)        38.3%                    + Human-to-robot transfer        54.3% (+42% relative)                    + Robot-to-robot transfer        55.4% (+45% relative)              Q5: Few-shot embodiment adaptation  Transfer to a new robot (YAM) with only 30 minutes of play data: retains strong language following and zero-shot generalization to novel objects.  6. Ablations                    Factor        Finding                            Data diversity        Diverse data » repetitive data (50% vs 33%) — robust IDM needs diverse state-action correspondences                    Model scale        14B » 5B (50% vs 21%) — smaller models hallucinate visually → erroneous actions. VLAs at any size (5B-32B) still fail on diverse data (0%)                    Architecture        AR ≈ BD in task progress, but AR produces smoother motions and is 3-4x faster              7. Strengths      Strong generalization story: 2x+ over SOTA VLAs on unseen tasks/environments, even without cross-embodiment pretraining    Data-efficient cross-embodiment: 10-20 min of video-only data yields 42%+ relative improvement    Practical real-time system: 38x speedup to reach 7Hz closed-loop control    Interesting failure mode: most failures come from video prediction errors, not action extraction — improving the video backbone directly improves the policy    Clean scaling signal: larger video models → better video quality → better actions (unlike VLAs where scaling doesn’t help with diverse data)    8. Limitations      Behavior cloning paradigm: WAMs are still fundamentally instruction → action, without the ability to do RL or counterfactual reasoning (as noted in the Tale of Two World Models discussion)    High-precision tasks: sub-centimeter tasks (key insertion, fine assembly) remain challenging    Long-horizon tasks: limited by action chunk horizon and context window    Single-embodiment pretraining: multi-embodiment joint pretraining not yet explored    No scaling laws: the relationship between model size, data size, and compute for WAMs is sti..."
    },{
      "title": "DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos",
      "url": "/posts/2026/03/dreamdojo-paper-notes/",
      "date": "2026-03-21",
      "tags": ["Robotics","World Models","Video Diffusion","Human Videos","Latent Actions","Cross-Embodiment","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  DreamDojo is a foundation action-conditioned world model (AC-WM) that learns diverse interaction physics from 44k hours of egocentric human videos — the largest video dataset to date for world model pretraining. To overcome the scarcity of action labels in human videos, it introduces continuous latent actions as unified proxy actions extracted via a self-supervised VAE. After post-training on small-scale robot data, DreamDojo demonstrates:      Strong OOD generalization to unseen objects, skills, and environments    Real-time inference at 10.81 FPS via an autoregressive distillation pipeline    Downstream applications: policy evaluation (Pearson r=0.995 with real-world), model-based planning (2x success rate improvement), and live teleoperation    This is an AC-WM (actions-in) — the counterpart to WAMs like DreamZero. See also A Tale of Two World Models for the WAM vs. AC-WM debate.  Paper Info      Title: DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos    Authors: Shenyuan Gao, William Liang, Kaiyuan Zheng, Ayaan Malik, Seonghyeon Ye, et al.    Affiliation: NVIDIA, HKUST, UC Berkeley, UW, Stanford, KAIST, UofT, UCSD, UT Austin    Date: 2026-02-06    arXiv: 2602.06949    Project page: dreamdojo-world.github.io    1. Motivation  Existing robot world models are trained on limited robot data and confined to in-distribution settings. The key bottleneck:      Robot data is scarce and expensive — hardware variability, teleoperation cost, mostly expert demonstrations    Real-world diversity is nearly infinite — objects, scenes, skills far exceed any robot dataset    Expert-only data lacks stochasticity — models don’t learn to respond to counterfactual actions    The insight: human videos capture the same underlying physics as robot interactions, despite the embodiment gap. And human videos are available at massive scale.  2. DreamDojo-HV Dataset  The paper curates the largest egocentric human video dataset for world model pretraining:                    Dataset        Type        Hours        Trajectories        Skills        Scenes                            DROID        Robot        350        76k        86        564                    AgiBot-World        Robot        2.9k        1,000k        87        106                    In-lab        Human        55        13.9k        35        1                    EgoDex        Human        829        30k        194        5                    DreamDojo-HV        Human        43,827        1,135k        6,015        1,135k                    Total mixture        Human        44,711        1,179k        >6,015        >1,135k              Compared to the largest prior robot datasets: 15x longer duration, 96x more skills, 2,000x more scenes.  DreamDojo-HV covers home, retail, transport, food, repair, and many other daily scenarios, collected via crowdsourcing with text annotations for each episode.  3. Approach  3.1 Latent Actions as Proxy Actions  The central technical challenge: human videos don’t have fine-grained action labels. Three options considered:                    Method        Pros        Cons                            Action-free pretraining        Simple        Ignores causality → poor controllability                    Hand pose extraction (HaMeR/MANO)        Precise for hands        Can’t capture arm/locomotion; fails under occlusion                    Latent actions (proposed)        Self-supervised, cross-embodiment, captures all motions        Proxy, not ground truth              The latent action model is a 700M spatiotemporal Transformer VAE:      Encoder: takes two consecutive frames \\(f_t, f_{t+1}\\), extracts a compact latent vector \\(\\hat{a}_t\\) (dim=32) representing the action between frames    Decoder: reconstructs \\(f_{t+1}\\) from \\(\\hat{a}_t\\) and \\(f_t\\)    Information bottleneck: forces the model to disentangle the most critical motion information                      \\(\\mathcal{L}_{\\theta,\\varphi}(f_{t+1}) = \\mathbb{E}_{q_\\varphi(\\hat{a}        f_{t:t+1})} \\log p_\\theta(f_{t+1}        \\hat{a}, f_t) - \\beta D_{KL}(q_\\varphi(\\hat{a}        f_{t:t+1}) | p(\\hat{a}))\\)              Key finding: the learned latent actions transfer across embodiments — frames with similar latent actions show the same motion regardless of whether performed by a human or robot (see Fig. 3 in the paper).  3.2 World Model Architecture  Built on Cosmos-Predict2.5 (latent video diffusion model with DiT blocks):      Action injection: actions are chunked to match the temporal compression ratio of the video tokenizer (4 frames per latent). Each chunk of 4 consecutive actions conditions the corresponding latent frame.    Relative actions: transform absolute actions to relative for better generalization.    Causal chunked injection: future actions don’t condition current predictions — respects causality.    3.3 Training Objective  Standard flow matching loss + a temporal consistency loss:  \\(\\mathcal{L}_{\\text{temporal}}(\\theta) = \\mathbb{E}\\left[\\sum_{i=1}^{K-1} |(z_{i+1} - z_i) - (v_{i+1} - v_i)|^2\\right]\\)  This supervises the transitions between frames, not just individual frames — directly encourages learning object dynamics and action following. Found to accelerate action controllability learning and improve object completeness.  3.4 Three-Phase Training      Pretraining on human videos (In-lab + EgoDex + DreamDojo-HV) with latent action conditioning    Post-training on target robot data — reset action conditioning layer, learn new action space    Distillation — convert to autoregressive, few-step model for real-time inference    3.5 Distillation Pipeline  Based on Self Forcing (Huang et al., 2025):      Warmup: regress student predictions to teacher’s ODE solutions (teacher forcing)    Distillation: student generates from its own previous outputs, supervised by KL divergence between teacher and student distributions — minimizes train-test mismatch    Key innovation: student generates \\(N’ > N\\) frames (longer than teacher horizon) during training to simulate long rollouts and reduce compounding error.  Result: 35 denoising steps → 4 steps, bidirectional → causal attention, enabling 10.81 FPS real-time inference.  4. Key Results  Scaling data improves everything  Adding more human data consistently improves OOD performance:                    Pretraining Data        In-lab PSNR        Counterfactual PSNR                            No pretraining        20.576        20.472                    In-lab only        20.913        20.755                    In-lab + EgoDex        20.972        20.797                    In-lab + EgoDex + DreamDojo-HV        21.016        20.852              Latent actions match ground-truth actions                    Conditioning Method        In-lab PSNR        EgoDex PSNR                            No pretraining        20.576        19.952                    Action-free pretraining        20.797        19.924                    Latent action        20.913        20.344                    Ground-truth action (ideal)        20.960        20.474              Latent actions close most of the gap to ground-truth labels — and are infinitely more scalable.  Human preference: scaling model helps                    Comparison        Physics Correctness        Action Following                            DreamDojo-2B > Cosmos-Predict2.5        62.5%        63.5%                    DreamDojo-14B > Cosmos-Predict2.5        73.5%        72.6%                    DreamDojo-14B > DreamDojo-2B        72.5%        65.5%              Distillation: real-time with minimal degradation                    Model        FPS        Predict Len        Context Len                            Teacher        2.72        12 frames        1 frame                    Student (distilled)        10.81        4 frames        12 frames              The student is 4x faster and has better context awareness (12-frame sliding window vs. 1-frame conditioning).  Downstream applications  Policy evaluation: Pearson correlation r=0.995 between DreamDojo-predicted success rates and real-world success rates across 6 policy checkpoints. Near-perfect ranking.  Model-based planning: Sample N action proposals from a policy ensemble, simulate all with DreamDojo, select best via a value model. Result: ~2x improvement in success rate over uniform sampling.  Live teleoperation: Real-time teleoperation of a virtual G1 robot using PICO VR controller on a single RTX 5090.  Architecture ablations                    Modification        GR-1 Val PSNR        Counterfactual PSNR                            Baseline        16.199        19.448                    + Relative actions        16.522        19.482                    + Chunked injection        17.626        20.783                    + Temporal consistency loss        17.630        20.980              Chunked injection is th..."
    },{
      "title": "CUDA Agent: Large-Scale Agentic RL for High-Performance CUDA Kernel Generation",
      "url": "/posts/2026/03/cuda-agent-paper-notes/",
      "date": "2026-03-21",
      "tags": ["LLM","Reinforcement Learning","CUDA","Code Generation","Agentic AI","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  LLMs are decent at general coding but struggle to beat torch.compile at writing optimized CUDA kernels. CUDA Agent fixes this with large-scale agentic reinforcement learning: a scalable data pipeline (6K synthesized operator tasks), a skill-augmented CUDA development environment with anti-hacking safeguards, and multi-stage warm-up techniques (RFT + Value Pretraining) that stabilize 150-step RL training. Result: 100%, 100%, 92% faster rate over torch.compile on KernelBench Level-1/2/3 – blowing past Claude Opus 4.5 and Gemini 3 Pro by ~40% on the hardest split.  Paper Info      Title: CUDA Agent: Large-Scale Agentic RL for High-Performance CUDA Kernel Generation    Authors: Weinan Dai*, Hanlin Wu*, Qiying Yu, Huan-ang Gao, Jiahao Li, Chengquan Jiang, Weiqiang Lou, Yufan Song, Hongli Yu, Jiaze Chen, Wei-Ying Ma, Ya-Qin Zhang, Jingjing Liu, Mingxuan Wang, Xin Liu, Hao Zhou    Affiliations: ByteDance Seed, Institute for AI Industry Research (AIR) Tsinghua University, SIA-Lab    arXiv: 2602.24286    Date: March 2, 2026    Paper type: LLM agent training / CUDA kernel optimization / reinforcement learning    1. Problem and Motivation  GPU kernel optimization is critical for deep learning performance but requires deep hardware expertise. Despite LLMs excelling at general programming, they remain uncompetitive with compiler-based systems like torch.compile for CUDA kernel generation.  Existing approaches fall into two camps, both with fundamental limits:      Training-free refinement (STARK, ReGraphT, EvoEngineer): hand-designed heuristics with execution feedback. Performance is capped by the base model’s intrinsic CUDA ability.    Fine-tuning within fixed loops (Kevin, CUDA-L1, ConCuR): multi-turn execution-feedback pipelines. These waste context on all previous solutions and constrain the agent’s autonomy to learn its own debugging/profiling strategies.    Neither paradigm fundamentally improves the model’s intrinsic CUDA optimization capability.  2. Method  CUDA Agent has three pillars: data, environment, and RL techniques.  2.1 Scalable Data Synthesis Pipeline  High-quality CUDA training data is scarce. The pipeline:      Seed Problem Crawling: mine reference operators from torch and transformers libraries    Combinatorial Synthesis: LLMs sample up to 5 operator classes and compose them into fused tasks – because fused problems require joint optimization (shared registers/SMEM/occupancy), not just chaining individually-optimized ops    Rubric-based Filtering: keep only executable, deterministic, non-trivial problems with 1-100ms eager runtime; exclude KernelBench-similar cases    This yields CUDA-Agent-Ops-6K, a curated operator-level dataset.  2.2 Skill-Integrated Agent Loop  The agent loop follows a ReAct-style paradigm (reason → act → observe) compatible with OpenHands, using standard tools (BashTool, GlobTool, MultiEditTool, TodoWriteTool). On top of this, CUDA Agent gets a SKILL.md that formalizes the CUDA optimization workflow:      Profile the native PyTorch implementation to find bottlenecks    Implement custom CUDA operators targeting identified hotspots    Compile and evaluate in a GPU sandbox, iterate until correct    Repeat until ≥5% speedup over torch.compile    Anti-reward-hacking measures (this is a highlight):      Evaluation scripts are file-permission protected – the agent can’t modify them    Context managers forbid fallback to torch.nn.functional    5 random inputs for correctness validation    Proper GPU synchronization, warm-up iterations, and averaged measurements    No web search tools – all solutions from local environment only    2.3 Robust Reward Scheduling  Instead of raw speedup (noisy, biased toward easy kernels), they use a discrete milestone-based reward:  \\(r = \\begin{cases} -1 & \\text{if correctness check fails} \\ 3 & \\text{if faster than both eager and compile} \\ 2 & \\text{if faster than eager only} \\ 1 & \\text{otherwise (correct but not faster)} \\end{cases}\\)  where “faster” means >5% speedup. This normalized reward avoids outlier-driven optimization.  2.4 Multi-Stage Warm-up for Stable Training  The core training instability comes from domain distribution mismatch – CUDA code is <0.01% of pretraining data, causing low-probability tokens and exploding importance sampling ratios.  The fix is a multi-stage warm-up:      Single-Turn RL: standard PPO on the base model (Seed1.6, 23B active / 230B total MoE) for basic CUDA generation ability    Actor Initialization (RFT): collect agent trajectories from the single-turn model, reject-sample for high-quality ones (positive reward, no hallucinated tool calls), fine-tune    Critic Initialization (Value Pretraining): pretrain the critic on trajectory data with GAE targets so it can immediately provide useful advantage estimates    Agentic RL: full PPO with 128K context, up to 150 agent turns during training (200 at eval)    Without RFT → entropy explodes → policy collapses within ~17 steps.Without Value Pretraining → critic can’t estimate values → trajectory lengths explode.  3. Experiments and Main Results  Benchmark: KernelBench (250 operator tasks across Level 1-3).  Setup: Seed1.6 base model, batch size 1024, 128 H20 GPUs for profiling, Docker-based sandboxes.  Key Results (vs. torch.compile)                    Model        L1 Faster Rate        L2 Faster Rate        L3 Faster Rate        L3 Speedup                            GLM 4.6        32%        11%        10%        0.62x                    Kimi K2        39%        15%        6%        0.29x                    Gemini 3 Pro        72%        76%        52%        1.17x                    Claude Opus 4.5        72%        69%        50%        1.10x                    CUDA Agent        97%        100%        90%        1.52x              Three takeaways:      CUDA Agent massively outperforms proprietary models, especially on harder tasks (~40% gap on L3)    Level 2 (operator sequences / fusion) shows the biggest win: 100% faster rate, 2.80x speedup. Compiler heuristics can’t handle non-trivial fusion patterns; the agent explores a much larger design space    Learned optimization policies can consistently beat static compiler heuristics    Ablation Highlights                    Variant        Overall Faster Rate (vs. Compile)        Overall Speedup (vs. Compile)                            w/o Agent Loop        14.1%        0.69x                    w/o Robust Reward        60.4%        1.25x                    w/o RFT        49.8%        1.05x                    w/o Value Pretraining        50.9%        1.00x                    Full CUDA Agent        96.8%        2.11x              Every component matters, but the agent loop is the biggest contributor – without it, the model can barely beat torch.compile at all.  4. Strengths and Limitations  Strengths:      Comprehensive anti-reward-hacking design (file permissions, fallback prevention, measurement rigor)    The multi-stage warm-up is well-motivated and cleanly ablated – each stage addresses a specific instability mode    The combinatorial data synthesis is clever: fused operator tasks create genuinely novel optimization challenges    SOTA results with large margins, especially on the harder splits    Limitations:      Built on Seed1.6 (230B MoE) – unclear how much transfers to smaller models    KernelBench is the only benchmark; real-world kernel optimization has additional complexities (library integration, memory management across kernel boundaries)    128 H20 GPUs for the sandbox pool is a significant resource requirement    The paper notes ChatGPT-5 series models “declined to respond to CUDA-related prompts” – an interesting but unelaborated observation    5. Takeaways      Agentic RL > fixed pipelines for code optimization tasks. Letting the model learn its own debugging and profiling strategies (via agent loop) is far more effective than constraining it to pre-designed multi-turn templates.    Domain-specific warm-up is critical when the target domain is a tiny fraction of pretraining data. The RFT → Value Pretraining → PPO pipeline is a reusable recipe for other low-resource agentic RL domains.    Discrete milestone rewards > continuous speedup rewards for optimization tasks with noisy measurements. The robust reward design avoids outlier-driven training while still incentivizing genuine performance gains.    LLM-based kernel generation is now competitive with (and often superior to) compiler-driven optimization. This is a significant milestone – it suggests a path where AI agents handle performance-critical low-level optimization that currently requires deep hardware expertise.    概要  大语言模型在通用编程方面表现出色，但在编写优化的 CUDA 内核时仍然难以超越 torch.compile。CUDA Agent 通过大规模 智能体强化学习 解决了这一问题：可扩展的数据合成管线（6K 合成算子任务）、配备防作弊机制的技能增强 CUDA 开发环境，以及多阶段预热技术（RFT + Value Pretraining）稳定 150 步 RL 训练。最终结果：在 KernelBench Level-1/2/3 上相比 torch.compi..."
    },{
      "title": "CFGRL: Diffusion Guidance Is a Controllable Policy Improvement Operator",
      "url": "/posts/2026/03/cfgrl-paper-notes/",
      "date": "2026-03-21",
      "tags": ["Reinforcement Learning","Offline RL","Diffusion Models","Policy Improvement","Goal-Conditioned RL","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  CFGRL derives a direct, principled connection between classifier-free guidance (CFG) in diffusion models and policy improvement in RL. The key insight: a policy can be factored as prior x optimality, and the guidance weight w in CFG directly controls the degree of policy improvement — provably. This means:      Train with the simplicity of supervised learning (conditional diffusion/flow matching)    Get policy improvement for free at test time by tuning the guidance weight w — no retraining needed    Works as a drop-in replacement for advantage-weighted regression (AWR) in offline RL    Especially powerful for goal-conditioned BC (GCBC): standard GCBC is just CFGRL with w=1; setting w>1 provably improves the policy, often doubling success rates — without learning any value function    Paper Info      Title: Diffusion Guidance Is a Controllable Policy Improvement Operator    Authors: Kevin Frans, Seohong Park, Pieter Abbeel, Sergey Levine    Affiliation: UC Berkeley    Date: 2025-05-29    Venue: arXiv preprint, under review    arXiv: 2505.23458    Code: github.com/kvfrans/cfgrl    1. Motivation  Two ends of the spectrum for learning from offline data:                    Approach        Training        Optimality        Scalability                            Behavioral cloning        Simple (supervised)        Only as good as data        Excellent                    Offline RL        Complex (value functions, policy gradients)        Can improve beyond data        Notoriously tricky              Can we get policy improvement while keeping the simplicity of supervised learning?  2. Core Idea: Product Policies  Define an improved policy as a product of two factors:                    \\(\\pi(a        s) \\propto \\hat{\\pi}(a        s) \\cdot f(A^{\\hat{\\pi}}(s,a))\\)              where \\(\\hat{\\pi}\\) is the reference (behavior) policy and \\(f\\) is a non-negative, monotonically increasing function of the advantage \\(A^{\\hat{\\pi}}(s,a)\\).  Theorem 1 (Policy Improvement): If \\(f\\) is non-negative and non-decreasing in advantage, then the product policy \\(\\pi\\) is guaranteed to improve over \\(\\hat{\\pi}\\):  \\(J(\\pi) \\geq J(\\hat{\\pi})\\)                    Theorem 2 (Controllable Improvement): For \\(0 \\leq w_1 < w_2\\), the attenuated product \\(\\pi_{w_2}(a        s) \\propto \\hat{\\pi}(a        s) f(A(s,a))^{w_2}\\) is a further improvement over \\(\\pi_{w_1}\\):              \\(J(\\pi_{w_1}) \\leq J(\\pi_{w_2})\\)  Higher w = more improvement (but also more divergence from reference → eventual distribution shift).  3. Connection to Diffusion Guidance  The product policy’s score function decomposes additively:                    \\(\\nabla_a \\log \\pi(a        s) = \\nabla_a \\log \\hat{\\pi}(a        s) + \\nabla_a \\log p(o        s,a)\\)              Using classifier-free guidance (Bayes’ rule trick), this becomes:                    \\(\\nabla_a \\log \\hat{\\pi}(a        s) + w \\cdot (\\nabla_a \\log \\hat{\\pi}(a        s, o) - \\nabla_a \\log \\hat{\\pi}(a        s))\\)              This is exactly the standard CFG formula! The guidance weight w directly controls the degree of policy improvement. Both the unconditional and conditional scores come from the same network trained with a simple flow matching objective:  \\(\\mathcal{L}(\\theta) = \\mathbb{E}_{s,a \\sim \\mathcal{D}} |v_\\theta(a_t, t, s, o) - (a - a_0)|^2\\)  where \\(o \\in {\\emptyset, 0, 1}\\) is the optimality label (with 10% dropout for unconditional training).  Why this matters vs. AWR                    Property        AWR        CFGRL                            Temperature/weight        \\(1/\\beta\\) (fixed at train time)        \\(w\\) (tunable at test time)                    Gradient distribution        Dominated by outlier high-advantage samples        Even weighting across batch                    Retraining needed to tune?        Yes        No                    Empirical scaling        Saturates around \\(1/\\beta = 10\\)        Continues improving beyond              4. Special Case: Goal-Conditioned BC (GCBC)                    This is where CFGRL truly shines. Standard GCBC trains a goal-conditioned policy \\(\\pi(a        s,g)\\). The paper makes the connection more explicit than most prior GCBC writeups:                                \\(\\pi(a        s, g) = \\frac{\\hat{\\pi}(a        s)\\, p_\\gamma(g        s,a)}{p_\\gamma(g        s)} \\propto \\hat{\\pi}(a        s) \\cdot Q^{\\hat{\\pi}}(s, a, g)\\)              The second factor satisfies the conditions of Theorem 1. Therefore:      Standard GCBC = CFGRL with w=1 (implicit, no improvement)    CFGRL with w>1 = provably improved GCBC — for free!    The sampling formula is simply:                    \\(\\nabla_a \\log \\hat{\\pi}(a        s) + w \\cdot (\\nabla_a \\log \\pi(a        s, g) - \\nabla_a \\log \\hat{\\pi}(a        s))\\)              No value function needed. Just train a goal-conditioned flow policy and an unconditional flow policy (same network with dropout), then tune w at test time.  5. Key Results  Offline RL (ExORL benchmark, with learned value function)  CFGRL consistently outperforms AWR on most tasks:                    Task        AWR        CFGRL                            walker-stand        603        782                    walker-walk        444        608                    walker-run        247        282                    quadruped-run        485        571                    cheetah-run        168        216                    cheetah-run-backward        146        262                    jaco-reach-top-right        33        72              Goal-Conditioned BC (OGBench, no value function)  CFGRL as drop-in GCBC improvement (selected results, flat policies):                    Task        Flow GCBC        CFGRL        Improvement                            pointmaze-large-navigate        74        77        +4%                    pointmaze-giant-navigate        4        30        7.5x                    antmaze-medium-navigate        42        53        +26%                    humanoidmaze-medium-navigate        8        19        2.4x                    visual-cube-single-play        13        37        2.8x                    visual-scene-play        25        40        +60%              One detail I found especially convincing in the PDF is that these gains are not coming from heavy per-task retuning. For the main GCBC table, the authors use a single fixed guidance strength of w = 3 and still get broad improvements across state-based and pixel-based tasks.  With hierarchical policies (HCFGRL), gains are even larger:                    Task        Flow HGCBC        HCFGRL                            antmaze-medium-navigate        67        90                    antmaze-large-navigate        61        78                    antmaze-giant-navigate        14        38                    humanoidmaze-large-navigate        11        38                    cube-double-play        21        42              Scaling behavior  The guidance weight w provides a reliable knob:      Performance steadily increases with w up to a divergence point    The divergence point is further out than AWR’s temperature saturation    w can be swept without retraining — just re-run inference    6. Strengths      Elegant theory: clean, provable connection between CFG and policy improvement with formal guarantees (Theorems 1 & 2)    Extreme simplicity: train conditional diffusion/flow model with supervised loss → tune w at test time → done    No value function needed for GCBC setting — improvement literally comes for free    Test-time tunable: unlike AWR where \\(\\beta\\) is baked into training, w can be swept without retraining    Fixes AWR’s gradient issue: even gradient magnitudes across batch vs. outlier-dominated in AWR    Broad applicability: state-based, visual, hierarchical, offline RL, goal-conditioned settings    7. Limitations      One-step improvement only: CFGRL provides one step of policy improvement over the reference, not iterative optimization — not a full RL algorithm    Distribution shift at high w: theoretical guarantees hold but practical performance degrades when w is too large (divergence from reference)    Assumes a separate value-learning story in offline RL: in the AWR-style setting, CFGRL replaces policy extraction, not the upstream Q / V training pipeline    Requires offline data quality: still fundamentally limited by the support of the dataset — can improve suboptimal data but can’t discover entirely new behaviors    Not SOTA offline RL: the authors explicitly note CFGRL is a tool (replacing AWR), not a complete offline RL system — advanced methods like policy gradients could extrapolate further    8. Takeaways      CFG = policy improvement: diffusion guidance weight directly and provably controls the degree of policy improvement — one of the cleanest theory-to-practice connections in recent RL    GCBC users: use guidance! Standard GCBC is CFGRL ..."
    },{
      "title": "ACG: Action Coherence Guidance for Flow-based VLA Models",
      "url": "/posts/2026/03/acg-paper-notes/",
      "date": "2026-03-21",
      "tags": ["Robotics","VLA","Flow Matching","Diffusion Policy","Test-time Guidance","Action Coherence","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  Action Coherence Guidance (ACG) is a training-free, test-time guidance technique for flow-matching-based VLA models (GR00T-N1, pi0, SmolVLA, etc.). The problem: diffusion/flow policies memorize noise in human demos (jerks, pauses, jitter), producing temporally incoherent action sequences that cause failures in fine-grained manipulation. ACG’s solution: construct an incoherent denoising vector by replacing self-attention maps with identity matrices (forcing each action token to attend only to itself), then guide away from this incoherent direction during sampling. Results: +6.7 pp average success rate across RoboCasa, DexMimicGen, and real-world SO-101 tasks, with especially large gains on fine manipulation (+23.1 pp button pressing, +28.8 pp real-world pick-and-place). No retraining required.  Paper Info      Title: ACG: Action Coherence Guidance for Flow-based VLA models    Authors: Minho Park, Kinam Kim, Junha Hyung, Hyojin Jang, Hoiyeong Jin, Jooyeol Yun, Hojoon Lee, Jaegul Choo    Affiliation: KAIST AI    arXiv: 2510.22201    1. Problem: Action Incoherence in Flow-based VLAs  Flow matching policies trained via imitation learning have high generative capacity — but this capacity also memorizes imperfections in human demos:      Jerks during teleoperation    Pauses and hesitations    Jitter from hand tremor    This degrades action coherence: the smoothness and consistency of successive actions within an action chunk. During deployment, incoherent actions cause:      Instability at critical moments — fumbling near objects, pushing them away    Trajectory drift — small noise accumulates, deviating from the desired path    This is especially catastrophic for fine-grained manipulation (button pressing, insertion, precise grasping).  2. Method: Action Coherence Guidance  2.1 CFG Recap (and why it doesn’t work well for VLAs)  Standard CFG for flow policies:  \\(v_\\theta^{\\text{CFG}(\\lambda)} = (1+\\lambda) \\, v_\\theta(\\mathbf{A}t^\\tau, \\mathbf{o}_t, \\ell_t, \\tau) - \\lambda \\, v\\theta(\\mathbf{A}_t^\\tau, \\mathbf{o}_t, \\varnothing, \\tau)\\)  Problem: in VLAs, removing the language condition \\(\\ell_t\\) changes the action distribution dramatically — the “unconditional” direction is not meaningful and causes unstable behavior.  2.2 ACG: Guide Away from Incoherence  Instead of guiding toward a condition, ACG guides away from incoherence:  \\(v_\\theta^{\\text{ACG}(\\lambda)} = (1+\\lambda) \\, v_\\theta(\\mathbf{A}t^\\tau, \\mathbf{o}_t, \\ell_t, \\tau) - \\lambda \\, v\\theta^{\\text{IC}}(\\mathbf{A}_t^\\tau, \\mathbf{o}_t, \\ell_t, \\tau)\\)  where \\(v_\\theta^{\\text{IC}}\\) is the incoherent denoising vector — same model, same inputs, but with modified self-attention.  2.3 Constructing the Incoherent Vector  The key insight: in transformer-based flow policies, self-attention is what creates temporal coherence between action tokens. Each token (representing an action at a specific timestep) attends to all other tokens:  \\(\\text{Attn}(Q, K, V) = \\text{softmax}\\left(\\frac{QK^\\top}{\\sqrt{d}}\\right) V\\)  To generate an incoherent action sequence, replace the attention map with an identity matrix:  \\(\\text{Attn}^{\\text{IC}}(Q, K, V) = I \\cdot V = V\\)  This forces each action token to attend only to itself — no temporal communication — producing a temporally disconnected action chunk. Then ACG steers the generation away from this incoherent direction.  Implementation details:      Replace self-attention in layers 4-6 (out of 8 total) with identity attention    Share the first half of layers between base and incoherent passes → ~1.5x compute overhead    Guidance scale \\(\\lambda = 3.0\\) (default)    3. Key Results  Main comparison (GR00T-N1 backbone)                    Method        RoboCasa        DexMG        Real: Strawberries        Real: Tic-Tac-Toe        Average                            Vanilla GR00T-N1        32.6%        40.6%        43.6%        38.3%        38.8%                    Ensemble (n=2)        34.0%        40.3%        56.7%        45.0%        44.0%                    Feature Smoothing        34.4%        42.4%        57.8%        45.0%        44.9%                    CFG        35.0%        41.5%        50.0%        43.3%        42.5%                    WNG        35.0%        42.0%        65.6%        48.3%        47.7%                    ACG (Ours)        39.3%        44.0%        74.4%        56.7%        53.6%              ACG outperforms all baselines by a clear margin, especially on real-world tasks.  Fine-grained tasks see the largest gains                    Skill        Vanilla        ACG        Improvement                            Button pressing        —        —        +23.1 pp                    Insertion        —        —        +11.8 pp                    Real-world pick-and-place        —        —        +28.8 pp              Action coherence metrics                    Method        ATV (rad/s, ↓)        Jerk_RMS (×10³ rad/s³, ↓)                            Vanilla GR00T-N1        1.314        1.353                    Ensemble (n=5)        0.984        1.172                    CFG        1.332        1.317                    Incoherent (\\(v_\\theta^{\\text{IC}}\\))        4.509        1.993                    ACG        1.130        1.156              The incoherent variant is indeed much worse than baseline (validating the design), and ACG achieves the best smoothness while maintaining accuracy (unlike ensemble which smooths but loses precision).  Ablation highlights      Guidance scale: performance improves up to ~3.0, then degrades (divergence from pretrained distribution)    Number of incoherent layers: 2-6 layers all help; robust to this choice    Layer position: middle/later layers work best; early layers can hurt    Complementary with Self-GAD: ACG improves intra-chunk coherence, Self-GAD improves inter-chunk coherence; combining both yields further gains    4. Connection to CFGRL  This paper has an interesting conceptual link to CFGRL. Both use guidance at test time to improve flow policy outputs — but for different purposes:                             CFGRL        ACG                            Guides toward        Optimality (higher advantage)        Temporal coherence                    Guides away from        Unconditional (no goal)        Incoherent (no self-attention)                    Requires        Optimality labels        Nothing (architectural perturbation)                    Improves        Return / success rate        Action smoothness → success rate              Both demonstrate that test-time guidance is a powerful, underexplored tool for robot policy improvement.  5. Strengths      Training-free: zero additional training, works on any flow-based VLA    Principled design: identity attention → incoherence is clean and well-motivated    Large real-world gains: +28.8 pp on real SO-101 tasks — not just sim improvements    Fine-grained manipulation: biggest gains exactly where they matter most    Complementary: works alongside other guidance methods (Self-GAD)    6. Limitations      ~1.5x compute overhead: requires an extra (partial) forward pass for the incoherent vector    Hyperparameter sensitivity: guidance scale too high → performance degrades    Shallow network assumption: replacing layers 4-6 of 8 works; unclear if the same recipe scales to much deeper networks    Only tested on GR00T-N1: applicability to other VLA architectures (pi0, SmolVLA) not empirically verified    7. Takeaways      Action coherence is a real bottleneck in flow-based VLA policies — demo noise gets memorized and causes deployment failures, especially for fine manipulation    Self-attention = temporal coherence: replacing attention maps with identity is a clean way to construct a “negative example” for guidance    Test-time guidance is broadly useful for robot policies — not just for image/video generation. Both ACG (coherence) and CFGRL (optimality) show this    Intra-chunk coherence matters more than inter-chunk: ACG outperforms Self-GAD, and combining both helps further    Practical recipe: for anyone deploying GR00T-N1 or similar flow-based VLAs, ACG is essentially free performance — no retraining, just modify inference    References      [Paper] arXiv:2510.22201    本文支持通过顶部导航栏的语言切换按钮在 English / 中文 之间切换。  概要  动作一致性引导（ACG）是一种适用于基于 flow matching 的 VLA 模型（GR00T-N1、pi0、SmolVLA 等）的免训练、测试时引导技术。问题：扩散/flow 策略会记忆人类演示中的噪声（抖动、停顿、颤抖），产生时间上不连贯的动作序列，导致精细操作失败。ACG 的解决方案：通过将自注意力图替换为单位矩阵（迫使每个动作 token 只关注自身）来构造不连贯的去噪向量，然后在采样过程中远离这个不连贯方向。结果：在 RoboCasa、DexMimicGen 和真实世界 SO-101 任务上平均成功率提升 +6.7 pp，精细操作增益尤其显著（按钮按压 +23.1 pp，真实世界抓取 +28.8 pp）。无需重新训练。  论文信息      标题: ACG: Action Coherence Guidance for Flow-based VLA models    作者: Minho Park, Kinam Kim, Junha Hyung, Hyojin Jang, Hoiyeong Jin, Jooyeol Yun, Hojoon Lee, Jaegul Choo    机构: KAIST AI    arXiv: 2510.22201    1. 问题：Flow-based VLA 中的动作不连贯  基于 flow matching 的策略通过模仿学习训练，具有很强的生成能力——但这种能力也会记忆人类演示中的不完美： ..."
    },{
      "title": "Tacmap: Bridging the Tactile Sim-to-Real Gap via Geometry-Consistent Penetration Depth Map",
      "url": "/posts/2026/03/tacmap-paper-notes/",
      "date": "2026-03-19",
      "tags": ["Robotics","Tactile Sensing","Sim-to-Real","Reinforcement Learning","Dexterous Manipulation","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  Tactile sim-to-real is hard because raw tactile images are full of sensor-specific optical artifacts (reflections, lighting, camera noise). Tacmap sidesteps this by defining a shared geometric representation – the penetration depth map (deform map) – that can be computed analytically in simulation and learned from real tactile images via a translation network. Both domains meet in this “common geometric space,” enabling zero-shot sim-to-real transfer of RL policies for contact-rich tasks like in-hand rotation. The approach is fast enough for massive parallel training in Isaac Lab.  Paper Info      Title: Tacmap: Bridging the Tactile Sim-to-Real Gap via Geometry-Consistent Penetration Depth Map    Authors: Lei Su, Zhijie Peng, Renyuan Ren, Shengping Mao, Juan Du, Kaifeng Zhang, Xuezhou Zhu    Affiliations: Sharpa, HKUST, NVIDIA    arXiv: 2602.21625    Paper type: tactile simulation / sim-to-real transfer / dexterous manipulation    1. Problem and Motivation  Vision-based tactile sensors (VBTS) like GelSight and DIGIT capture rich contact information via a camera observing elastomer deformation. Training policies that use these sensors requires millions of interactions, making simulation essential. But current tactile simulation faces a dilemma:      Analytical methods (e.g., TACTO): fast depth-buffer rendering, but oversimplifies elastomer physics – large sim-to-real gap.    Empirical methods (e.g., Taxim): supervised by real data, but poor generalization to novel geometries.    Physics-based methods (e.g., FEM): high fidelity, but computationally prohibitive for large-scale RL.    An additional blind spot: most simulators assume flat sensor surfaces. Curved fingertips (common in anthropomorphic hands) cause projection distortions that existing tools handle poorly.  2. Method  2.1 Core Insight: Deform Map as Common Geometric Space  Raw tactile images differ wildly between sim and real due to optics. But the underlying deformation geometry is the same physical quantity in both domains. Tacmap defines a unified representation: the penetration depth map (deform map) – a pixel-wise map of how deeply an object penetrates the sensor elastomer.      In simulation: compute the deform map analytically via ray-casting.    In the real world: train a neural network to translate raw tactile images into deform maps.    Both domains produce the same type of output, eliminating the need to simulate complex optical effects.  2.2 Simulation: Geometric Rendering Pipeline  The sensor geometry is defined by two surfaces:      Undeformed sensor surface \\(S_u\\): the physical resting shape of the elastomer.    Virtual sensing surface \\(S_s\\): positioned at a fixed offset exterior to the sensor, defining the interaction zone.    For each pixel \\((u,v)\\) on an \\(H \\times W\\) grid over \\(S_s\\), a ray is cast along the surface normal toward the sensor interior. The deformation value is:  \\(d(u,v) = \\max(0, z_s - \\max(z_u, z_o))\\)  where \\(z_s\\) is the ray origin on \\(S_s\\), \\(z_u\\) is the undeformed surface coordinate, and \\(z_o\\) is the first intersection with the object mesh. This naturally handles curved fingertips by computing in local normal-projection space rather than assuming a flat plane.  2.3 Real World: Automated Data Collection + Translation  An automated 3-axis motion stage presses geometric indenters into the real sensor under controlled conditions. The indenter’s precise 3D pose is recorded, and ground-truth deform maps are computed using the same geometric projection logic as simulation. This yields a paired dataset \\(\\mathcal{D} = {(I_{\\text{raw}}^{(i)}, M_{\\text{gt}}^{(i)})}\\).  A ResNet-based encoder-decoder is trained to map raw tactile images to deform maps: \\(\\hat{M} = \\Phi(I_{\\text{raw}})\\), minimizing pixel-wise MSE against the kinematically-derived ground truth.  2.4 Three Tactile Information Streams  Tacmap provides three synchronized signals:      Net Force \\(F\\): from physics engine in sim; from a trained regression network in real.    Contact Position \\(P\\): from contact sensor in sim; from centroid of deform map in real.    Deform Map \\(M\\) (the main contribution): dense, pixel-wise penetration depth.    3. Implementation  Integrated into both Isaac Lab and MuJoCo:      A Multi-Mesh Ray Caster pre-defines tactile sensing points and directions on the sensor surface.    GPU-accelerated ray-casting computes penetration depth in parallel across thousands of environments.    In Isaac Lab: uses the Raycaster API for massive parallelism. In MuJoCo: uses mj_ray function.    The tactile sensing resolution is decoupled from physics collision mesh resolution, so high-fidelity tactile feedback doesn’t compromise physics solver stability.  4. Experiments and Results  Sim-to-Real Fidelity  Tested with cylindrical and square indenters on the SharpaWave hand’s tactile fingertips:                    Object        Contact Position Error        Deform Depth Error        Net Force \\(L_2\\) Error        Deform IoU                            Square        0.66 mm        18.53%        0.28 N        88.21%                    Cylinder        0.96 mm        14.71%        0.61 N        85.67%              The simulated and real deform maps show “remarkable structural similarity” across compression sequences. Force alignment between sim and real is highly correlated.  Computational Efficiency      GPU memory: near-linear growth from 16 to 8192 parallel environments (ray-casting is much lighter than FEM).    Rendering throughput: negligible degradation of overall simulation speed even with thousands of concurrent environments.    Trainable on a single consumer-grade GPU.    Zero-Shot Sim-to-Real Transfer: In-Hand Rotation      Policy trained with PPO exclusively in simulation, using the Tacmap stream as observation.    Deployed directly on the physical SharpaWave hand without any real-world fine-tuning.    Successfully achieves smooth, continuous in-hand rotation of a spherical object.    The policy interprets real-world tactile images (translated to deform maps) and performs proactive finger coordination to prevent slips.    5. Strengths      Elegant core idea: abstract away sensor-specific optics, align sim and real in a shared geometric space. Simple and effective.    Geometry-agnostic: works for both flat and curved sensor surfaces via normal-projection space, unlike most existing simulators.    Computationally efficient: ray-casting is orders of magnitude cheaper than FEM while maintaining sufficient physical fidelity for policy transfer.    Practical validation: zero-shot transfer of a contact-rich RL policy (in-hand rotation) is a strong demonstration.    Dual-engine support: works in both Isaac Lab (for massive parallel RL) and MuJoCo.    6. Limitations      No shear/tangential force modeling: Tacmap only captures normal penetration depth. Shear strain and lateral forces (critical for slip prediction) are not represented.    Single downstream task: only in-hand rotation is demonstrated for sim-to-real transfer. More diverse tasks (assembly, insertion) would strengthen the claims.    Ray-casting scales with mesh complexity: as object meshes become more detailed, ray-casting overhead grows. Advanced acceleration structures are mentioned as future work.    Translation network generalization: the real-world image-to-deform translation network is trained on a limited set of geometric indenters. Generalization to arbitrary object shapes in the wild is not extensively tested.    7. Takeaways      The deform map is a clever abstraction layer. By standardizing both sim and real into penetration depth, you sidestep the hardest part of tactile sim-to-real (reproducing optical phenomena) and focus on what actually matters for control: contact geometry.    This is the third paper from Sharpa in my recent reading (after DexEMG and SaTA). Together they form a coherent stack: Tacmap for tactile sim-to-real, SaTA for spatially-anchored tactile policy learning, and DexEMG for lightweight teleoperation. All targeting the SharpaWave dexterous hand.    The approach is complementary to SaTA: Tacmap provides the sim-to-real bridge for training tactile policies in simulation, while SaTA provides the representation for using tactile data effectively at deployment. Combining them could enable sim-trained policies with spatially-anchored tactile reasoning.    The main open question is shear force. Normal penetration depth captures a lot, but tasks requiring slip detection or delicate force modulation need tangential information. The authors acknowledge this as a key direction.    本文支持通过顶部导航栏的语言切换按钮在 English / 中文 之间切换。  一句话总结  触觉 sim-to-real 之所以困难，是因为原始触觉图像充满了传感器特有的光学伪影（反射、照明、相机噪声）。Tacmap 通过定义一种共享几何表征 – 穿透深度图（deform map）来绕过这个问题：在仿真中通过解析方法计算，在真实世界中通过翻译网络从原始触觉图像学习。两个域在这一”公共几何空间”中对齐，实现了接触密集型任务（如手内旋转）的 零样本 sim-to-real 迁移。该方法速度足以支持 Isaac Lab 中的大规模并行训练。  论文信息      标题: Tacmap: B..."
    },{
      "title": "SaTA: Spatially-anchored Tactile Awareness for Robust Dexterous Manipulation",
      "url": "/posts/2026/03/sata-paper-notes/",
      "date": "2026-03-19",
      "tags": ["Robotics","Dexterous Manipulation","Tactile Sensing","Imitation Learning","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  Existing visuo-tactile learning methods can tell that contact happened, but struggle to reason about where and how contact relates to the hand’s geometry. SaTA fixes this by anchoring tactile features to the hand’s kinematic frame via forward kinematics + Fourier encoding + FiLM conditioning. The result: an end-to-end imitation learning policy that achieves sub-millimeter precision on tasks like bimanual USB-C mating in free space, light bulb installation, and card sliding – improving success rates by up to 30% and reducing completion times by ~27% over strong baselines.  Paper Info      Title: Spatially-anchored Tactile Awareness for Robust Dexterous Manipulation    Authors: Jialei Huang, Yang Ye, Yuanqing Gong, Xuezhou Zhu, Yang Gao, Kaifeng Zhang    Affiliations: Sharpa, Tsinghua University, Wuhan University, Shanghai Qi Zhi Institute    arXiv: 2510.14647    Paper type: tactile perception / dexterous manipulation / imitation learning    1. Problem and Motivation  Dexterous manipulation tasks like USB insertion or bulb threading demand sub-millimeter precision. At the critical moment of contact, fingers occlude the object from cameras, specular reflections degrade visual localization, and small perceptual errors accumulate into task failure. Vision-based tactile sensors (e.g., GelSight, DIGIT) provide rich contact information, but current learning frameworks fail to exploit it fully.  The paper identifies a core limitation: existing methods either:      Preserve raw tactile image richness but lack spatial localization (the policy knows contact happened but not where in hand-space).    Convert tactile data into geometric forms (e.g., point clouds) but lose fine details like contact texture and pressure distribution.    The key question: how to get both perceptual richness and spatial grounding in a single representation?  2. Method  2.1 Core Idea: Spatial Anchoring  SaTA anchors every tactile measurement to the hand’s URDF coordinate system (wrist frame) rather than the world or camera frame. The rationale: manipulation success depends on relative geometric relationships within the hand, not global positions. A USB insertion learned in one arm configuration should transfer to another.  2.2 Spatially-Anchored Tactile Encoder  For each tactile sensor (one per fingertip, 10 total on a dual-hand setup):      Forward kinematics: compute the sensor’s 6D pose (position + orientation) from current joint angles.    Fourier positional encoding: encode the 6D pose into multi-scale frequency features. Low frequencies capture coarse alignment; high frequencies capture fine adjustments (“rotate 2 degrees”, “translate 1mm”).    FiLM conditioning: use Feature-wise Linear Modulation to let spatial information modulate the tactile features from a ResNet encoder, rather than naive concatenation.    The result is a set of spatially-anchored tactile tokens – each preserving full tactile image features while carrying precise spatial context. The same edge pattern detected at the thumb vs. the index finger triggers different policy actions because FiLM enables context-dependent interpretation.  2.3 Policy Architecture  Built on ACT (Action Chunking with Transformers):      Inputs: robot joint states, RGB-D images (head + wrist cameras), and 10 spatially-anchored tactile tokens.    State encoder: cVAE to handle multi-modality and capture demonstration distribution.    Output: action chunk of 100 timesteps for smooth, anticipatory control.    Training: standard imitation learning from 200 expert teleoperation demonstrations per task.    The spatial anchoring idea is architecture-agnostic and could be plugged into Diffusion Policy or other frameworks.  3. Experiments and Results  Hardware      Dual-arm: two RealMan 7-DoF manipulators + Sharpa Wave 22-DOF dexterous hands.    Tactile: vision-based fingertip sensors (320x240 @ 30Hz) on each finger.    Visual: head-mounted stereo camera + two wrist fisheye cameras.    Tasks  Three tasks chosen specifically because visual information is severely degraded at the critical contact moment:      USB-C Mating (bimanual, free space): sub-mm positional tolerance, plug fully occludes port during approach.    Card Sliding: fan out cards at specific angles; requires force along card surface, not perpendicular.    Bulb Installation: thread engagement requires perpendicular alignment; small angular errors cause jamming.    Main Results                    Method        Card Sliding SR        USB-C Mating SR        Bulb Install SR        Avg SR                            Vision-Only        50%        0%        45%        31.7%                    Tactile-Flat (no anchoring)        60%        0%        70%        43.3%                    Tactile-Global (pose in proprioception)        65%        10%        65%        46.7%                    SaTA        95%        35%        100%        76.7%              Key observations:      On USB-C mating, all baselines essentially fail (0-10% SR). SaTA reaches 35% – still hard, but a qualitative leap.    First-contact success rate (correct alignment on first try) is 48.3% for SaTA vs. 25.0% for the best baseline, directly measuring geometric reasoning quality.    SaTA reduces average completion time by ~28% due to fewer trial-and-error attempts.    Ablation Study (Card Sliding)                    Configuration        SR                            SaTA (Full)        95%                    w/o FiLM (concat instead)        70%                    w/o Fourier encoding        70%                    World frame (instead of hand frame)        60%              Every component matters. The hand-frame anchoring is the most impactful single choice.  Failure Mode Analysis  Without spatial anchoring, baselines consistently fail in specific geometric ways:      Bulb: tilted insertion angle; cannot correct angular error from tactile feedback.    Card: applies force perpendicular to card surface (bending) instead of along it (sliding).    USB-C: cannot learn the thumb-index coordinated rubbing motion to adjust plug orientation.    These failures share a pattern: the policy detects contact but cannot map tactile patterns to correct spatial adjustments.  4. Three Levels of Tactile Sensing (from the paper’s discussion)  The authors propose a useful taxonomy:      Gating signals: binary contact detection to trigger phase transitions (~3 bits of information). Simple but crucial.    Geometric reasoning (this paper’s focus): high-precision local geometry to complement occluded vision. Requires spatial anchoring.    Force-dominant control: policies driven entirely by force/tactile feedback (e.g., pen spinning). Current teleoperation data collection limits this level because operators feel vibration, not actual force distributions.    5. Strengths      Clean, well-motivated design: the spatial anchoring idea is simple, principled, and architecture-agnostic.    Impressive task selection: USB-C mating in free space is genuinely hard and the kind of task that matters for real deployment.    Strong ablations: each component (FiLM, Fourier, hand frame) is justified with clear ablation results.    Failure mode analysis is thorough and provides insight into why spatial anchoring helps, not just that it helps.    6. Limitations      USB-C mating is still only 35% SR. The paper is honest about this – the task remains extremely challenging even with SaTA.    Teleoperation bottleneck: operators cannot feel actual force distributions during demonstration collection, so demonstrations are inherently vision-dominant. This limits progress toward force-dominant policies.    200 demonstrations per task is a moderate data requirement. The paper does not explore data efficiency or few-shot settings.    Single hardware platform: all experiments use the same Sharpa Wave hand. Generalization across hand morphologies is not tested.    7. Takeaways      Spatial grounding is the missing piece in current visuo-tactile learning. Simply feeding tactile images into a policy network wastes most of their geometric potential. Anchoring to the kinematic frame is a low-cost, high-impact change.    FiLM > concatenation for fusing spatial and tactile information. The same tactile pattern means different things at different fingers – modulation captures this; concatenation does not.    The three-level taxonomy (gating / geometric reasoning / force-dominant) is a useful mental model. Most current work barely reaches level 2. Level 3 requires better data collection (haptic feedback or RL), which is an open problem.    Interesting that the approach is from Sharpa (same group as DexEMG). They are building a full teleoperation + perception stack for dexterous manipulation.    本文支持通过顶部导航栏的语言切换按钮在 English / 中文 之间切换。  一句话总结  现有的视触觉学习方法能检测到接触发生，但难以推理接触在手部几何空间中的位置和含义。SaTA 通过前向运动学 + 傅里叶编码 + FiLM 条件调制将触觉特征锚定到手部运动学坐标系，实现了端到端的模仿学习策略，在双手 USB-C 对接、灯泡安装和卡牌滑..."
    },{
      "title": "DexEMG: Towards Dexterous Teleoperation System via EMG2Pose Generalization",
      "url": "/posts/2026/03/dexemg-paper-notes/",
      "date": "2026-03-19",
      "tags": ["Robotics","Dexterous Manipulation","Teleoperation","EMG","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  DexEMG is a lightweight teleoperation system that uses a commodity sEMG wristband to control a 22-DOF dexterous robotic hand. Instead of bulky exoskeletons or line-of-sight-constrained cameras, the operator wears a simple armband that captures forearm muscle signals. A neural network (EMG2Pose) maps those signals to continuous hand joint angles in real time. The system generalizes to unseen objects and cluttered environments, and can handle multi-stage tasks like desktop packaging and table wiping.  Paper Info      Title: DexEMG: Towards Dexterous Teleoperation System via EMG2Pose Generalization    Authors: Qianyou Zhao, Wenqiao Li, Chiyu Wang, Kaifeng Zhang    Affiliations: Sharpa, Shanghai Jiao Tong University    arXiv: 2603.05861    Paper type: teleoperation / dexterous manipulation / sEMG-based control    1. Problem and Motivation  High-fidelity teleoperation of dexterous hands is a prerequisite for deploying robots in unstructured domestic environments. The two dominant paradigms each have clear drawbacks:      Exoskeletons (e.g., CyberGrasp, Dexmo): precise but bulky, expensive, and cause operator fatigue.    Vision-based capture (e.g., Vicon, Leap Motion): either extremely expensive with strict environment requirements, or susceptible to self-occlusion when fingers are hidden by the palm or grasped objects.    sEMG is attractive because it reads neuromuscular signals directly from the forearm, is wearable and cheap, and is immune to visual occlusion. The key challenge is going from discrete gesture classification to continuous, high-dimensional pose estimation accurate enough for dexterous control.  2. Method  2.1 Data Collection via Kinematic Retargeting  The operator wears two devices simultaneously during data collection:      An 8-channel gForce sEMG armband on the forearm.    A Manus MoCap glove providing 35 skeletal keypoints as ground truth.    The captured human hand poses are retargeted to the 22-DOF Sharpa Wave hand via keypoint-based optimization:  \\(q^* = \\arg\\min_q \\sum_i | p_i^h - p_i^r(q) |_2^2\\)  A collision classifier checks the optimized joint angles and clamps to a safe manifold if self-collision is detected. The result is a paired dataset of sEMG streams and collision-free robotic joint angles.  2.2 EMG2Pose Architecture  The model follows an encoder-decoder design:      Encoder: raw sEMG input of shape \\((B, 8, T)\\) goes through two Conv1d blocks and two Time-Depth Separable (TDS) stages (2D conv + layer norm + feedforward + layer norm).    Decoder: an LSTM + MLP that predicts joint velocities \\(\\dot{\\theta}\\) rather than absolute angles. Poses are reconstructed iteratively: \\(\\theta_t = \\theta_{t-1} + \\dot{\\theta}_t\\), starting from a rest pose \\(\\theta_0\\).    The velocity-based approach decouples muscle activation intensity from static postures, reducing sensitivity to sensor displacement and signal drift during sustained grasping.  2.3 Deployment Pipeline  At deployment time the MoCap glove is removed. The operator wears only the sEMG armband and an HTC Vive Tracker for wrist tracking. The system runs inference on a sliding window of sEMG inputs, outputs action chunks of predicted joint angles, and executes the initial frames of each chunk for smooth, continuous control.  3. Experiments and Results  Pose Estimation Accuracy      Grasp tasks: MAE of 0.09 rad.    In-hand rotation tasks: MAE of 0.15 rad (more complex joint coupling and rapid transitions).    Generalization (Grasping)  Tested across 5 object categories (tiny, cylinder, sphere, irregular, deformable) with 20 trials each:                    Scenario        Overall SR        Overall DR                            Trained Objects        76.0%        14.5%                    Unseen Objects        66.0%        18.2%                    Novel Scenarios (cluttered)        56.0%        28.6%              The performance drop on unseen objects is moderate, suggesting the model learns generalizable motor patterns rather than overfitting to specific geometries. In novel (cluttered) scenarios the degradation is attributed mainly to arm-level planning difficulty rather than EMG model failure.  Long-Horizon Tasks                    Task        One-shot SR        With-retry SR                            Desktop Packaging        60%        80%                    Table Wiping        40%        70%              Wiping is harder because it requires sustained contact force; minor EMG drift causes the cloth to slip. With retry the system recovers well, indicating no irrecoverable failure states.  4. Strengths      Lightweight and cheap: a commodity sEMG armband replaces expensive exoskeletons or multi-camera setups.    Occlusion-immune: unlike vision-based systems, sEMG works even when fingers are hidden.    Velocity-based decoding: mitigates signal drift during sustained grasping.    Generalization: reasonable performance on unseen objects and cluttered environments without per-object recalibration.    Practical retry behavior: the system does not enter irrecoverable states after a failure, making it suitable for scalable data collection.    5. Limitations      Irregular and deformable objects remain challenging (25-50% SR in novel scenarios).    sEMG currently lacks the signal-to-noise ratio for extreme tactile precision tasks.    The system only controls the hand; arm-level motion still relies on separate wrist tracking (HTC Vive Tracker).    A latency-stability trade-off exists: longer input windows smooth noise but reduce responsiveness.    Still requires an initial calibration session (data collection with MoCap glove) though not per-object recalibration.    6. Takeaways      sEMG is a viable and underexplored modality for dexterous teleoperation. The key insight is predicting joint velocities rather than absolute angles, which makes the system more robust to sensor drift.    The approach offers a compelling cost-portability-performance balance for scalable data collection in unstructured environments. If you need to teleoperate dexterous hands for hours to collect demonstration data, wearing only a wristband is much more practical than an exoskeleton or being tethered to cameras.    The main bottleneck is not the EMG decoding itself but the precision ceiling of sEMG signals for fine manipulation. Combining sEMG with complementary sensing (e.g., tactile feedback or sparse vision) could push performance further.    本文支持通过顶部导航栏的语言切换按钮在 English / 中文 之间切换。  一句话总结  DexEMG 是一个轻量级遥操作系统，使用廉价的 sEMG（表面肌电）臂带 控制 22 自由度灵巧机器人手。操作者无需佩戴笨重的外骨骼或依赖受视线限制的摄像头，只需一个前臂臂带即可捕获肌肉信号。神经网络（EMG2Pose）将这些信号实时映射为连续的手部关节角度。该系统能泛化到未见过的物体和杂乱环境，并可完成桌面打包和擦桌子等多阶段任务。  论文信息      标题：DexEMG: Towards Dexterous Teleoperation System via EMG2Pose Generalization    作者：Qianyou Zhao, Wenqiao Li, Chiyu Wang, Kaifeng Zhang    机构：Sharpa、上海交通大学    arXiv：2603.05861    类型：遥操作 / 灵巧操作 / 基于 sEMG 的控制    1. 问题与动机  灵巧手的高保真遥操作是将机器人部署到非结构化家庭环境中的前提。现有两种主流方案各有明显缺陷：      外骨骼（如 CyberGrasp、Dexmo）：精度高但笨重、昂贵，容易使操作者疲劳。    视觉捕捉（如 Vicon、Leap Motion）：高端系统昂贵且环境要求苛刻；低成本方案在手指被手掌或被抓物体遮挡时容易失败。    sEMG 直接从前臂读取神经肌肉信号，可穿戴、成本低，且不受视觉遮挡影响。核心挑战在于从离散手势分类迈向足够精确的连续、高维姿态估计。  2. 方法  2.1 通过运动学重定向采集数据  数据采集时操作者同时佩戴两个设备：      前臂上的 8 通道 gForce sEMG 臂带。    提供 35 个骨骼关键点作为真值的 Manus 动捕手套。    采集到的人手姿态通过基于关键点的优化重定向到 22-DOF 的 Sharpa Wave 机器人手：  \\(q^* = \\arg\\min_q \\sum_i | p_i^h - p_i^r(q) |_2^2\\)  碰撞分类器检查优化后的关节角，如检测到自碰撞则钳制到安全流形。最终得到 sEMG 数据流与无碰撞机器人关节角的配对数据集。  2.2 EMG2Pose 网络架构  模型采用编码器-解码器设计：      编码器：原始 sEMG 输入形状为 \\((B, 8, T)\\)，经过两个 Conv1d 模块和两个时间深度可分离（TDS）阶段。    解码器：LSTM + MLP 预测关节速度 \\(\\dot{\\theta}\\) 而非绝对角度。姿态通过迭代重建：\\(\\theta_t = \\theta_{t-1} + \\dot{\\theta}_t\\)，初始值为静止姿态 \\(\\theta_0\\)。    基于速度的方法将肌肉激活强度与静态姿势解耦，降低了传感器偏移和持续抓取过程中信号漂移的敏感性。  2.3 部署流程  部署时移除动捕手套。操作者仅佩戴 sEMG 臂带和用于手腕追踪的 HTC Vive Tracker。系统对 sEMG 输入的滑动窗口进行推理，输出预测关节角的动作块（action chunk），执行每个块的起始帧以实现平滑连续控制。  3. 实验与结果  姿态估计精度      抓取任务：平均绝对误差 0.09 rad。    手内旋转任务：平均绝对误差 0.15 rad（涉及更复杂的关节耦合和快速转换）。    泛化能力（抓取）  在 5 类物体（微小、圆柱、球体、不规则、可变形）上各测试 20 次：                    场景        总体成功率        总体掉落率                            训练物体        76.0%        14.5%                    未见物体        66.0%        18.2%                    新场景（杂乱）        56.0%        28.6%              在未见物体上性能下降适中，说明模型学到了可泛化的运动模式而非过拟合到特定几何形状。新场景中的性能下降主要归因于臂级规划难度增加，而非 EMG 模型本身的失败。  长时序任务                    任务        一次成功率        允许重试成功率                            桌面打包        60%        80%                    擦桌子        40%        70%              擦桌子更难，因为需要持续接触力；EMG 轻微漂移会导致布料滑落。允许重试后系统恢复良好，说明不存在不可恢复的失败状态。  4. 优点      轻量且廉价：一个普通 sEMG 臂带取代了昂贵的外骨骼或多相机系统。    不受遮挡影响：与视觉系统不同，sEMG 在手指被遮挡时仍然有效。    基于速度的解码：缓解了持续抓取过程中的信号漂移问题。    泛化能力：在未见物体和杂乱环境中表现合理，无需逐物体重新标定。    良好的重试特性：失败后系统不会进入不可恢复状态，适合规模化数据采集。    5. 局限性      不规则和可变形物体 仍具挑战性（新场景中成功率 25-50%）。    sEMG 目前的信噪比不足以支撑极端触觉精度任务。    系统仅控制手部；臂级运动仍依赖独立的手腕追踪（HTC Vive Tracker）。    存在延迟-稳定性权衡：更长的输入窗口平滑噪声但降低响应速度。    仍需初始标定环节（使用动捕手套采集数据），虽然不需要逐物体重新标定。    6. 启示      sEMG 是灵巧遥操作中一种可行且尚未被充分探索的模态。核心洞察在于预测关节速度而非绝对角度，..."
    },{
      "title": "Cross-Hand Latent Representation for Vision-Language-Action Models",
      "url": "/posts/2026/03/xl-vla-paper-notes/",
      "date": "2026-03-14",
      "tags": ["Robotics","Dexterous Manipulation","Vision-Language-Action Models","Cross-Embodiment Learning","Latent Actions","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  XL-VLA tackles a real bottleneck in dexterous robot learning: every new robot hand comes with a different joint space, so standard VLA models do not scale cleanly across embodiments.  The paper’s solution is to learn a shared latent action space across multiple dexterous hands, then train a single VLA policy to predict latent action chunks instead of raw joint commands.  That simple abstraction works surprisingly well:      cross-hand mean success improves from 0.32 with pi0 to 0.72 with XL-VLA    on G1 cross-robot evaluation, mean success improves from 0.525 to 0.825    the model also shows zero-shot transfer to unseen hand-task combinations    My short reading is that the paper’s main contribution is not just a better dexterous policy. It is a useful systems argument that for cross-embodiment dexterous manipulation, the action representation is the real bottleneck.  Paper Info      Title: Cross-Hand Latent Representation for Vision-Language-Action Models    Authors: Guangqi Jiang, Yutong Liang, Jianglong Ye, Jia-Yang Huang, Changwei Jing, Rocky Duan, Pieter Abbeel, Xiaolong Wang, Xueyan Zou    Affiliations: UC San Diego, Amazon FAR, UC Berkeley    Project page: xl-vla.github.io    Paper type: dexterous manipulation / cross-embodiment learning / VLA    1. Problem Setting and Motivation  The paper starts from a clean observation: language has a fairly stable vocabulary, but robot actions do not.  This is especially painful for dexterous hands:      different hands have different numbers of fingers    different actuation structures    different joint parameterizations    new hardware keeps appearing quickly    That makes standard VLA training expensive and fragmented. Even if two hands should perform the same task, their raw action spaces are incompatible.  The paper asks two practical questions:      how can we define a unified action representation across a family of dexterous hands?    how can we integrate a new hand without retraining a full policy from scratch?    2. Core Idea  The central idea is to replace raw joint-space prediction with a shared latent action space.  For each hand h, the system learns:      a hand-specific encoder E_h    a hand-specific decoder D_h    These map between:      the hand’s own joint-space action chunk    a common latent action vector    The VLA policy itself is then hand-agnostic:      it takes vision, language, and previous latent action tokens    predicts the next latent action chunk    the hand-specific decoder turns that latent back into joint commands    This gives a very clean separation:      the VLA backbone does not need to understand every hand’s kinematics directly    embodiment-specific details are pushed into encoders and decoders    3. Method Breakdown  3.1 Action chunks instead of single-step actions  The model operates on action chunks rather than single actions.  Each chunk is:      64 joint-position commands    sampled at 20 Hz    corresponding to 3.2 seconds of motion    At time t, the policy receives:      image observations    language instruction    short history of joint states    previously executed action chunk    and predicts the next chunk.  This chunked setup is a reasonable fit for dexterous bimanual manipulation because many tasks require coordinated temporal structure, not just immediate motor responses.  3.2 Shared latent space via a multi-headed VAE-style autoencoder  The latent space is pretrained independently of the VLA backbone.  For each hand, the encoder produces a Gaussian posterior:      mean mu^(h)    variance sigma^(h)    and the decoder reconstructs that latent into the hand’s own joint configuration.  The paper uses lightweight MLPs for these hand-specific encoders and decoders, but all of them share the same latent distribution.  3.3 Three losses define the latent space  The latent space is shaped by three constraints:      Reconstruction loss L1    Retargeting loss L2    Latent KL regularization L3    Reconstruction loss  This ensures each hand can autoencode its own joint configurations accurately.  Retargeting loss  This is the most interesting part. The paper uses differentiable forward kinematics to align fingertip geometry across hands.  Instead of matching raw joint angles across embodiments, it matches:      pinch distances    pinch directions    between corresponding fingers.  That is a good design choice because cross-hand equivalence is geometric and functional, not joint-wise.  Latent KL loss  The KL term encourages the shared latent space to follow a standard Gaussian prior, making it smoother and easier to interpolate.  3.4 Training without paired cross-hand demonstrations  One of the strongest details in the paper is that the latent autoencoder is trained without paired cross-hand trajectories.  Instead, the method:      samples random joint configurations within each hand’s limits    encodes them into latent codes    decodes those latents through all hand-specific decoders    uses reconstruction and geometric retargeting losses to align the space    So the cross-hand alignment is effectively self-supervised through kinematics, not supervised through matched demonstrations.  That makes the approach much more scalable than collecting paired multi-hand teleoperation data just for latent alignment.  4. Experimental Setup  The authors collect a real-world teleoperation dataset with:      4 dexterous hands: Ability, Inspire, X-Hand1, Paxini DexH13    10 tasks    50 demonstrations per task per hand    2000 demonstrations total    about 2M state-action pairs    The tasks include:      prepare fruits    stack cans    sort cans    hand over bottle    reorganize lemons    pour sauce    rearrange boxes    push sugar    pour sugar    push cans    The hardware setup includes:      a bimanual xArm7 platform    a Unitree G1 humanoid    5. Main Results  5.1 Cross-hand VLA training  The core comparison is against pi0, trained under the same multi-hand multi-task setting.  Mean success across all tasks and hands:      pi0: 0.32    XL-VLA: 0.72    Per-hand means:      Ability: 0.37 -> 0.73    Inspire: 0.27 -> 0.68    Paxini: 0.35 -> 0.78    XHand: 0.29 -> 0.70    Those are large gains, especially because dexterous hands differ much more than ordinary grippers.  Task-wise, XL-VLA improves the mean success rate on every listed task category in Table 2, with especially large gains on:      Hand over Bottle    Sort Cans    Re-arrange Boxes    Pour Sugar    The broader point is clear: once the action representation is aligned, a single VLA backbone becomes much more reusable across embodiments.  5.2 Cross-robot scaling to G1  The paper also tests whether the latent action space helps when mixing data from:      tabletop xArm    humanoid G1    On four tasks, the reported G1 mean success improves from:      pi0: 0.525    XL-VLA: 0.825    This is a 57% relative improvement according to the paper.  That result matters because it suggests the latent space is not just smoothing over minor hand differences. It is helping across broader robot-system variation as well.  5.3 Zero-shot unseen-task transfer  The paper also evaluates zero-shot unseen-task generalization.  For each hand, some tasks are held out during training. The trained policy is then tested directly on those unseen hand-task combinations using the corresponding decoder.  The comparison baseline is pi0 + RT:      train a policy on XHand    retarget predicted trajectories to the other hands using kinematic retargeting    The reported result is qualitatively strong:      XL-VLA consistently outperforms the retargeting baseline    it never underperforms that baseline on any hand or task    gains are especially clear on fine-grained dexterous tasks    This is exactly where a latent action space should help: it transfers functional control patterns, not only fingertip geometry after the fact.  6. Ablation Results  6.1 Latent replay vs LAD  The paper compares its latent space with LAD, a supervised latent retargeting method.  Replay mean success:      LAD on Ability+Inspire: 0.60    XL-VLA on Ability+Inspire: 0.82    LAD on Paxini+XHand: 0.61    XL-VLA on Paxini+XHand: 0.81    This is a strong result because XL-VLA’s latent alignment is unsupervised, yet it still outperforms the supervised alternative.  6.2 Loss design matters  The loss ablations show a clean pattern:      removing L1 destroys reconstruction    removing the distance part of L2 hurts cross-hand distance preservation    removing the direction part of L2 hurts cross-hand directional consistency    removing L2 entirely causes the largest cross-embodiment degradation    That supports the paper’s modeling decision: a shared latent action space only works if it is explicitly shaped by cross-hand geometry.  6.3 Latent size should not be too large  The architecture and latent-dimension ablations suggest that very large latent space..."
    },{
      "title": "Visual Dexterity: In-Hand Reorientation of Novel and Complex Object Shapes",
      "url": "/posts/2026/03/visual-dexterity-paper-notes/",
      "date": "2026-03-14",
      "tags": ["Robotics","Dexterous Manipulation","In-Hand Manipulation","Reinforcement Learning","Sim-to-Real","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  This paper tackles a very hard version of dexterous manipulation: in-hand reorientation of novel, complex objects using only a single commodity depth camera plus joint sensing.  The authors’ key claim is not that they have solved dexterous reorientation perfectly. They have not. The real contribution is showing that a single sim-trained controller can:      reorient previously unseen object shapes    operate in real time at around 12 Hz    handle arbitrary target rotations in SO(3)    work in the much harder downward-facing hand setup    even perform in-air reorientation with a four-fingered hand    My short take is that this paper is important because it moves dexterous reorientation from a heavily constrained benchmark setting toward something much closer to a deployable real-world manipulation skill.  Paper Info      Title: Visual Dexterity: In-Hand Reorientation of Novel and Complex Object Shapes    Authors: Tao Chen, Megha Tippur, Siyang Wu, Vikash Kumar, Edward Adelson, Pulkit Agrawal    Affiliations: MIT, Tsinghua University, Meta AI, IAIFI    arXiv: 2211.11744    Paper type: dexterous manipulation / reinforcement learning / sim-to-real transfer    1. Problem and Motivation  In-hand reorientation is one of those manipulation tasks that looks narrow at first but is actually central to broader dexterity. If a robot picks up a tool, it usually cannot use that tool immediately. It first has to rotate the object into the right pose. So reorientation is not just a benchmark; it is a prerequisite for flexible tool use.  The paper argues that many previous systems only worked because they simplified the problem in one or more ways. Typical assumptions included:      only simple object shapes    only limited ranges of rotation    quasi-static manipulation    simulation-only results    object-specific pose estimators    expensive sensing setups    upward-facing hands instead of downward-facing ones    The downward-facing setup matters a lot. When the hand points downward, the controller has to manipulate the object while also preventing gravity from ending the episode immediately. That makes the task much closer to practical robot use and much farther from convenient laboratory settings.  2. Main Idea  The main technical idea is to learn a controller that maps:      a point cloud from a single depth camera    the hand’s proprioceptive state    a point-cloud goal representation    directly to joint commands for reorientation.  Instead of estimating object pose through an object-specific tracker, the method predicts actions directly from point clouds. This is a strong design decision because pose or keypoint representations often break the moment the object class changes or the geometry becomes awkward.  The larger claim is that direct perception-to-action control with point clouds can generalize to new object shapes better than explicit object-specific pose pipelines.  3. Training Pipeline  3.1 Teacher-student structure  The paper first notes that reinforcement learning from visual inputs is too expensive if the system has to learn both perception and control together from scratch. Their solution is a two-stage teacher-student pipeline.  The teacher is an RL policy trained in simulation with low-dimensional state information. The student is a visual policy trained to mimic the teacher.  This is a familiar pattern in dexterous learning, but what matters here is how the authors made it practical enough for multi-object training.  3.2 Faster visual training with synthetic point clouds  The paper identifies rendering speed as a serious bottleneck. Rendering-rich visual simulation would have made training take more than twenty days under their compute budget.  So they introduce a two-stage visual-policy training process:      first train with synthetic point clouds that avoid expensive rendering    then finetune with rendered point clouds to reduce the sim-to-real gap    They report this makes training about 5x faster.  That detail is easy to overlook, but it is important. A lot of sim-to-real visual RL papers quietly depend on training pipelines that are too slow for iteration. This paper explicitly tries to keep the pipeline experimentally usable.  3.3 Sparse convolutions for real-time control  To process point clouds quickly enough, the controller uses a sparse convolutional network. The final system runs at about 12 Hz in real time.  This is another pragmatic choice. The paper is not only asking whether the policy can succeed, but whether it can run at a control rate fast enough for dynamic reorientation.  4. Hardware and Setup  The real-world platform is built around an open-source D’Claw manipulator, with both:      a three-fingered version    a modified four-fingered version    The sensing stack is intentionally simple:      one Intel RealSense depth camera    joint encoders    The paper emphasizes that the hardware costs less than $5,000, which is a major contrast to many prior dexterous systems that depend on expensive robot hands, tactile sensing suites, or motion capture for operation.  This cost claim matters because the contribution is partly methodological and partly infrastructural. The paper is trying to show that meaningful dexterous reorientation research does not have to sit behind a six-figure hardware barrier.  5. Experimental Story  The authors train on 150 objects in simulation, then evaluate on real-world objects not used for training. They consider two main settings:      reorientation with a supporting table surface    reorientation in the air without support    They also separate:      in-distribution objects from the training set    out-of-distribution held-out objects    This gives the paper a pretty clean narrative: first demonstrate real-world table-supported reorientation, then show robustness to surface changes, then escalate to the harder in-air condition.  6. Table-Supported Reorientation  The easier setting is still nontrivial: the hand faces downward, but the object can use the table as support. This is a form of extrinsic dexterity.  6.1 Three fingers are enough on the table  With a supporting surface, the three-fingered manipulator is already effective.  For train objects with rigid fingertips, the paper reports:      81% success within 0.4 radians    95% success within 0.8 radians    For held-out test objects with rigid fingertips:      45% success within 0.4 radians    75% success within 0.8 radians    This already shows something important: the controller does generalize, but precision degrades significantly on new shapes.  6.2 Soft fingertips help OOD generalization  The authors then switch from rigid fingertips to soft elastomer-coated fingertips.  This does not really change in-distribution performance much, but it improves held-out generalization:      OOD success within 0.4 radians rises from 45% to 55%    OOD success within 0.8 radians rises from 75% to 86%    That is a very believable robotics result. Better compliance and friction help reduce the brittleness of contact-rich manipulation, especially on unfamiliar geometries.  6.3 Robustness to support materials  The paper also evaluates different table materials, including rough cloth, smooth cloth, slippery acrylic, perforated bath mat, and an uneven doormat.  The qualitative takeaway is that the controller behaves reasonably consistently across these different supporting surfaces, suggesting some robustness to altered contact dynamics.  7. In-Air Reorientation  This is the paper’s hardest and most interesting setting.  7.1 Three fingers fail, four fingers matter  When the supporting surface is removed, the previously trained controllers fail by dropping the object. The paper’s solution is to move to a four-fingered hand and modify the reward so the policy is encouraged to avoid using external support.  The result is strong conceptually: when trained with the right reward structure, in-air reorientation emerges.  The authors argue that four fingers help because:      there are more possible finger configurations that can stabilize the object    the redundancy makes the system more tolerant to action errors    That explanation is plausible and consistent with the reported learning curves.  7.2 Accuracy remains similar when the object is not dropped  A nice nuance in the results is that when the object is not dropped, the orientation error distribution in air is similar to the supported setting. This suggests the harder part is not necessarily precise target alignment; it is maintaining stable grasp and contact during dynamic reorientation.  7.3 Reorientation time  The controller is also fairly fast. The paper reports a median reorientation time under about seven seconds across full-SO(3) targets.  This is a useful contrast with earlier work that could reorient under narrower assumptions but much more slowly.  8. Generalization to Dail..."
    },{
      "title": "TiPToP: A Modular Open-Vocabulary Planning System for Robotic Manipulation",
      "url": "/posts/2026/03/tiptop-paper-notes/",
      "date": "2026-03-14",
      "tags": ["Robotics","Manipulation","Task and Motion Planning","Vision-Language Models","Embodied AI","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  TiPToP is a modular robotic manipulation system that turns a stereo RGB observation plus a natural-language instruction into a complete manipulation plan, without any robot training data.  Its core claim is simple but important: for long-horizon tabletop manipulation, a pipeline built from foundation-model perception + explicit task-and-motion planning + precise execution can compete with, and often outperform, a large end-to-end VLA policy that was fine-tuned on hundreds of hours of embodiment-specific demonstrations.  What makes the paper stand out is not only the benchmark result. It is the systems argument that modularity is still a strong design choice in robotics, because it gives:      better semantic grounding on distractor-heavy tasks    stronger multi-step reasoning through symbolic planning    clearer failure diagnosis at the component level    easier cross-embodiment deployment    Paper Info      Title: TiPToP: A Modular Open-Vocabulary Planning System for Robotic Manipulation    Authors: William Shen, Nishanth Kumar, Sahit Chintalapudi, Jie Wang, Christopher Watson, Edward Hu, Jing Cao, Dinesh Jayaraman, Leslie Pack Kaelbling, Tomas Lozano-Perez    Affiliations: MIT CSAIL, University of Pennsylvania    arXiv: 2603.09971    Project page: tiptop-robot.github.io    Paper type: robotic manipulation / modular systems / task and motion planning    1. Problem Setting and Motivation  The paper studies a practical manipulation setting:      input: a stereo RGB image pair and a language instruction    output: robot joint trajectories and gripper commands that complete the task    The target is not just simple pick-and-place. The tasks include:      distractor-heavy object selection    semantic grounding such as “matching plate” or “largest toy”    multi-step manipulation with obstacle removal or packing    The authors position TiPToP against a strong baseline: pi0.5-DROID, a vision-language-action model fine-tuned on 350 hours of embodiment-specific demonstrations.  The motivation is clear:      end-to-end VLAs are appealing, but expensive in data and hard to debug    classical TAMP is structured and interpretable, but has usually been too brittle or too tightly engineered    recent foundation models make it possible to revisit modular planning with much stronger perception    2. System Overview  TiPToP is split into three modules:      Perception    Planning    Execution    The full policy is planner-based. It observes the scene once at the beginning and then executes an open-loop plan.  2.1 Perception module  The perception module builds an object-centric scene representation from the initial stereo observation and the language instruction.  Its two branches run in parallel:      a 3D vision branch for depth and grasp generation    a semantic branch for object detection, segmentation, and goal grounding    The key ingredients are:      FoundationStereo for dense stereo depth    M2T2 for 6-DoF grasp proposals    Gemini Robotics-ER 1.5 for open-vocabulary object detection and symbolic goal grounding    SAM-2 for segmentation    The output is a set of per-object meshes, candidate grasps, and symbolic goal predicates.  2.2 Planning module  Planning is handled by cuTAMP, a GPU-parallelized task-and-motion planner.  Given the symbolic goal, TiPToP:      enumerates candidate plan skeletons    samples grasps, placements, IK solutions, and trajectories    optimizes these parameters under collision and feasibility constraints    invokes cuRobo for collision-free motion generation    This is the paper’s central engineering point: instead of hoping a policy implicitly discovers long-horizon structure, TiPToP builds that structure explicitly.  2.3 Execution module  The robot executes the planned trajectory with a joint impedance controller.  This makes accurate tracking a first-class requirement. Because TiPToP does not replan during execution, errors from slipping, missed grasps, or object motion can directly cause task failure.  3. Technical Details That Matter  Several implementation choices are more important than they may look at first glance.  3.1 From pixels to symbolic goals  Instead of using a VLM only for captions or labels, TiPToP asks the VLM to produce symbolic goals such as On(a, b).  That matters because the planner can then reason over:      which objects are relevant    what relations need to hold    which multi-step action sequence could satisfy them    This is what enables tasks like sorting by color or placing items on matching containers.  3.2 Convex-hull object completion  Each segmented object is converted into a watertight mesh by projecting observed points downward and taking the convex hull.  This is a pragmatic choice:      it is cheap    it provides conservative geometry for collision checking    but it also causes errors on concave shapes like bananas    The paper’s failure analysis shows this approximation is one of the main weak points.  3.3 Open-loop planning as a tradeoff  TiPToP uses a single initial observation and then executes open-loop.  This gives:      fast task completion    strong geometric consistency with the planner    But it also removes:      recovery after failed grasps    correction after object slip    adaptation to unexpected scene changes    The paper is honest that this tradeoff is currently one of the biggest limitations.  4. Experimental Results  The evaluation spans 28 tasks and 165 trials across:      simulation    the authors’ DROID setup    an external evaluation team’s DROID setup    The tasks are grouped into:      simple    distractor    semantic    multi-step    4.1 Main comparison against pi0.5-DROID  The headline result is:      TiPToP: 98/165 successes, 74.6%    pi0.5-DROID: 55/165 successes, 52.4%    The most interesting pattern is not that TiPToP wins everywhere. It does not.  Instead:      on simple tasks, the two systems are fairly close    on distractor tasks, TiPToP is much stronger    on semantic tasks, TiPToP is much stronger    on multi-step tasks, TiPToP is again much stronger    This matches the architecture:      VLM grounding helps when language and semantic selection matter    TAMP helps when multi-step structure and collision constraints matter    end-to-end reactive control still helps on fragile grasps and execution recovery    4.2 Time-to-success  TiPToP is also often faster than the VLA baseline on successful trials.  Examples from Table II:      can -> mug (sim): 18.6s vs 41.0s    crackers -> tray (simple): 14.9s vs 32.2s    crackers -> tray (medium): 14.9s vs 45.2s    The reason is straightforward: TiPToP executes a planned trajectory directly, while the reactive VLA may spend extra time probing, retrying, or idling.  5. Failure Analysis  This section is one of the paper’s strongest parts.  The authors manually analyzed 173 additional real-world trials and traced failures to specific modules. The dominant categories are:      grasping failures: 31 / 55 failures    scene completion errors: 13 / 55    VLM errors: 6 / 55    cuTAMP failures: 5 / 55    The big takeaway is that grasping and execution robustness dominate the remaining error budget.  In other words, the planning stack is already fairly strong. The larger problems are:      bad grasp proposals    slip during transport    mesh approximation errors from partial observation    lack of visual feedback during execution    This is exactly the kind of conclusion that modular systems make easier to reach.  6. Why the Paper Is Interesting  I think the paper makes three useful arguments.  6.1 Modular systems are still competitive  There is a strong current narrative that large end-to-end policies will absorb everything. TiPToP pushes back with a concrete counterexample: if task structure matters, explicit planning can still be very competitive.  6.2 Better debugging is a real research advantage  Because the system is decomposed, the authors can identify whether failures come from:      perception    mesh completion    grasp generation    planning    execution    That is much more actionable than simply reporting a task-level failure rate.  6.3 Cross-embodiment deployment matters  The authors also show deployment on UR5e and WidowX AI. This is important because many robotics systems look strong only inside one tightly controlled stack. TiPToP argues for a reusable interface between perception, planning, and embodiment-specific execution.  7. Limitations  The paper is strong, but the limitations are substantial and worth keeping in view.      Open-loop execution is the biggest weakness. Many failures could likely be recovered with re-perception and re-planning.    Single-view perception limits object visibility and mesh quality.    Convex hull geometry is too crude for concave objects and can distort collision reasoning.    The system still depends on a fairly heavyweight collection of exter..."
    },{
      "title": "Reward Prediction with Factorized World States",
      "url": "/posts/2026/03/statefactory-paper-notes/",
      "date": "2026-03-14",
      "tags": ["LLM Agents","World Models","Reward Modeling","Planning","State Representation","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  This paper asks a very practical question for planning agents:  Can we predict useful rewards in a zero-shot way without training a task-specific reward model, simply by building a better state representation?  The answer proposed here is StateFactory, a method that factorizes text observations into hierarchical object-attribute world states, then predicts reward as the semantic similarity between the current state and a dynamically interpreted goal state.  The paper contributes both:      a new benchmark, RewardPrediction, with 2,454 trajectories across five domains    a zero-shot reward prediction framework that beats strong baselines    The headline numbers are:      60% lower EPIC distance than VLWM-critic    8% lower EPIC distance than LLM-as-a-Judge    planning gains of +21.64% on AlfWorld    planning gains of +12.40% on ScienceWorld    My short read is that the paper’s strongest idea is simple: reward quality is largely a state-representation problem.  Paper Info      Title: Reward Prediction with Factorized World States    Authors: Yijun Shen, Delong Chen, Xianming Hu, Jiaming Mi, Hongbo Zhao, Kai Zhang, Pascale Fung    Affiliations: East China Normal University, HKUST    arXiv: 2603.09400    Project page: statefactory.github.io    Paper type: agent planning / reward prediction / world-state representation    1. Problem and Motivation  Planning agents need more than just next-state prediction. They also need to know:      whether a predicted state is closer to the goal    how much progress an action makes    which branch of a plan is more promising    The usual fix is to train a supervised reward model. But the paper points out a real problem:      supervised reward models can overfit to training domains    reward labels can encode dataset-specific biases    generalization to new goals and environments can collapse    So the paper explores a different route:      do not train a domain-specific reward predictor    instead, build a structured state representation    estimate reward by measuring distance between current state and goal state    This is a good framing because it shifts the problem from “how do I fit a scalar critic?” to “how do I represent world state so that semantic distance actually reflects task progress?”  2. The RewardPrediction Benchmark  One of the paper’s strongest contributions is the benchmark itself.  RewardPrediction contains:      2,454 unique trajectories    step-wise actions, observations, and ground-truth rewards    five interactive domains    The five domains are:      AlfWorld    ScienceWorld    TextWorld    WebShop    BlocksWorld    This mix is useful because it spans:      embodied planning    scientific reasoning    text-adventure procedural tasks    web navigation    classical symbolic planning    2.1 Why this benchmark matters  Most prior setups evaluate sparse end success or domain-specific reward quality. This benchmark instead focuses on step-wise progress estimation, which is much closer to what a planner actually needs.  The evaluation metric is EPIC distance, which compares predicted reward sequences against ground-truth reward sequences. Lower is better.  I think this is the right choice for the paper’s claim, because the goal is not only to know who wins at the end, but whether the model’s reward landscape is actually aligned with task progress.  2.2 Benchmark construction  The data construction is also thoughtful.  Each task instance includes:      a positive trajectory    a negative trajectory    Positive trajectories come from expert demonstrations and are densified with interpolated progress rewards. Negative trajectories come from random policies and are filtered to avoid accidental overlap with expert behavior.  That design makes it harder for a method to game the benchmark with shallow heuristics such as trajectory length or generic optimism.  3. StateFactory  The core method is StateFactory, a representation-based reward predictor.  Instead of regressing reward directly from raw text history, it decomposes the process into:      state extraction    goal interpretation    hierarchical routing    The final reward is computed as semantic similarity between the extracted current state and the interpreted goal state.  4. Method Breakdown  4.1 State extraction  StateFactory converts each observation into a structured state:      a set of objects    each object paired with attributes and values    The paper writes each object as an identity plus attribute-value pairs, such as:      object identity: Mug    attribute: location    value: on the table    This is a stronger abstraction than:      raw observation text, which is noisy    plain object-centric state, which often entangles multiple properties together    The state extraction is also recurrent and goal-conditioned. It uses:      current observation    previous state    previous action    previous goal interpretation    original goal text    That matters because state tracking in long-horizon tasks is not just parsing a snapshot. It is updating a belief over evolving world state.  4.2 Goal interpretation  The paper argues that static goal representations can create an illusion of progress because they do not adapt to changes during execution.  So StateFactory treats goal interpretation as a dynamic process:      start from the language goal    repeatedly refine its grounded meaning using current state and trajectory context    This is an important design choice. In practical planning, the operational meaning of a goal often becomes clearer as the agent interacts with the environment.  4.3 Hierarchical routing  This is the mechanism that turns structured states into rewards.  For each goal object, the method:      finds the best matching object in the current state    checks identity similarity    checks attribute-value similarity    aggregates these scores into local progress    averages over all goal objects to get global reward    The matching is hierarchical:      first align the right object    then align the right attributes    then measure how close the values are    This is much more interpretable than asking an LLM to emit a reward directly.  5. Main Results on RewardPrediction  Table 1 is the key result.  5.1 Zero-shot comparison  Overall EPIC distance:      VLWM-critic: 0.738    LLM-as-a-Judge (best listed): 0.322    StateFactory: 0.297    So StateFactory improves substantially over the zero-shot baselines, especially against VLWM-critic.  The abstract summarizes this as:      60% lower EPIC distance than VLWM-critic    8% lower EPIC distance than LLM-as-a-Judge    Those numbers match the main takeaway: explicit structured state helps reward generalization.  5.2 Comparison to supervised reward models  The supervised models are especially interesting because they expose the generalization gap.  When trained on a single domain, they do very well in-domain, but their error rises sharply on unseen domains. The paper reports an average 138% increase in error when transferring out of domain.  That is probably the most important result in the paper besides StateFactory itself:  supervised reward modeling is powerful in-domain, but brittle for cross-domain zero-shot planning.  5.3 Nearing the supervised upper bound  A particularly strong result is that StateFactory’s zero-shot performance gets close to the supervised model trained on all domains.  That does not mean supervision is useless. It means that for this task, a strong structured representation can recover a large fraction of what supervised critics were learning.  6. Ablations  The ablations are well aligned with the paper’s claim.  6.1 Representation granularity matters  Figure 5(a) shows a clear progression:      raw observations are worst    plain textual states are better    object-centric states are better still    full object-attribute factorization is best    The argument is convincing:      raw observations contain too much distractor text    object-only states still entangle attributes    object-attribute factorization separates the parts of state that actually change during task progress    6.2 Dynamic goal interpretation is close to oracle  The online goal interpretation performs only slightly worse than an offline oracle goal-state setup, with about a 0.02 EPIC gap according to the paper.  That is a strong sign that the dynamic goal grounding mechanism is not the main bottleneck.  6.3 Better reasoning models help  The paper also shows that stronger LLM backbones and “thinking” modes improve factorization quality. This is a good sign for scalability: the method should benefit as reasoning models improve.  6.4 Embedding quality matters  StateFactory depends on semantic embeddings to measure similarity. The paper shows a strong correlation between triplet-based embedding accuracy and final reward performance.  That is useful because it clarifies w..."
    },{
      "title": "DiT4DiT: Jointly Modeling Video Dynamics and Actions for Generalizable Robot Control",
      "url": "/posts/2026/03/dit4dit-paper-notes/",
      "date": "2026-03-14",
      "tags": ["Robotics","Vision-Language-Action Models","Video Diffusion","Humanoid Robotics","Policy Learning","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  DiT4DiT argues that robot policies should not rely only on representations inherited from static image-text pretraining. Instead, it uses a video diffusion transformer to model future dynamics and then conditions an action diffusion transformer on intermediate denoising features from that video model.  The central message is strong: video generation can act as a much better scaling proxy for robot policy learning than semantic-only visual pretraining. In the paper, this gives:      98.6% average success on LIBERO    50.8% average success on RoboCasa-GR1    better sample efficiency than semantic-centric baselines by over 10x    faster convergence by up to 7x    My short reading is that this paper is not just proposing another VLA variant. It is making a broader claim that future dynamics modeling is a more useful foundation for control than static vision-language semantics alone.  Paper Info      Title: DiT4DiT: Jointly Modeling Video Dynamics and Actions for Generalizable Robot Control    Authors: Teli Ma, Jia Zheng, Zifan Wang, Chunli Jiang, Andy Cui, Junwei Liang, Shuo Yang    Affiliations: Mondo Robotics, HKUST(GZ), HKUST    arXiv: 2603.10448    Project page: dit4dit.github.io    Paper type: robot policy learning / video-action models / diffusion transformers    1. Problem Setting and Motivation  The paper starts from a clean criticism of current VLA systems:      most robot policies inherit backbones pretrained on static image-text data    physical dynamics must then be learned from relatively limited robot action data    this creates a mismatch between what the representation is good at and what control actually needs    By contrast, modern video generation models are trained to predict temporally coherent and physically plausible futures. The authors argue that these models already internalize:      motion priors    temporal structure    causal transitions    implicit physical dynamics    So the main question becomes:      can video generation be used as an effective proxy task for robot control?    if yes, how should video features be connected to action generation?    2. Core Idea  DiT4DiT combines:      a Video DiT that predicts future visual dynamics    an Action DiT that predicts robot actions    The key design choice is that the action model is not conditioned on fully reconstructed future frames. Instead, it uses intermediate hidden states from the video denoising process.  That is a good idea for two reasons:      it keeps the action policy tied to temporally grounded dynamics rather than only final pixels    it avoids forcing control to depend on a fully rendered future prediction    Conceptually, the method turns video generation into a source of actionable latent dynamics, rather than treating it as an auxiliary output.  3. Method Breakdown  3.1 Video DiT as a dynamics backbone  The video side is initialized from Cosmos-Predict2.5-2B. Observations and future frames are encoded into latent space with a frozen VAE, and the Video DiT is trained with flow matching to predict future latent dynamics conditioned on the current observation and language goal.  The paper formulates the interpolation path as:  x_tau = (1 - tau) x_0 + tau z  and trains a velocity field to recover the target flow:  v*(x_tau, tau) = z - x_0  This is standard flow matching machinery, but the important part is how the hidden activations are reused downstream.  3.2 Action DiT conditioned on denoising features  The action model is adapted from GR00T-N1 and acts as a separate flow-matching transformer.  Its inputs include:      robot proprioceptive state    noisy action trajectory    learnable future tokens    hidden features extracted from the video denoising process    Cross-attention fuses these signals so that action prediction is grounded in the visual dynamics encoded by the Video DiT.  3.3 Tri-timestep design  One of the most interesting design choices is the paper’s tri-timestep scheme.  It uses three different timesteps:      tau_v for video generation, sampled uniformly    tau_f for feature extraction, fixed to stabilize visual conditioning    tau_a for action generation, sampled from a Beta-based scheme to emphasize important control phases    This is a pragmatic solution to a real optimization problem: the model wants stochastic diffusion training for video generation, but stable conditioning for action learning.  3.4 Joint dual flow-matching objective  The model is trained end-to-end with a joint loss:      one flow-matching term for video prediction    one flow-matching term for action prediction    The action loss is conditioned on hidden features from the video branch, and the total objective balances both terms with a scalar coefficient.  I think this is the paper’s most important technical contribution. It is not just “video features help actions”; it is that video and action diffusion are optimized together so the latent space becomes useful for control.  4. Why the Proxy Objective Matters  Before presenting the main system results, the paper runs a useful validation study: it compares three training paradigms as proxy objectives for policy learning:      grounding    FLARE-style latent modeling    video generation    The reported conclusion is that video generation is the strongest scaling proxy. Relative to the semantic-centric alternatives, it:      improves sample efficiency by more than 10x    speeds convergence by up to 7x    maintains better scaling behavior as data increases    That result is important beyond this one architecture. If it continues to hold, it suggests a different recipe for robot foundation models:      use semantics for task understanding    use video dynamics for control-relevant representation learning    5. Experimental Results  The evaluation spans simulation and real-world humanoid deployment.  5.1 LIBERO  On LIBERO, DiT4DiT reaches a new reported state of the art with 98.6% average success.  Per-suite results from Table 1:      Spatial: 98.4%    Object: 99.6%    Goal: 98.6%    Long: 97.6%    The LIBERO-Long result is especially important because it supports the paper’s main claim: modeling future dynamics helps with extended-horizon manipulation, not just short reactive behaviors.  The comparison against the parameter-matched baseline is also clean:      Qwen3DiT: 96.6%    DiT4DiT: 98.6%    So the gain is not just model size. It comes from the representation and training design.  5.2 RoboCasa-GR1  On the 24-task RoboCasa-GR1 tabletop benchmark, DiT4DiT reaches 50.8% average success.  This beats:      GR00T-N1.5: 41.8%    GR00T-N1.6: 40.8%    Qwen3DiT: 36.2%    The gap over Qwen3DiT is particularly notable: 14.6 absolute points. That is a strong signal that the improvement is coming from video dynamics rather than simply swapping one large backbone for another.  The paper also notes that DiT4DiT achieves the best result on 16 of 24 tasks, with especially clear gains on tasks requiring:      precise spatial coordination    articulated object interaction    longer multi-stage execution    5.3 Real-world Unitree G1  The real-world evaluation uses a Unitree G1 humanoid robot across seven household tasks, with only an egocentric camera as visual input.  The paper reports that DiT4DiT consistently outperforms both:      GR00T-N1.5    Qwen3DiT    A qualitative point that stands out is how badly the static-VLM-style baseline transfers: the paper states that Qwen3DiT nearly collapses in the real world and remains below 10% on every task, including 0% on several tasks.  That contrast supports the paper’s broader thesis: video-dynamics pretraining appears to transfer more naturally to physical interaction than static semantic pretraining alone.  6. Generalization  The paper includes both simulated and real-world zero-shot generalization tests.  In simulation, they train on bottle-only tasks and evaluate on unseen objects such as:      can    cup    milk    wine    DiT4DiT substantially outperforms Qwen3DiT under this object substitution setting.  In the real world, they evaluate three kinds of shifts:      category changes    object substitution    quantity variation    Examples include changing the kind of cups or vases, swapping the packed object, and changing the number of cups in the scene.  The high-level takeaway is that DiT4DiT is more robust to surface-level appearance shifts while preserving the underlying physical interaction pattern.  7. Ablations and Efficiency  The ablations are useful because they test exactly where the gains come from.  7.1 Best feature layer is not the last layer  The best action conditioning comes from a middle-to-deep video transformer layer, with layer 18 reported as the best default.  This makes sense: early layers are too local, while the final denoising layers are too specialized for pixel reconstruction.  7.2 One denoising step is enough  A particularly interesting..."
    },{
      "title": "TabletopGen: Instance-Level Interactive 3D Tabletop Scene Generation from Text or Single Image",
      "url": "/posts/2026/03/tabletopgen-paper-notes/",
      "date": "2026-03-11",
      "tags": ["3D Scene Generation","Embodied AI","Tabletop Scenes","Single-Image Reconstruction","Vision-Language Models","Simulation","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  TabletopGen is a training-free pipeline for generating instance-level, simulation-ready 3D tabletop scenes from either text or a single image.  Its core idea is to avoid generating a whole tabletop scene monolithically. Instead, it:      turns text into a reference image when needed    segments and completes each object instance separately    reconstructs each instance into a 3D asset    solves layout recovery with a two-stage alignment module:              DRO for object rotation        TSA for translation and metric scale              This decomposition is what makes the method strong: it preserves object count and style better than retrieval-heavy or whole-scene reconstruction baselines, and it sharply reduces object collisions in dense tabletop layouts.  Paper Info      Title: TabletopGen: Instance-Level Interactive 3D Tabletop Scene Generation from Text or Single Image    Authors: Ziqian Wang, Yonghao He, Licheng Yang, Wei Zou, Hongxuan Ma, Liu Liu, Wei Sui, Yuxin Guo, Hu Su    Affiliations: University of Chinese Academy of Sciences, D-Robotics, Institute of Automation CAS, Horizon Robotics    arXiv: 2512.01204    Project page: d-robotics-ai-lab.github.io/TabletopGen.project    Paper type: 3D scene generation / embodied AI / simulation pipeline    1. Problem Setting and Motivation  The paper focuses on a very practical gap in embodied AI: most existing 3D scene generation systems are not well suited for dense tabletop manipulation scenes.  The authors argue that a useful tabletop scene for robotic simulation should satisfy three conditions:      each object should be an independent, geometrically complete 3D instance    the arrangement should be functionally meaningful, not random    the final scene should be physically plausible, especially collision-free    This is harder than generic room-scale scene generation because tabletops contain:      many small objects packed into a limited area    frequent occlusion in a single image    fine-grained spatial relations that matter for manipulation    The paper’s critique of prior work is straightforward:      retrieval-based methods are limited by fixed asset libraries    text-to-3D scene planning methods are better at coarse layouts than dense small-object tabletop reasoning    single-image 3D scene reconstruction methods struggle with occlusion, incomplete instances, pose errors, and interpenetration    2. Core Idea  TabletopGen uses a unified pipeline for both text input and single-image input.      If the input is text, an LLM first expands it into a detailed prompt, then a text-to-image model generates a realistic reference image.    If the input is already an image, that image is used directly.    From there, the framework runs four stages:      Instance Extraction    Canonical 3D Model Generation    Pose and Scale Alignment    3D Scene Assembly    The most important design choice is the instance-first decomposition: rather than reconstructing the whole scene jointly, the method reconstructs each object separately and only then solves the layout.  3. Method Breakdown  3.1 Instance extraction from the reference image  The pipeline first uses an MLLM to infer open-vocabulary object categories in the scene. It then applies GroundedSAM-v2 to get per-instance masks.  Because tabletop scenes are heavily occluded and object boundaries are often incomplete, the paper does not rely on ordinary inpainting. Instead, it uses a multimodal generative completion model that redraws each segmented object into a clearer, high-resolution instance image.  That step matters because downstream 3D generation quality depends heavily on whether each instance is visually complete.  3.2 Per-instance canonical 3D reconstruction  Each completed object image is passed to an image-to-3D diffusion model to produce a 3D mesh.  These per-instance meshes initially live in arbitrary local coordinates, so the method performs canonical coordinate alignment. An MLLM reasons about upright orientation using both visual evidence and semantic priors, then rotates each model so it is properly aligned with the tabletop world frame.  This is an important practical benefit over retrieval pipelines:      the system is not constrained to a fixed asset database    appearance and geometry can better match the reference    object-level editing becomes easier later    3.3 DRO: Differentiable Rotation Optimizer  The first half of the layout solver is DRO, which estimates each object’s rotation.  The paper renders each candidate object with a differentiable renderer and minimizes a tri-modal matching loss:  L_rot = λ_s L_sil + λ_e L_edge + λ_a L_app  where:      L_sil is a soft-IoU silhouette loss    L_edge is a contour-matching loss based on distance transforms    L_app is a DINOv2 perceptual feature loss    This is a strong part of the paper. Instead of asking a VLM to guess object orientation in one shot, the method uses a render-and-optimize loop that directly compares projected 3D geometry against the target instance.  3.4 TSA: Top-view Spatial Alignment  After recovering rotation, TabletopGen estimates translation and scale with TSA.  This addresses the classic single-view ambiguity problem: from one image, absolute placement and metric size are hard to infer reliably.  The pipeline:      synthesizes a top-view image from the front-view reference    detects top-view 2D boxes for each instance    queries an MLLM for commonsense physical size priors    selects a reliable anchor object using the proposed RMA-Score    The score is:  RMA(i) = A_px(i) / (1 + (ε_ratio / τ)^2)  where larger visible objects with better aspect-ratio consistency are preferred as anchors.  Conceptually, TSA is doing a useful compromise:      use generative and language models for semantic spatial reasoning    constrain the final estimate with explicit geometric heuristics    3.5 Scene assembly  Once rotation, translation, and scale are estimated, the pipeline imports all instances into Isaac Sim, applies transforms, and assigns collision properties through convex decomposition. This converts visually reconstructed objects into a simulation-ready interactive tabletop scene.  4. Experimental Results  The evaluation covers 78 test samples with different table shapes and tabletop categories, including office, dining, workbench, and more stylized scenes.  The baselines are:      ACDC (retrieval-based)    Gen3DSR    MIDI    4.1 Quantitative gains  TabletopGen reports the best numbers across perceptual, semantic, and physical metrics.  Some headline results from Table 1:      LPIPS: 0.4483 vs 0.4559 for MIDI    DINOv2: 0.8383 vs 0.7070 for MIDI    CLIP: 0.9077 vs 0.8867 for MIDI    object collision rate: 0.42% vs 17.39% for MIDI    scene collision rate: 7.69% vs 98.72% for MIDI    The collision numbers are the most convincing part of the paper. Many prior methods can produce something visually plausible from the reference view, but collision-free assembly is what actually determines whether a scene is usable for embodied simulation.  4.2 GPT and human evaluation  On GPT-4o-based scoring, the method gets the best average score (6.19) across visual fidelity, image alignment, and physical plausibility.  The user study with 128 participants is also strong:      average human score: 5.56    second-best baseline: 3.57    overall preference for TabletopGen: 83.13%    That large margin suggests the gains are not limited to one metric choice.  4.3 Ablations  The ablation study is clean and useful because it isolates the two key geometric modules:      removing DRO increases object collision rate from 0.42% to 1.27%    removing TSA raises it much more sharply to 5.50%    removing both yields severe placement failures and 62.82% scene-level collision rate    My reading is that TSA contributes most of the physical plausibility gain, while DRO stabilizes orientation quality and improves visual consistency.  5. What I Find Most Interesting  The paper’s strongest contribution is not just another better benchmark score. It is the claim that tabletop generation should be treated as a compositional geometry-and-reasoning problem, not only as a big generative modeling problem.  Three aspects stand out:      instance-first reconstruction is a better fit for manipulation scenes than whole-scene synthesis    explicit geometric optimization is still necessary, even in an era of strong multimodal models    physical plausibility is treated as a first-class target instead of an afterthought    This makes the work particularly relevant for simulation data generation, robot benchmarking, and sim-to-real pipelines.  6. Strengths      Clear focus on tabletop scenes, which are genuinely important for embodied manipulation.    Good systems design: each stage solves a specific bottleneck rather than forcing one model to do everything.    Strong physical-plausibility results, especiall..."
    },{
      "title": "Towards Human-Like Manipulation through RL-Augmented Teleoperation and Mixture-of-Dexterous-Experts VLA",
      "url": "/posts/2026/03/mode-vla-paper-notes/",
      "date": "2026-03-11",
      "tags": ["Robotics","Vision-Language-Action","Dexterous Manipulation","Bimanual Manipulation","Teleoperation","Tactile Sensing","Reinforcement Learning","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  This paper pushes VLA-style robot control toward human-like bimanual dexterous manipulation. The core idea is to split the problem into two parts:      IMCopilot: RL-trained in-hand manipulation skills that both assist humans during teleoperation and act as callable low-level skills during inference    MoDE-VLA: a VLA architecture that injects force and tactile feedback into a pretrained backbone through a dedicated sparse-expert pathway    The result is a system that can handle harder contact-rich tasks such as gear assembly, charger plugging, tube rearranging, and apple peeling, with a clear gain over the baseline pi_0 backbone.  Paper Info      Title: Towards Human-Like Manipulation through RL-Augmented Teleoperation and Mixture-of-Dexterous-Experts VLA    Authors: Tutian Tang, Xingyu Ji, Wanli Xing, Ce Hao, Wenqiang Xu, Lin Shao, Cewu Lu, Qiaojun Yu, Jiangmiao Pang, Kaifeng Zhang    arXiv: 2603.08122    Topic: dexterous manipulation, multimodal VLA, force/tactile fusion, shared-autonomy teleoperation    1. Motivation  Most successful VLAs still operate in a relatively easy regime:      low-DoF grippers    visually guided actions    limited contact reasoning    This paper focuses on the harder setting of 63-DoF bimanual dexterous manipulation, where the robot must coordinate:      visual perception    language-conditioned task execution    arm torques and contact forces    fingertip tactile feedback    in-hand object rotation and grasp stabilization    The authors argue that current VLA pipelines break down here for three reasons:      Data collection is too hard: direct teleoperation of high-DoF bimanual systems is cognitively demanding.    One policy struggles to cover all skills: gross motion, insertion, peeling, and in-hand rotation are qualitatively different.    Force/tactile inputs are not easy to fuse: naive concatenation can hurt a pretrained VLA rather than help it.    2. Core Idea  The framework combines two complementary components:  2.1 IMCopilot  IMCopilot is a set of RL-trained atomic in-hand manipulation skills, especially:      stable grasp maintenance    in-hand object rotation around a target axis    It serves two roles:      during teleoperation: the operator controls gross motion and triggers IMCopilot through foot pedals for difficult in-hand phases    during inference: the high-level VLA emits a trigger signal, and IMCopilot takes over hand-level control when needed    This is a practical shared-autonomy design. Instead of forcing the human or the VLA to solve every dexterous subproblem directly, the system delegates the hardest contact-rich finger coordination to a specialized low-level controller.  2.2 MoDE-VLA  MoDE-VLA stands for Mixture-of-Dexterous-Experts VLA. It extends a pretrained VLA backbone with a modality-specific branch for:      force: arm joint torque readings    tactile: 6-DoF force/wrench readings from ten fingertip sensors    The design has three important pieces:      Dedicated pathway for force and tactile tokens instead of naive state concatenation    Sparse MoE routing so different experts can specialize to different contact regimes    Residual injection so multimodal corrections refine the pretrained action prediction rather than overwrite it    This is the architectural part I found most convincing. The paper is not just saying “more sensors help”; it explains why these sensors should be routed differently because arm-level torques and fingertip contact patterns carry different physical meanings.  3. Method Details  3.1 Platform and sensing  The robot platform includes:      dual 7-DoF arms    dual 22-DoF dexterous hands    fingertip tactile sensors on all ten fingers    stereo head cameras and wrist cameras    The data collection setup uses:      upper-body exoskeleton    exoskeleton gloves    VR headset    force/tactile visualization in VR    vibrotactile fingertip feedback    This makes the teleoperation system much richer than vision-only data collection.  3.2 RL training for IMCopilot  IMCopilot skills are trained in simulation with PPO and teacher-student distillation.  The policy observes:      short proprioceptive history    fingertip contact forces    target rotation axis    The reward encourages:      rotation around the desired axis    low unwanted linear motion    low torque and joint work    stable motion    The high-level takeaway is simple: the paper isolates in-hand dexterity as a reusable skill module instead of expecting the VLA to learn everything end-to-end from limited demonstrations.  3.3 MoDE-VLA action generation  The base VLA is built on a pretrained pi_0-style flow-matching backbone. MoDE adds force and tactile tokens, lets them interact with the backbone through self-attention, routes them through sparse experts, and then generates residual corrections.  The paper uses:      E = 8 experts    top-k = 1 routing    action horizon H = 50    N = 10 Euler denoising steps at inference    The action vector contains:      arm actions    hand actions    other actions including waist motion    an IMCopilot trigger scalar    When the trigger is active, hand actions are delegated to IMCopilot.  4. Experiments and Main Results  The paper evaluates four contact-rich tasks:      Apple Peeling    Tube Rearranging    Gear Assembling    Charger Plugging    All methods are evaluated over 20 trials per task.  4.1 Teleoperation benefits from force/tactile feedback  The paper reports that force/tactile feedback improves demonstration quality and collection efficiency. One example given is Gear Assembling:      without feedback: 100 trials in 75 minutes, 85 successful demonstrations    with feedback: 100 trials in 65 minutes, 93 successful demonstrations    That is a practical result: multimodal sensing helps before learning even starts.  4.2 IMCopilot strongly improves in-hand rotation  For in-hand manipulation, plain teleoperation is much weaker than IMCopilot:      Ping-pong ball: 10% -> 83%    Tennis ball: 67% -> 93%    Apple: 27% -> 90%    Overall: 34% -> 89%    This is one of the clearest findings in the paper. The authors are not just using RL as a benchmark skill; they show it directly fixes a bottleneck in data acquisition.  4.3 MoDE-VLA vs. baseline  Compared with the pretrained backbone pi_0, the proposed method improves average success rate from 15% to 34% across the four tasks.  Task-level results:      Apple Peeling: task failure for baseline, proposed method reaches 30% SR and 73% peel completion ratio    Tube Rearranging: 8% -> 30%    Gear Assembling: 40% -> 60%    Charger Plugging: 5% -> 15%    The absolute numbers are still modest, especially for the hardest tasks, but the direction is consistent: contact-aware sensing plus skill hierarchy helps.  4.4 Ablations  The ablations show each component matters:      without force: average SR drops to 23%    without tactile: average SR drops to 26%    without IMCopilot: apple peeling PCR drops from 73% to 25%    Interpretation:      force matters most for insertion and contact onset    tactile helps with slip-sensitive hand interactions    IMCopilot is crucial for the peel-and-rotate loop    5. Why This Paper Is Interesting  I think the strongest aspect of the paper is its systems framing.  A lot of VLA work assumes that scaling a single end-to-end policy is enough. This paper takes a different position:      use teleoperation, but augment it with autonomy    use a pretrained VLA, but refine it with modality-aware residual experts    use end-to-end action generation, but keep a specialized low-level controller for in-hand dexterity    That feels much closer to how capable robotic systems will likely be built in practice.  6. Limitations  A few limitations are worth keeping in mind:      the final success rates are still not high enough for robust deployment    evaluation covers only four tasks    the system depends on specialized hardware: dexterous hands, tactile sensors, exoskeletons, and VR teleoperation    IMCopilot currently focuses on a small set of atomic in-hand skills rather than a broad manipulation library    So this is better viewed as a strong research prototype than a general-purpose deployment recipe.  7. My Takeaways      Shared autonomy is a good data strategy for dexterous manipulation. Humans do not need to control every fine contact event manually.    Force and tactile signals should not be fused naively into a pretrained VLA. The modality-specific residual path is a sensible design.    Hierarchical skill invocation is probably necessary for long-horizon dexterous tasks such as peeling, tool use, and regrasping.    The paper is especially relevant if you care about the next step beyond simple gripper-based VLA benchmarks.    这篇文章支持通过网站顶部语言切换按钮在 English / 中文 之间切换。  TL;DR  这篇论文尝试把 VLA 风格的机器人控制推进到更接近 人类双手灵巧操作 的阶段。核心思路是把问题拆成两部分：      IMCopilot：一组通过强化学习训练得到的手内操作技能，既能在遥操作时辅助人..."
    },{
      "title": "UltraDexGrasp: Learning Universal Dexterous Grasping for Bimanual Robots with Synthetic Data",
      "url": "/posts/2026/03/ultradexgrasp-paper-notes/",
      "date": "2026-03-09",
      "tags": ["Robotics","Dexterous Grasping","Bimanual Manipulation","Synthetic Data","Sim2Real","Point Clouds","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  UltraDexGrasp tackles a missing piece in dexterous manipulation: universal grasping for bimanual robots across multiple grasp strategies. Instead of focusing only on one hand or one grasp type, the paper builds a synthetic-data pipeline that supports:      two-finger pinch    three-finger tripod    whole-hand grasp    bimanual grasp    Using this pipeline, the authors build UltraDexGrasp-20M, a 20-million-frame dataset over 1,000 objects, and train a point-cloud policy that achieves 84.0% average success in simulation and 81.2% average success in real-world zero-shot sim-to-real grasping.  Paper Info      Title: UltraDexGrasp: Learning Universal Dexterous Grasping for Bimanual Robots with Synthetic Data    Authors: Sizhe Yang, Yiman Xie, Zhixuan Liang, Yang Tian, Jia Zeng, Dahua Lin, Jiangmiao Pang    Affiliations: Shanghai AI Laboratory, CUHK, Zhejiang University, HKU, Peking University    Project page: yangsizhe.github.io/ultradexgrasp    arXiv: 2603.05312    Code: UltraDexGrasp GitHub    1. Motivation  The paper starts from a clear observation: human grasping is naturally strategy-dependent.      small objects are often handled with pinch or tripod grasps    medium objects can be grasped with one full hand    large or heavy objects often require both hands    Current robotic dexterous grasping work usually does not cover this full space. Most prior work is limited to:      parallel grippers    single dexterous hands    one grasp style at a time    For bimanual dexterous robots, the main bottleneck is data. The paper argues that generating high-quality universal grasp data is hard because we need:      physically plausible contact    good geometric conformity    arm-level kinematic feasibility    dual-arm coordination    multiple grasp strategies for different object regimes    That is the niche UltraDexGrasp is trying to fill.  2. Core Idea  The key contribution is not just a policy. It is a data generation framework that combines:      optimization-based grasp synthesis    planning-based demonstration generation    This lets the system first find feasible grasp poses, then convert them into closed-loop, coordinated arm-hand trajectories that can be executed and filtered in simulation.  The output is a large-scale multi-strategy dataset, which is then used to train a universal grasp policy.  3. Data Generation Pipeline  3.1 Optimization-based grasp synthesis  For a given object and robot, the system first synthesizes candidate bimanual grasps by optimizing:      hand pose    finger joint positions    contact forces    under constraints such as:      forward kinematics    joint limits    friction cone feasibility    hand-object collision avoidance    hand-hand collision avoidance    The formulation is shared across grasp strategies; the main difference between pinch, tripod, whole-hand, and bimanual grasp is which hand contact points are activated.  For each object, the method generates 500 candidate grasps, then filters them using:      physical plausibility checks    inverse-kinematics reachability    collision checking    Finally, it selects a preferred grasp based on the shortest SE(3) distance from the current end-effector pose, which makes execution easier and more natural.  3.2 Planning-based demonstration generation  Once the preferred grasp is selected, the whole grasping process is split into four stages:      pregrasp    grasp    squeeze    lift    Bimanual motion planning is used to produce collision-free coordinated trajectories. In simulation, the robots execute the planned motions with PD control, and the trajectory is kept only if the object is stably lifted.  The success condition is fairly concrete:      the object must rise at least 0.17 m    it must stay elevated for at least 1 second    This is a useful design choice because it turns grasp data generation into a physically validated process, not only a kinematic one.  3.3 Dataset scale  Using this process, the authors build UltraDexGrasp-20M:      20 million frames    1,000 objects    multiple grasp strategies    The paper also notes that simulated rendering includes an imaged robot point cloud, which helps reduce the sim-to-real gap because at deployment time the robot’s own geometry is known.  4. Policy Design  The grasp policy is intentionally simple.      input: scene point cloud    encoder: PointNet++-style point encoder    aggregator: decoder-only transformer with unidirectional attention    output: arm and hand control commands    The point cloud is first downsampled to 2,048 points, then encoded with two set-abstraction layers.  Two design choices seem especially important in the paper:      the unidirectional attention mechanism for aggregating scene features    bounded Gaussian distribution prediction for actions    The authors emphasize that the policy is meant to be simple and clean, so the paper’s gains should be interpreted mainly as evidence that the data pipeline is strong enough to support universal grasping.  5. Main Results  5.1 Simulation benchmark  The simulation benchmark evaluates grasping on 600 objects, split by size:      small    medium    large    Results against DP3 and DexGraspNet are strong:      DP3: 46.7 average success    DexGraspNet: 58.8 average success    UltraDexGrasp policy: 84.0 average success    More specifically:      seen small: 78.8    seen medium: 84.3    seen large: 90.4    unseen small: 76.9    unseen medium: 85.8    unseen large: 87.5    The average performance on unseen objects is about 83.4%, which is the main generalization result.  An important comparison is that DexGraspNet cannot handle large objects in this setup because it only synthesizes unimanual grasps. That highlights why multi-strategy bimanual data matters.  5.2 Data scaling and policy quality  The paper notes that the raw data-generation pipeline itself has a grasping success rate of 68.5%, but once the policy is trained on more than 1M frames, policy performance significantly exceeds the generator.  That is a nice result: the learned policy is not just imitating noisy demonstrations; it is actually distilling and improving on the large synthetic dataset.  5.3 Ablation study  The ablations show the policy architecture is not arbitrary:      without bounded distribution prediction: 73.5%    without unidirectional attention: 68.2%    full model: 84.0%    So both design choices contribute materially, and the attention design appears particularly important.  5.4 Real-world results  The real-world setup uses:      two UR5e robots    two 12-DoF XHands    two Azure Kinect DK cameras    The policy is tested on 25 real objects across small, medium, and large categories.  Reported real-world results:      DP3: 46.7    DexGraspNet: 62.3    UltraDexGrasp policy: 81.2    By object size:      small: 72.0    medium: 82.2    large: 89.3    These are strong numbers for direct zero-shot sim-to-real deployment, especially since the policy is trained only on synthetic data.  6. Why This Paper Matters  I think the most valuable aspect of the paper is its problem scope.  A lot of dexterous grasping work asks:      can one hand grasp many objects?    This paper instead asks:      can a robot choose among multiple dexterous grasp strategies, including bimanual ones, using one training framework?    That is much closer to the real-world version of grasping.  The second reason the paper matters is that it shows a realistic pipeline for scaling data:      synthesis for diverse contact-rich grasps    planning for executable trajectories    simulation filtering for physical validity    policy learning for generalization and speed    7. Strengths      Strong focus on a genuinely underexplored setting: universal bimanual dexterous grasping.    The dataset-generation pipeline is concrete, scalable, and physically grounded.    Supports multiple grasp strategies instead of a single grasp mode.    Strong simulation and real-world results with synthetic-data-only training.    Clear evidence that the learned policy outperforms both the raw generator and strong baselines.    8. Limitations and Open Questions      The paper focuses on grasping, not the subsequent manipulation after grasp acquisition.    The evaluation is still mostly object lifting; more task-oriented or functional grasp benchmarks would be useful.    The policy is trained on point clouds with known robot geometry, so robustness to worse sensing conditions remains unclear.    The architecture is relatively task-specific; it is not obvious yet whether the same setup scales to more general dexterous manipulation beyond grasping.    The object set is broad, but it would be interesting to see more articulated, deformable, or cluttered scenarios.    9. Takeaways  My main takeaway is that UltraDexGrasp makes a strong case that multi-strategy dexterous grasping can be learned from synthetic data alone, provided the ..."
    },{
      "title": "One Hand to Rule Them All: Canonical Representations for Unified Dexterous Manipulation",
      "url": "/posts/2026/03/one-hand-to-rule-them-all-paper-notes/",
      "date": "2026-03-09",
      "tags": ["Robotics","Dexterous Manipulation","Cross-Embodiment Learning","Grasping","Hand Morphology","Sim2Real","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  This paper asks a simple but important question: can we train dexterous manipulation policies that are not tied to one specific robot hand?  Their answer is a canonical hand representation that maps many dexterous hands into:      a shared morphology parameter space    a shared canonical URDF    a shared action space    This lets a policy condition on hand morphology and transfer across embodiments. The paper shows cross-hand grasp learning, smooth latent interpolation over hand designs, and strong zero-shot generalization to unseen LEAP Hand variants, including 81.9% zero-shot success on an unseen 3-finger LEAP Hand.  Paper Info      Title: One Hand to Rule Them All: Canonical Representations for Unified Dexterous Manipulation    Authors: Zhenyu Wei, Yunchao Yao, Mingyu Ding    Affiliations: University of North Carolina at Chapel Hill    Project page: zhenyuwei2003.github.io/OHRA    Paper type: dexterous manipulation / cross-embodiment representation paper    1. Motivation  Most dexterous manipulation work still assumes a fixed robot hand.  That causes two major problems:      policies trained on one hand do not transfer well to other hands with different finger numbers, joint layouts, or kinematics    datasets collected for different hands cannot be easily pooled for shared learning    This is a real bottleneck, because dexterous hands differ a lot:      3-finger, 4-finger, and 5-finger designs    different DoF counts    different joint orders and axis conventions    different URDF coordinate systems and kinematic trees    The paper’s core claim is that we need a representation-level solution before we can get scalable cross-hand learning.  2. Core Idea  The method introduces a canonical representation for dexterous hands with two linked pieces:      a canonical parameterization that describes morphology and kinematics in a learning-friendly vector form    a canonical URDF that standardizes embodiment-specific action spaces into one unified control interface    In the base version, the hand is represented by 82 parameters. The canonical URDF supports up to:      5 fingers    22 DoF    Inactive joints are simply treated as dummy variables, so different hand embodiments can still share one action space.  That is the central design choice: instead of learning separate policies for each hand, learn one policy in a unified embodiment space and condition it on the hand’s canonical description.  3. Method Breakdown  3.1 Canonical URDF  The canonical URDF captures a human-inspired hand topology while enforcing consistent coordinate conventions.  Important design choices include:      using a unified right/left-hand kinematic convention    modeling links with capsule primitives to simplify geometry while preserving essential structure    standardizing joint axes and frame definitions across hands    This matters because raw URDFs are too heterogeneous for direct learning. Even two similar hands may use incompatible global or local axis conventions, which makes direct parameter sharing difficult.  3.2 Canonical parameter space  The canonical representation encodes:      palm geometry    finger geometry    finger origins    thumb orientation    joint axes    joint availability / presence    The paper also provides an extended 173-parameter version for higher-fidelity modeling, but the main experiments use the more compact 82-parameter design.  The point is not to exactly preserve every URDF implementation detail. The point is to preserve the key geometric and kinematic features that matter for cross-embodiment learning.  3.3 Unified action space  The canonical URDF gives every hand a shared control interface. A policy outputs actions in the canonical space, and those actions can then be interpreted on different hands according to which joints are active.  This is arguably the most practical contribution in the paper, because it turns “cross-hand transfer” from an abstract representation problem into a policy-learning problem that standard neural networks can actually use.  3.4 Latent hand morphology learning  The authors train a VAE on synthetic hand samples generated from the canonical parameter space.  Key details:      65,536 synthetic hand configurations are sampled    the VAE maps canonical hand parameters into a 16-dimensional latent space    The learned latent space is structured enough that interpolation between two hand embodiments yields smooth transitions in:      finger number    finger spacing    palm size    overall morphology    This is a useful sanity check that the canonical space is not just valid symbolically, but also geometrically meaningful for learning.  4. Experiments  The paper evaluates four things:      latent-space structure    physical fidelity of the canonical hand model    cross-embodiment grasp learning    zero-shot transfer to unseen hand morphologies    4.1 VAE latent interpolation  The latent interpolation figures show smooth transitions between very different hand designs, such as a compact 3-finger hand and a more anthropomorphic high-DoF hand.  This is a strong qualitative result because it suggests the representation captures a continuous morphology manifold rather than a bag of disconnected hand templates.  4.2 In-hand reorientation  To test whether the canonical URDF preserves useful dynamics, the paper compares RL policies trained on:      original hand URDFs    canonical versions of those hands    They evaluate on Shadow Hand and LEAP Hand using an in-hand rotation task. Reported results show the canonical version performs comparably to, and in some cases slightly better than, the original version:      Shadow (Original): 369.66 steps-to-fall, 9.09 cumulative rotation    Shadow (Canonical): 390.62 steps-to-fall, 10.92 cumulative rotation    LEAP (Original): 397.62 steps-to-fall, 5.63 cumulative rotation    LEAP (Canonical): similar performance trend with no obvious fidelity collapse    This matters because a unified URDF is only useful if it preserves enough physics and kinematics for control.  4.3 Cross-embodiment grasping  The grasping experiments use grasps from three very different dexterous hands:      Allegro    Barrett    Shadow Hand    The policy is trained in the canonical representation and compared with baselines as well as embodiment-specific training.  A few important findings:      unified training outperforms embodiment-specific training across all three hands    the lightweight canonical grasp model runs very fast, about 0.13 s inference time    performance is competitive with stronger grasp pipelines despite using a relatively simple model    Reported unified-vs-specific results:      Allegro: 84.2 vs 82.1    Barrett: 88.1 vs 87.6    Shadow Hand: 62.9 vs 55.4    That Shadow Hand gain is especially notable because it suggests the shared embodiment space lets harder hands benefit from data from easier ones.  4.4 Zero-shot transfer to unseen LEAP Hand variants  This is the most interesting experiment in the paper.  The authors create a large family of LEAP Hand variants by removing links from fingers, yielding many different morphologies. They train on a subset and test on unseen variants.  Main takeaway:      the policy can generalize zero-shot to previously unseen hand morphologies    one highlighted result is 81.9% zero-shot success on an unseen 3-finger LEAP Hand variant    The paper also shows that explicit hand conditioning is crucial. If the wrong hand condition is used, performance drops sharply, especially in zero-shot settings.  4.5 Real-world results  The real-world evaluation is run on a Franka arm with different LEAP Hand variants over 10 objects.  Reported average grasp success:      leap_3333 (trained): 83/100    leap_3033 (trained): 75/100    leap_3033 (zero-shot): 71/100    leap_3303 (trained): 70/100    leap_3303 (zero-shot): 71/100    The key point is that zero-shot policies are close to the trained ones, which is strong evidence that the morphology-conditioned policy is doing meaningful cross-hand generalization rather than memorizing one embodiment.  5. Why This Paper Matters  I think this paper is important because it shifts cross-embodiment dexterous manipulation from “can we transfer grasps between a few hands?” to “can we define a shared embodiment language for many hands?”  That framing matters because:      it scales better than hand-specific pipelines    it makes heterogeneous grasp data reusable    it opens the door to universal dexterous manipulation policies that can adapt to new hand hardware    The representation is the real contribution here. The grasp model itself is intentionally simple; the point is to show the representation is strong enough that even a simple policy can work well across embodiments.  6. Strengths      Very clear and useful problem framing around cross-embodiment dexterous learning.    Canonical URDF plus canonical paramet..."
    },{
      "title": "MEM: Multi-Scale Embodied Memory for Vision Language Action Models",
      "url": "/posts/2026/03/mem-paper-notes/",
      "date": "2026-03-09",
      "tags": ["Robotics","Vision-Language-Action","Memory","Long-Horizon Control","Embodied AI","Manipulation","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  MEM adds explicit memory to Vision-Language-Action models by splitting memory into two parts:      short-term video memory for recent visual context, self-occlusion handling, and quick strategy adaptation    long-term language memory for compact semantic summaries of what has already happened in a long task    The paper’s main claim is that end-to-end robot policies need multi-scale memory, not just a longer observation window. MEM lets a VLA solve tasks that require memory over up to 15 minutes, while still respecting real-time inference constraints.  Paper Info      Title: MEM: Multi-Scale Embodied Memory for Vision Language Action Models    Authors: Marcel Torne, Karl Pertsch, Homer Walke, Kyle Vedder, Suraj Nair, Brian Ichter, Allen Z. Ren, Haohuan Wang, Jiaming Tang, Kyle Stachowicz, Karan Dhabalia, Michael Equi, Quan Vuong, Jost Tobias Springenberg, Sergey Levine, Chelsea Finn, Danny Driess    Affiliations: Physical Intelligence, Stanford University, UC Berkeley, MIT    Project page: pi.website/research/memory    Paper type: robotics/VLA systems paper    1. Motivation  The paper starts from a gap in current VLAs:      many strong VLAs act only on the current observation    some memory-augmented methods simply append a dense history of past observations    that approach becomes expensive and still does not distinguish between fine-grained short-term context and abstract long-term task state    For real robot tasks, these two memory types matter for different reasons:      short-term memory helps with self-occlusion, tracking recent failures, and adjusting manipulation online    long-term memory helps remember semantic progress, such as which ingredients have already been collected or which subtasks are done    The authors argue that a single uniform memory mechanism is a poor fit for both.  2. Core Idea of MEM  MEM splits action generation into a low-level and high-level process.      The low-level policy uses a short horizon of dense observations plus a subtask instruction.    The high-level policy predicts both the next subtask instruction and an updated language memory summarizing past semantic events.    This gives the model a mixed-modal memory system:      video memory for recent detailed visual context    language memory for long-horizon compressed task history    That design is the real contribution of the paper. It is not just “more context”; it is a structured memory interface matched to different timescales.  3. Method Breakdown  3.1 Language memory for long-horizon context  The language memory is a running semantic summary of what has already happened in the episode. Instead of passing all previous observations or all previous subtasks verbatim, the model updates a compact textual state over time.  This is important because:      it compresses long-horizon history much better than raw images    it keeps the model focused on semantically relevant events    it avoids exploding context length for tasks that span many minutes    The paper also shows that a naive alternative, simply concatenating previous subtask instructions, works noticeably worse because training-time demonstrations rarely contain repeated failures, while inference-time rollouts often do.  3.2 Video encoder for short-horizon context  For short-term memory, MEM uses a video encoder that extends a ViT-style image encoder into a video encoder by interleaving:      spatial attention within each frame    causal-temporal attention across frames    An important engineering detail is that this encoder:      compresses time before passing features into the VLA backbone    preserves the token count seen by the backbone at roughly the single-frame level    introduces no new learnable parameters compared with the standard image ViT    That makes it possible to initialize from pre-trained VLM weights while keeping inference latency under real-time constraints.  3.3 Integrating MEM into pi0.6  The paper instantiates MEM inside the pi0.6 VLA. During pre-training, the model sees sequences of six observations with one-second stride. During post-training, the observation horizon can be expanded significantly, up to around one minute for observation-based memory, while the language memory covers much longer horizons.  The broader point is that the memory capability is developed during large-scale pre-training, not bolted on only at the end.  4. Main Results  4.1 Long-horizon manipulation tasks  The headline result is that MEM enables long-horizon tasks such as:      recipe setup    kitchen cleanup    grilled cheese preparation    These tasks require maintaining task-relevant memory over up to fifteen minutes. The paper shows that memoryless pi0.6 struggles badly on them, while the full MEM system makes them much more tractable.  The ablations are especially informative:      removing video memory hurts tasks that require recent timing, occlusion handling, or adaptation    removing language memory hurts tasks that require long-range semantic progress tracking    using naive text memory without compression performs much worse than learned semantic summaries    This supports the paper’s central claim that both memory scales are necessary.  4.2 In-context adaptation  One of the most interesting parts of the paper is that memory is not only for remembering task progress. It also enables in-context adaptation of manipulation strategy.  The authors demonstrate this on tasks like:      picking up a chopstick at an out-of-distribution table height    opening a refrigerator when the door-opening direction is ambiguous    With memory, the policy can use recent failed attempts as context and switch strategy. The reported gains are substantial:      about +11% on chopstick pickup    about +62% on fridge opening    The memoryless baseline cannot adapt as effectively because it cannot explicitly condition on what was just tried and failed.  4.3 Core memory capability benchmarks  The paper also evaluates memory-intensive skills including:      partial observability    counting    timing    spatial memory    Example tasks include finding an object hidden in one of four drawers, unpacking groceries without forgetting items inside the bag, counting coffee scoops, cooking for the right duration, and remembering which parts of a window have already been cleaned.  Across these tasks, MEM is reported as the only approach that performs strongly across the full range of memory demands, outperforming:      no-memory VLAs    pooled observation-memory baselines    proprio-only memory baselines    4.4 Memory without sacrificing standard manipulation  A useful systems result is that MEM does not just improve memory-heavy tasks. The paper also shows it remains competitive on manipulation tasks that do not require memory, suggesting the memory machinery does not degrade core dexterous control.  5. Why This Paper Matters  I think the most important idea here is conceptual: robot memory should be structured by abstraction level and timescale.  That is more convincing than the simpler “just add more frames” strategy for three reasons:      recent visual context and long-term semantic state are fundamentally different objects    runtime constraints matter a lot for real robot control    long-horizon deployment requires compression, not only larger context windows    In that sense, MEM feels closer to a practical robot architecture than a pure sequence-model scaling exercise.  6. Strengths      Clear separation between short-term and long-term memory roles.    Strong systems focus: the method is designed around real-time latency constraints, not only benchmark accuracy.    Good ablation story showing why both memory modalities matter.    The in-context adaptation result is more interesting than standard memory benchmarks because it shows memory changing behavior, not just recall.    The paper evaluates a fairly broad set of embodied tasks across different memory requirements.    7. Limitations and Open Questions      The language memory still depends on the model learning useful semantic compression; it is not guaranteed to preserve every critical detail.    The approach is integrated into a large proprietary-style VLA stack (pi0.6), so reproducibility and accessibility may be limited compared with fully open systems.    It is still episodic memory. The paper explicitly frames longer-term deployment memory across days, weeks, or months as future work.    It remains unclear how robust the memory summaries are under very long failure chains or heavily out-of-distribution task structures.    The paper demonstrates strong system behavior, but it is harder to isolate how much comes from the memory design itself versus the scale of the underlying training setup.    8. Takeaways  My main takeaway is that MEM gives a strong recipe for long-horizon robot control:      keep recent observations in a compact ..."
    },{
      "title": "LESSMIMIC: Long-Horizon Humanoid Interaction with Unified Distance Field Representations",
      "url": "/posts/2026/03/lessmimic-paper-notes/",
      "date": "2026-03-09",
      "tags": ["Robotics","Humanoid","Long-Horizon Control","Distance Fields","Reinforcement Learning","Sim2Real","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  LESSMIMIC proposes a unified representation for humanoid-object interaction based on distance fields (DFs). Instead of conditioning a humanoid policy on motion references or crafting task-specific rewards per skill, the paper represents interaction through:      distance to the object surface    distance-field gradients    velocity components decomposed into surface-normal and tangential directions    The result is a single whole-body policy that can generalize across object geometry and scale, recover from failures, and compose multiple skills over long horizons. The paper’s central argument is that a good interaction representation matters as much as the control algorithm.  Paper Info      Title: LESSMIMIC: Long-Horizon Humanoid Interaction with Unified Distance Field Representations    Authors: Yutang Lin, Jieming Cui, Yixuan Li, Baoxiong Jia, Yixin Zhu, Siyuan Huang    Affiliations: Peking University, BIGAI, Beijing Institute of Technology    Project page: lessmimic.github.io    arXiv: 2602.21723    1. Motivation  The paper targets a familiar limitation in humanoid manipulation:      reference-based methods can produce high-quality motions, but they are tightly coupled to the geometry and trajectories seen in demonstrations    reference-free methods are more flexible, but usually rely on task-specific rewards and end up as isolated single-skill policies    So the real question becomes: what interaction representation would let a humanoid:      act without reference motions at inference time    generalize to new object shapes and scales    compose different interaction skills inside one policy    recover when the world deviates from the nominal script    The authors argue that distance fields provide exactly that interface.  2. Core Idea  The key move in LESSMIMIC is to represent humanoid-object interaction in a geometry-aware but motion-agnostic way.  For each humanoid link position relative to the object, the distance field provides:      the distance to the nearest surface    the local surface gradient / normal    a way to decompose link velocity into:              normal motion: approach / push / apply force        tangential motion: slide / traverse along the surface              This representation is meant to capture the structure of interaction rather than memorizing a particular reference trajectory.  That is the paper’s main conceptual contribution. Instead of telling the humanoid “follow this motion,” it tells the humanoid “reason about how your body is moving relative to the object’s geometry.”  3. Method Overview  3.1 DF-based interaction representation  At the representation level, the policy observes geometric cues derived from the object’s distance field. The paper emphasizes that absolute coordinates alone are not enough, because they do not tell the agent whether it is approaching, pressing, sliding, or disengaging from the object.  So the method uses:      DF values    DF gradients    velocity decomposition into normal and tangential parts    These interaction features over time are encoded by a VAE into an interaction latent z_t, which smooths the signal and provides a compact policy input.  3.2 Three-stage training pipeline  The training pipeline has three stages:            Interaction skill pre-trainingA teacher pi_mimic tracks retargeted human motions with physics-aware residual compensation. A student pi_base is then trained via DAgger-style behavior cloning, but crucially without access to reference motions at inference.              Discriminative post-trainingThe student is fine-tuned with RL under geometry randomization. Instead of reference tracking rewards, the policy is guided by Adversarial Interaction Priors (AIP), which regularize interaction validity in the DF latent space.              Visual-motor distillationA MoCap-conditioned full policy is distilled into a vision-based policy that uses egocentric depth only, so the system can deploy without motion-capture infrastructure.        This is a clean design: mimic for physical feasibility, RL for geometric generalization, and distillation for deployability.  4. Why Distance Fields Help  The value of DFs here is not just that they encode geometry. The representation has three useful properties:      it is local, so it stays meaningful across different object shapes    it is continuous and differentiable, so it provides smooth geometric feedback    it is task-agnostic, so the same interface can support pushing, pickup, carrying, and sitting interactions    That last point is the most important one. LESSMIMIC is trying to build a shared language for heterogeneous humanoid-object interactions.  5. Main Results  5.1 Generalization across object scale and shape  The paper evaluates four interaction tasks under object scale variation:      PickUp    SitStand    Push    Carry    The training scale is 1.0x, while evaluation ranges from 0.4x to 1.6x. For pickup, the paper also varies object shape across boxes, cylinders, and spheres.  Key takeaways:      reference-based baselines degrade sharply once object scale shifts away from the reference motion setup    LESSMIMIC remains much more stable across scales    the MoCap-conditioned full model reports 80-100% success on several scale-generalization settings for PickUp and SitStand    the policy generalizes in the real world from trained box-like objects to a soccer ball, which is a good qualitative sign that the policy is using geometry rather than memorized motion patterns    This is the strongest result in the paper: the representation seems to support geometry generalization better than motion-conditioned baselines.  5.2 Long-horizon skill composition  The paper then tests whether one policy can handle multiple heterogeneous tasks in sequence without environment resets.  They evaluate randomly ordered task compositions of length:      N = 5    N = 10    N = 15    N = 25    N = 40    The full MoCap-based LESSMIMIC policy reports:      61.7% success at N=5    38.1% at N=10    23.5% at N=15    9.0% at N=25    2.1% at N=40    Those numbers are not huge at the longest horizon, but they matter because the ablated variants collapse to nearly zero much earlier. The result suggests the unified DF representation actually helps preserve cross-skill consistency over long execution chains.  5.3 Failure recovery  One of the paper’s qualitative claims is that the policy can recover from perturbations. For example, after an object is dropped, the humanoid can re-initiate pickup from the object’s new location instead of failing permanently because a reference trajectory was broken.  This makes sense given the representation design: if the policy is grounded in current geometry rather than a fixed motion target, it has a better chance to re-plan implicitly through closed-loop control.  5.4 Real-world deployment  The paper evaluates both a MoCap-based and a vision-based policy on a real humanoid platform.  Reported results include:      MoCap-based              PickUp 22 cm^3: 10/10        PickUp 60 cm^3: 8/10        SitStand 12 cm: 8/10        SitStand 46 cm: 10/10              Vision-based              PickUp 22 cm^3: 8/10        PickUp 60 cm^3: 7/10              The vision model is weaker than the MoCap-conditioned one, but still reasonably effective, which supports the claim that the DF-based interaction logic can be distilled into depth-based deployment.  6. Ablation Insights  The ablations are useful because they show the system is not succeeding from one trick alone.      Removing AIP significantly hurts robustness, implying the adversarial interaction prior is important for geometry-consistent interaction.    Removing synthetic physicalization hurts contact-rich tasks, especially carrying, showing that physically valid teacher trajectories matter.    Removing geometry randomization causes severe overfitting outside the training scale.    Removing RL post-training leaves behavior cloning insufficient for strong generalization.    Replacing the Transformer with an MLP hurts performance, especially on tasks with longer temporal dependencies.    Overall, the paper argues that the full stack is necessary: representation, physically grounded pre-training, adversarial post-training, and sufficient model capacity.  7. Strengths      Clear problem framing around representation rather than only reward design or imitation quality.    The DF formulation is intuitive and task-general.    Strong evidence for scale/shape generalization relative to motion-tracking baselines.    Long-horizon skill composition is a meaningful benchmark, not just isolated single-task success.    The paper includes a plausible path from privileged MoCap training to egocentric-depth deployment.    8. Limitations and Open Questions      The vision-based model still shows a noticeable gap from the MoCap-conditioned version.    The long-horizon ..."
    },{
      "title": "D-REX: Differentiable Real-to-Sim-to-Real Engine for Learning Dexterous Grasping",
      "url": "/posts/2026/03/d-rex-paper-notes/",
      "date": "2026-03-09",
      "tags": ["Robotics","Dexterous Grasping","Sim2Real","Differentiable Simulation","Gaussian Splatting","Imitation Learning","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  D-REX is a real-to-sim-to-real framework for dexterous grasping that tries to solve a specific bottleneck in sim-to-real transfer: the simulator often has the wrong object physics, especially object mass. The paper combines:      Gaussian-Splat-based object reconstruction for visually realistic digital twins    a differentiable physics engine for mass identification from robot interaction videos    human-video-to-robot-demo transfer for supervision    a force-aware grasping policy conditioned on the identified mass    The key idea is simple but useful: if the simulator can infer the object’s mass from real robot-object interactions, then the learned grasp policy can apply more appropriate grasp force and transfer more reliably to the real world.  Paper Info      Title: D-REX: Differentiable Real-to-Sim-to-Real Engine for Learning Dexterous Grasping    Authors: Haozhe Lou, Mingtong Zhang, Haoran Geng, Hanyang Zhou, Sicheng He, Zhiyuan Gao, Siheng Zhao, Jiageng Mao, Pieter Abbeel, Jitendra Malik, Daniel Seita, Yue Wang    Affiliations: University of Southern California, University of California Berkeley    Venue: ICLR 2026 conference paper    arXiv: 2603.01151    Project page: drex.github.io    1. Motivation  The paper starts from a practical issue in robot learning:      simulation is cheap and scalable for policy learning    but sim-to-real performance depends heavily on whether the simulator matches real-world object dynamics    geometry alone is not enough; mass mismatch often causes grasp failure because the required grasp force changes with object weight    The authors focus on building a pipeline that can recover a physically plausible digital twin from real observations, then use that twin to train better dexterous grasping policies.  2. What D-REX Does  D-REX has four main stages:      Real-to-Sim reconstruction: reconstruct scene/object geometry from RGB videos using Gaussian Splat representations, then derive collision meshes for simulation.    Mass identification: execute consistent robot actions in the real world and in simulation, then optimize object mass so the simulated trajectory matches the real one.    Human-to-robot demo transfer: recover human hand/object motion from RGB videos and retarget them into robot joint trajectories.    Policy learning: train a dexterous grasping policy conditioned on object geometry and identified mass.    This makes the pipeline a genuine “real-to-sim-to-real” loop instead of only using reconstruction for visualization.  3. Core Technical Idea  3.1 Differentiable mass identification  The central optimization target is to infer object mass m by minimizing the discrepancy between simulated and real trajectories:  min_{m > 0} L_traj(m) = sum_t || s_t^sim(m) - s_t^real ||_2^2    where the object state s = [p, q]^T contains 3D position and quaternion orientation.  The simulator uses a differentiable rigid-body/contact model and computes gradients of the trajectory loss with respect to mass. The paper implements the state update with a semi-implicit Euler integrator so gradients can backpropagate through the dynamics rollout.  In effect, the robot pushes an object, the system compares simulated motion with real motion, and the mass is adjusted until the simulated motion becomes consistent with reality.  3.2 Human demonstration transfer  Human RGB videos are processed to estimate:      articulated hand pose    object pose through time    These are then retargeted to a robotic hand using Dex-Retargeting, producing robot-executable trajectories that serve as demonstrations for learning grasp poses.  3.3 Force-aware policy learning  The learned policy does not only predict hand joint positions. It also predicts force-related outputs and explicitly conditions on the identified object mass.  This is the main learning claim of the paper: grasping should be mass-aware, because a pose that works for a light object may fail for a heavy object if the applied force is not adjusted.  4. Why This Is Interesting  Many sim-to-real manipulation papers focus on:      geometry reconstruction    domain randomization    direct imitation from human videos    D-REX argues that physical parameter identification, especially mass, should be part of the loop. That framing is useful because it turns “better reconstruction” into something task-relevant: the policy can exploit the recovered mass to learn force-aware grasps instead of only shape-aware grasps.  5. Experiments and Main Results  5.1 Mass identification  The paper evaluates mass identification by pushing objects with the same actions in the real world and in simulation, then optimizing mass from trajectory mismatch.  Reported findings:      across diverse objects, percentage error ranges roughly from 4.8% to 12.0%    for objects with the same geometry but different internal densities, the estimated mass error is under 13 g    simulated trajectories with optimized mass match real trajectories much better than trajectories using an incorrect lighter mass    This is the strongest systems contribution in the paper: the differentiable engine appears accurate enough to recover task-relevant mass information from interaction data.  5.2 Force-aware grasping  The grasping experiments show that:      policies trained for one mass perform well mainly on that mass    mismatched mass leads to failures from too much or too little force    policies using identified mass achieve performance comparable to policies using ground-truth mass    The cross-evaluation table is especially intuitive: train on one density, evaluate on another, and success drops sharply when mass no longer matches.  5.3 End-to-end tabletop grasping  The method is compared against:      DexGraspNet 2.0    Human2Sim2Robot    Across eight tabletop objects with different shapes and masses, D-REX reports consistently higher success rates and lower variance. The paper emphasizes that baseline performance degrades as objects get heavier, whereas the proposed force-aware policy remains more stable.  5.4 Runtime  The system is not lightweight, but it is practical as an offline pipeline:      object reconstruction uses about 300-340 RGB images    offline reconstruction takes about 30-35 minutes per object    mass identification takes about 1.43-1.68 s per iteration    convergence is typically around 200 epochs, or roughly 5-20 minutes    This is acceptable if the goal is offline digital-twin construction followed by policy training.  6. Strengths      Clear systems story: reconstruction, identification, demo transfer, and policy learning are tightly connected.    Good task framing: identifying mass is a concrete and manipulation-relevant target.    Differentiable physics is used for a meaningful downstream benefit rather than as a standalone novelty.    The force-aware policy claim is experimentally interpretable: wrong mass leads to the wrong force.    The pipeline uses only RGB observations plus robot interaction data, which is appealing from a deployment standpoint.    7. Limitations and Open Questions      The identified physical parameter is mainly mass; other contact properties such as friction and compliance are still major contributors to sim-to-real error.    The paper relies on several upstream components, including pose estimation, reconstruction, and retargeting, so the full pipeline may be brittle in harder settings.    Runtime is clearly offline; this is not yet close to online adaptation.    It is not obvious how well the approach scales to more contact-rich tasks such as in-hand reorientation or dynamic non-prehensile manipulation.    The grasping policy is conditioned on a single inferred mass value, which may be too coarse for objects with complicated internal mass distributions.    8. Takeaways  My main takeaway is that the paper makes a strong case for task-driven system identification in dexterous manipulation. Instead of treating digital twins as purely visual assets, D-REX uses differentiable simulation to recover the part of physics that most directly matters for grasp success.  For robotics research, this suggests a useful recipe:      reconstruct the object    identify the missing physical parameters from interaction    train the policy with those parameters explicitly in the loop    That is a more convincing route to sim-to-real transfer than relying on geometry reconstruction alone.  本文支持通过顶部导航中的语言切换按钮在 English / 中文 之间切换。  TL;DR  D-REX 是一个面向灵巧抓取的 real-to-sim-to-real 框架，核心想解决 sim-to-real 中一个非常实际的问题：仿真里的物体物理参数常常不对，尤其是质量。论文把以下几个部分串了起来：      基于 Gaussian Splat 的物体重建，用于构建视觉上逼真的数字孪生    基于可微物理引擎的质量识别，从真实机器人交互视频中反推出物体质量    从人类演示视频到机器人演示轨迹的迁移    一个以识别质量为条件的力感知抓取策略    核心思想很直接：如果仿真器能从真实交互中推断出物体质量，那么学到的抓取策略就能施加更合适的抓力，从而更稳定地迁移到真实世界。  论文信息      标题: D-REX: Differentiable Real-to-Sim-to-Real Engine for Learning Dexterous Grasping    作者: Haozhe Lou, Mingtong Zhang, Haoran Geng, Hanyang Zhou,..."
    },{
      "title": "EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data (arXiv 2026)",
      "url": "/posts/2026/02/egoscale-paper-notes/",
      "date": "2026-02-26",
      "tags": ["Robotics","Dexterous Manipulation","Vision-Language-Action","Human-to-Robot Transfer","Egocentric Data","Scaling Laws","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  EgoScale argues that human-to-robot transfer for dexterous manipulation is largely a scaling problem:      pretrain a flow-based VLA on 20,854 hours of action-labeled egocentric human video    use explicit wrist motion + retargeted 22-DoF hand actions as supervision    add a small aligned human-robot mid-training stage (~50h human + 4h robot)    then post-train on target robot tasks    The paper reports:      a near-perfect log-linear scaling law between human data size and human action prediction loss (R^2 = 0.9983)    strong correlation between that offline loss and real-robot dexterous manipulation performance    one-shot transfer on unseen dexterous tasks with minimal robot supervision    cross-embodiment transfer to Unitree G1 (tri-finger hand) with clear gains from human pretraining    Paper Info      Title: EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data    Authors: Ruijie Zheng, Dantong Niu, Yuqi Xie, Jing Wang, Mengda Xu, Yunfan Jiang, et al.    Affiliations: NVIDIA, UC Berkeley, University of Maryland    arXiv: 2602.16710    Version: v1 (arXiv, 2026-02-18; PDF header date 2026-02-19)    Project page: EgoScale (NVIDIA GEAR)    1. What Problem the Paper Targets  The paper asks a concrete question:  Can large-scale egocentric human manipulation data become a practical supervision source for high-DoF dexterous robot manipulation, not just low-DoF grippers or narrow settings?  The key challenge is that human data is:      abundant but noisy    not naturally paired with robot actions    collected under different embodiment/sensor/control spaces    2. Core Idea (Two-Stage Human-to-Robot Transfer)  EgoScale uses a simple but effective recipe:      Stage I: Human pretraining (scale)Train a flow-based VLA on large egocentric human data using explicit action supervision.    Stage II: Aligned human-robot mid-training (alignment)Co-train on a much smaller dataset with matched viewpoints and aligned human/robot play data.    Stage III: Task post-trainingFine-tune on task-specific robot demonstrations.    The framing I found useful: decouple scale from embodiment alignment.      Stage I provides diversity and manipulation priors.    Stage II grounds those priors into executable robot control.    3. Action Representation (Why Transfer Works Better)  Instead of only learning from visuals, the model is supervised with physically meaningful action targets:      relative wrist motion (end-effector motion, invariant to global camera motion)    retargeted hand joint actions in a 22-DoF Sharpa hand space    This matters because the paper shows action representation choice strongly affects downstream dexterous performance:      wrist-only performs poorly on precise finger/contact tasks    fingertip-based is better but unstable/inconsistent    retargeted joint-space hand actions are the most consistent    4. Model and Training Setup (Useful Details)      Policy: flow-based VLA (VLM backbone + DiT action expert), similar in spirit to GR00T N1-style design    Unified modeling across human and robot data:              human demos use a learnable placeholder for missing proprioception        embodiment-specific MLP adapters handle different robot state/hand action spaces              Training recipe (paper-reported):      Stage I (human pretraining): 20K+ hours, 100K steps, 256 GB200 GPUs, batch 8192, LR 5e-5    Stage II (aligned mid-training): 50K steps, batch 2048, LR 3e-5    Stage III (post-training): 10K steps, batch 512, LR 3e-5    5. Main Experimental Results  5.1 Human pretraining is the main driver  Across five dexterous tasks (shirt rolling, card sorting, tongs fruit transfer, bottle-cap unscrewing, syringe liquid transfer):      human pretraining consistently improves performance over training from scratch    the paper reports over 55% average task-completion improvement    Human Pretrain + Midtrain performs best overall    The qualitative takeaway is strong: large human data helps even when it is noisy and not tightly aligned to the target robot.  5.2 Clear scaling law (the most important result)  They pretrain on 1k / 2k / 4k / 10k / 20k hours and show:      average downstream task completion increases from 0.30 -> 0.71 (1k to 20k hours)    no saturation within the tested range    optimal human validation loss follows:    L = 0.024 - 0.003 * ln(D)  with R^2 = 0.9983, where D is human pretraining hours.  This is the paper’s strongest claim: offline human-action prediction loss predicts real-robot dexterous performance.  5.3 One-shot transfer on unseen dexterous tasks  With one robot demo plus aligned human demos (after pretraining + mid-training), the policy reaches:      0.88 success on Fold Shirt    0.55 success on Unscrewing Water Bottles    The paper emphasizes that this does not emerge from human pretraining alone or embodiment-specific data alone.  5.4 Cross-embodiment transfer to Unitree G1  They also test transfer to Unitree G1 with a 7-DoF tri-finger hand (very different from the 22-DoF Sharpa hand setup).      Human pretraining + embodiment-aware mid-training improves G1 task performance over G1-only training on the same data.    The intro also highlights 30%+ absolute success-rate improvement on evaluated G1 tasks vs no human pretraining.    This supports their “embodiment-agnostic motor prior” claim.  6. Why This Paper Matters (My Take)  I think the paper is important for three reasons:      It moves the human-to-robot transfer discussion from “can it work?” to “how does it scale?”    It shows a practical recipe where huge noisy human data + small aligned robot data is better than either alone    It treats hand articulation supervision as first-class, which is essential for dexterous manipulation (not just arm motion)    7. Strengths      Strong scale: 20,854 hours is unusually large for human-action-labeled egocentric manipulation    Clear experimental structure tied to concrete RQs    Convincing scaling-law analysis with downstream correlation    Practical transfer recipe with modest robot data in mid-training    Includes both one-shot transfer and cross-embodiment evaluation    8. Limitations / Open Questions  Some limitations are explicit, and some are my reading:      The method still needs aligned mid-training data to unlock the strongest transfer behavior    Human action labels rely on SLAM + hand-pose estimation / retargeting, which can be noisy or biased    The strongest results may depend on substantial infrastructure/compute (large-scale pretraining)    The paper shows scaling up to 20k hours, but not yet the boundary where gains saturate    It would be useful to see more ablations on which parts of the 20k-hour corpus matter most (domain/task diversity vs raw volume)    9. Takeaways for Robotics Research      Large-scale human egocentric data is becoming a credible supervision source for dexterous robot learning, not just coarse imitation.    For dexterous VLAs, action representation design (especially hand supervision) is a major lever.    A scalable path may be:              massive human pretraining for priors        small aligned human-robot data for grounding        task-specific robot post-training for execution quality              10. What I’d Revisit Later      Exact composition of the 20,854h dataset and which subsets drive gains    How performance scales with model size jointly with data size    Whether weaker/unlabeled egocentric video can help via self-supervised pretraining before action supervision    本文支持通过网站顶部语言切换按钮在 English / 中文 间切换。  TL;DR  EgoScale 的核心观点是：面向灵巧操作（dexterous manipulation）的人到机器人迁移，本质上很大程度是一个 规模化（scaling）问题。      先在 20,854 小时带动作标签的人类第一视角视频上预训练 flow-based VLA    使用 手腕相对运动 + 重定向后的 22-DoF 手部关节动作作为监督    再加入一个很小的 人机对齐中间训练（mid-training）阶段（约 50h 人类 + 4h 机器人）    最后在具体机器人任务上做 post-training    论文给出的关键结果包括：      人类数据规模与动作预测验证损失之间存在近乎完美的 对数线性 scaling law（R^2 = 0.9983）    这个离线损失与 真实机器人灵巧操作性能强相关    在极少机器人监督下实现 one-shot 新任务迁移    可迁移到 Unitree G1（三指手） 等不同 embodiment，且人类预训练带来明显收益    论文信息      标题: EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data    作者: Ruijie Zheng, Dantong Niu, Yuqi Xie, Jing Wang, Mengda Xu, Yunfan Jiang, 等    机构: NVIDIA, UC Berkeley, University of Maryland    arXiv: 2602.16710    版本: v1（arXiv 日期 2026-02-18；PDF 首页日期 2026-02-19）    项目主页: EgoScale (NVIDIA GEAR)    1. 这篇论文在解决什么问题  论文关注的核心问题是：  大规模 人类第一视角操作数据，能否真正成为 高自由度灵巧手机器人操作 的有效监督来源，而不只是低 DoF 夹爪或受限场景里的迁移？  困难点在于，人类数据通常：      规模大，但噪声高    没有天然配对的机器人动作标签    与机器人在形态（embodiment）、传感器、控制空间上存在显著差异    2. 核心方法（两阶段人到机器人迁移）  EgoScale 的做法可以概括为一个简洁有效的三阶段流程：      Stage I: 人类预训练（解决规模）在大规模人类第一视角数据上做 VLA 预训练，学习操作先验。    Stage II: 人机对齐 mid-training（解决对齐）用小规模但视角/动作空间更对齐的人类-机器人 play 数据做中间训练。    Stage III: 任务 post-training用目标任务的机器人演示数据做微调。    我认为这篇论文最值得借鉴的点是：把“规模”和“embodiment 对齐”解耦。      Stage I 提供多样性和通用操作结构    Stage II 把这些结构锚定到可执行的机器人控制空间    3. 动作表示设计（为什么迁移更有效）  作者没有只依赖视觉特征，而是显式监督“物理上有意义”的动作..."
    },{
      "title": "Visual-tactile pretraining and online multitask learning for humanlike manipulation dexterity (Science Robotics 2026)",
      "url": "/posts/2026/02/visuotactile-pretraining-online-multitask-dexterity/",
      "date": "2026-02-25",
      "tags": ["Robotics","Dexterous Manipulation","Vision-Tactile Learning","Multitask Learning","Reinforcement Learning","Science Robotics","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  This paper presents a strong real-world dexterous manipulation system that combines:      visual-tactile self-supervised pretraining from human demonstrations    online multitask imitation learning (with RL-trained per-task experts) for a single unified policy    Key practical result: with only a monocular webcam + low-cost binary tactile sensing, the system achieves about 85% average real-world success across multiple complex multifingered tasks and generalizes to several unseen tasks with related coordination patterns.  Paper Info      Title: Visual-tactile pretraining and online multitask learning for humanlike manipulation dexterity    Authors: Qi Ye, Qingtao Liu, Siyun Wang, Jiaying Chen, Yu Cui, Ke Jin, Huajin Chen, Xuan Cai, Gaofeng Li, Jiming Chen    Venue: Science Robotics (Research Article)    Published: 2026-01-28    DOI: 10.1126/scirobotics.ady2869    Why This Paper Matters  Dexterous manipulation is hard because the robot must handle:      high-dimensional finger control    contact-rich dynamics    occlusions (vision misses many contacts)    poor sample efficiency if trained end-to-end with RL only    The paper’s core contribution is not just “add touch,” but a two-stage learning design:      Learn a multisensory representation from human demonstrations (observation stage).    Learn a unified action policy via interaction + online imitation (practice stage).    This separation is a practical systems decision that reduces optimization difficulty.  Method Overview  1. Stage 1: Visual-Tactile Pretraining from Human Demonstrations  The authors pretrain a visual-tactile encoder with a masked autoencoder-style objective using human demonstrations paired with tactile glove signals.  Main design details:      RGB image tokens + tactile event tokens    modality-specific masking    cross-modal Transformer encoder    a learnable integration token (named IPL token) to aggregate multisensory information    decoder reconstructs masked visual and tactile inputs    Important idea:      tactile signals are reduced to binary contact events (touch / no-touch), which simplifies transfer across sensor types and helps the model learn when and where contact-relevant visual evidence appears.    2. Stage 2: Unified Multitask Policy via RL + Online Imitation Learning  They first train task-specific expert policies in simulation (PPO), then distill them into a single multitask policy.  Instead of only using offline expert trajectories, they use online dataset aggregation:      roll out the current unified policy    query the corresponding task expert on visited states    add those state-action pairs to the training set    train the unified policy by imitation loss    This reduces observation drift and compounding errors compared with pure offline imitation.  System / Setup Highlights      Platform: Shadow Hand mounted on a robotic arm    Sensors: monocular RGB webcam + 20 piezoresistive tactile sensors    Control frequency: 15 Hz in real-world deployment    Runtime: standard laptop (reported i9-12900K + RTX 4070)    Reported low sensing cost: about $250 for camera + tactile setup    Tasks:      5 seen/training tasks in real-world evaluation: bottle cap turning, faucet screwing, lever sliding, tabletop reorientation, in-hand reorientation    3 unseen tasks for generalization: pencil sharpening, screw unfastening, snack sleeve sliding    Main Results (What Stood Out)  1. Strong Real-World Performance  The paper reports:      about 87% average success on in-distribution real objects (3D-printed replicas)    about 85% average success on out-of-distribution daily objects    This is notable because the tasks require coordinated multifinger contact and the sensing setup is relatively simple.  2. Generalization to Unseen Tasks (Related Coordination Patterns)  They test three unseen tasks and condition the policy with related seen-task IDs:      pencil sharpening: 9/10 successes    screw unfastening: 6/10 successes    snack sleeve sliding: 8/10 successes    This is not arbitrary zero-shot generalization; it works best when the new task shares similar hand-object coordination patterns with training tasks.  3. Visual + Tactile Beats Single-Modality Policies  Compared with vision-only or tactile-only variants:      multimodal policy exceeds 80% success after training (on training object set)    single-modality baselines plateau below 70%    unimodal policies show much larger sim-to-real degradation (real-world performance on unseen printed objects drops sharply)    This supports the paper’s main argument that touch complements monocular vision under occlusion, lighting variation, and ambiguous textures.  4. Robustness to Sensor Variants and Lighting      The policy transfers across multiple tactile sensor types because it uses binary tactile events    In bottle cap turning, tested alternative tactile setups all succeeded in the reported trials    Under lighting variation, visual-tactile policies remain much more stable than vision-only policies    5. Online Multitask Imitation Learning Helps  The proposed online imitation strategy outperforms:      pure RL    offline IL    IL + RL fine-tuning    The explanation is sensible: querying experts on states visited by the current unified policy reduces distribution mismatch.  Why the “Humanlike” Claim Is Interesting  The paper analyzes tactile contact-duration patterns and reports that visual-tactile pretraining produces contact dynamics closer to human demonstrations than unimodal pretraining.  They also visualize attention maps for the integration (IPL) token and show:      visual-tactile attention focuses on hands and manipulated objects    attention changes with contact state / object dynamics    vision-only attention is less task-relevant and less stable    This is one of the stronger interpretability sections in the paper because it connects representation learning to robustness and transfer.  Strengths      Clear systems framing: pretraining for perception + online imitation for control    Real hardware validation on multifinger dexterous tasks    Strong multimodal ablations (V vs T vs VT)    Practical low-cost sensing setup    Good robustness analysis (lighting, tactile sensor variants, unseen tasks)    Convincing explanation for why binary tactile events can still guide attention    Limitations / Open Questions (My Reading)      Generalization is strong but mostly to tasks with related coordination patterns, not arbitrary new manipulation behaviors    The pipeline still depends on simulation training, task-specific rewards, and expert-policy training    Tactile input is deliberately simplified to binary events, which helps transfer but may discard rich force/geometry information    Arm motion is restricted in the setup (focus is on hand/finger dexterity), so full-arm dexterous manipulation remains open    Takeaways for Research / Practice      If you are building dexterous manipulation systems, multimodal pretraining + simple tactile events may be a better investment than trying to solve everything with vision-only RL.    Binary tactile abstractions are a strong engineering choice when hardware heterogeneity and sim-to-real transfer matter.    Online expert querying / dataset aggregation is a practical way to stabilize unified multitask policies.    本文支持通过网站顶部语言切换按钮进行 English / 中文 切换。  TL;DR  这篇论文提出了一个很强的真实世界灵巧操作系统，核心由两部分组成：      基于人类示范的视觉-触觉自监督预训练    基于任务专家策略的在线多任务模仿学习（统一策略）    关键结果是：只使用单目摄像头 + 低成本二值触觉信号，系统在多个复杂多指操作任务上实现了约 85% 的真实世界平均成功率，并能泛化到若干与训练任务具有相似手-物协调模式的未见任务。  论文信息      标题: Visual-tactile pretraining and online multitask learning for humanlike manipulation dexterity    作者: Qi Ye, Qingtao Liu, Siyun Wang, Jiaying Chen, Yu Cui, Ke Jin, Huajin Chen, Xuan Cai, Gaofeng Li, Jiming Chen    期刊/会议: Science Robotics（Research Article）    发表日期: 2026-01-28    DOI: 10.1126/scirobotics.ady2869    为什么这篇论文值得看  灵巧手操作难点主要在于：      多指高维控制空间    接触丰富、动力学复杂    视觉遮挡严重（很多接触状态看不见）    仅靠端到端 RL 训练样本效率低、训练不稳定    这篇论文的关键不只是“加了触觉”，而是采用了一个两阶段学习框架：      先通过人类示范学习多模态表示（观察阶段）    再通过交互 + 在线模仿学习训练统一动作策略（练习阶段）    这种“感知表示学习”和“控制策略学习”解耦的设计非常工程化，也更容易优化。  方法概览  1. 阶段一：基于人类示范的视觉-触觉预训练  作者使用类似 MAE（Masked Autoencoder）的自监督目标，对视觉-触觉编码器进行预训练。输入来自人类示范视频及其触觉手套信号。  主要设计包括：      RGB 图像 token + 触觉事件 token    模态特定 masking    跨模态 Transformer 编码    一个可学习的融合 token（IPL token）用于聚合多模态信息    解码器重建被 mask 的视觉与触觉输入    一个很实用的设计点是：      将触觉统一成二值接触事件（接触/未接触）    这样做能简化不同触觉传感器之间的迁移问题，同时帮助模型学习与接触相关的视觉线索“何时出现、出现在哪里”。  2. 阶段二：基于 RL 专家 + 在线模仿学习的统一多任务策略  作者先在仿真中训练每个任务的专家策略（PPO），再蒸馏成一个统一多任务策略。  他们没有只用离线专家轨迹，而是采用在线数据聚合：      用当前统一策略与环境交互    在访问到的状态上查询对应任务专家动作    将这些状态-动作对加入数据集    用模仿损失训练统一策略    这样能减轻纯离线模仿学习中的分布偏移和误差累积问题。  系统与实验设置亮点      平台：Shadow Hand（安装在机械臂上）    传感器：单目 RGB 摄像头 + 20 个压阻式触觉传感器    真实世界控制频率：15 Hz    运行平台：标准笔记本（论文报告为 i9-12900K + RTX 4070）    感知硬件成本（摄像头 + 触觉）约 250 美元    任务设置：      5 个训练/已见任务：拧瓶盖、拧水龙头、拨杆滑动、桌面重定向、手内重定向    3 个未见任务：削铅笔、拧松螺丝、零食袋套筒滑动    主要结果（..."
    },{
      "title": "A Review of Learning-Based Dynamics Models for Robotic Manipulation (Science Robotics 2025)",
      "url": "/posts/2026/02/learning-based-dynamics-models-review/",
      "date": "2026-02-24",
      "tags": ["Robotics","Manipulation","Dynamics Models","World Models","Model-Based Control","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  This Science Robotics review is a strong, robotics-centered survey of learning-based dynamics models for manipulation, with one especially useful organizing idea:      the design space is largely shaped by the state representation    The paper builds a clear taxonomy from less-structured to more-structured representations:      pixels    latent states    3D particles    keypoints    object-centric states    and analyzes trade-offs across:      perception difficulty    inductive bias / sample efficiency    generalization    interpretability    control-time computational cost    If you work on world models / model-based control for manipulation, this review is worth reading because it connects representation choice -> dynamics architecture -> control method -> task suitability in a very practical way.  Paper Info      Title: A review of learning-based dynamics models for robotic manipulation    Authors: Bo Ai, Stephen Tian, Haochen Shi, Yixuan Wang, Tobias Pfaff, Cheston Tan, Henrik I. Christensen, Hao Su, Jiajun Wu, Yunzhu Li    Venue: Science Robotics (Review article)    Publication date: 2025-09-17    DOI: 10.1126/scirobotics.adt1497    1. Why This Review Matters  There are many papers on world models, learned simulators, and model-based control, but they often focus on a single object type, a single sensing setup, or one task family. This review is valuable because it asks a broader robotics question:  How should we design learned dynamics models for manipulation when the environment is partially observable, contact-rich, and task-dependent?  The answer the paper emphasizes is not “use architecture X.” Instead, the authors argue that a major design choice is the state representation used by the perception+dynamics+control stack. That framing is practical and reusable.  2. Core Framework: Perception, Dynamics, Control  The paper formalizes manipulation under a POMDP and decomposes learned dynamics pipelines into three modules:      Perception module g: estimate a task-relevant state s_t from observations (and possibly action/history)    Dynamics module \\hat{T}: predict state transitions s_t -> s_{t+1} given action a_t    Control module \\pi: planning or policy learning using the learned dynamics model    This decomposition is simple but important. In practice, many failures come from the interaction between these modules, especially when a representation is good for one stage (e.g., efficient dynamics learning) but hard for another (e.g., robust state estimation).  3. Main Taxonomy: State Representations (The Most Useful Part)  The review organizes methods by state representation and repeatedly highlights a central trade-off:      more structure usually improves inductive bias and sample efficiency    but often makes perception/state estimation harder    3.1 Pixel representations (2D pixel space)  Pixel-based models treat dynamics learning as action-conditioned video prediction.  Strengths:      minimal explicit state-estimation pipeline    broad applicability to many modalities (RGB, depth, tactile images, density fields)    can leverage large-scale video modeling advances (transformers, diffusion)    Weaknesses:      high-dimensional prediction space -> data hungry    can hallucinate under partial observability    expensive for high-frequency control    standard video metrics often do not align with control quality    My takeaway: pixel models are attractive for generality, but control reliability remains difficult unless you add stronger priors or huge data.  3.2 Latent representations  Latent-state models compress observations into a lower-dimensional z_t, then predict dynamics in latent space.  The review nicely separates:      reconstruction-based representation learning    reconstruction-free objectives (e.g., inverse dynamics, contrastive, reward-predictive)    and discusses probabilistic vs deterministic latent dynamics (e.g., RSSMs vs MLP/CNN predictors).  Strengths:      efficient control-time inference    good sample efficiency when latent structure is well chosen    widely used in real-world model-based RL / manipulation    Weaknesses:      latent quality depends heavily on training objective    task-specific objectives may hurt transfer    generalization across object counts / scene configurations is still limited    3.3 3D particle representations  Particle representations explicitly encode geometry and local interactions, making them especially strong for deformable / nonrigid manipulation.  Common modeling choices:      GNN-based particle interaction models (e.g., DPI-Net / GNS-style families)    convolutional particle interaction architectures (e.g., SPNets-style)    Strengths:      strong physical inductive bias    sample efficiency    good fit for deformable objects, granular materials, fluids    natural integration with multimodal sensing (vision + touch)    Weaknesses:      state estimation from observations is hard (occlusion, tracking, correspondences)    scalability/cost issues for dense graphs    This is a recurring theme in the review: particle models can be excellent dynamics models, but perception can become the bottleneck.  3.4 Keypoint representations  Keypoints are sparse, task-relevant points (2D/3D) with implicit or explicit semantics.  The review covers:      supervised keypoint learning    unsupervised keypoint discovery    zero-shot keypoint detection using vision foundation models (CLIP/DINO-style features, etc.)    Strengths:      compact and efficient    often good for control and real-time planning    can generalize across object instances when keypoints capture consistent task structure    Weaknesses:      sensitive to occlusion and temporal consistency errors    keypoint extraction quality is critical    3.5 Object-centric representations  Object-centric models represent scenes as discrete interacting entities and explicitly model relations.  Strengths:      good for multi-object reasoning and compositional generalization    natural fit for graph-based relational dynamics    high-level abstraction often matches rearrangement/manipulation tasks    Weaknesses:      difficult perception problem (instance segmentation, inverse rendering, object proposals)    less suitable for highly deformable/continuous materials    4. Representation Choice Is Really a Control Design Choice  One of the best messages in the paper is that representation choice is not just a perception or modeling preference. It directly affects control:      planning stability    computational cost    whether gradients are useful    how badly model errors are exploited during optimization    The review discusses two main control paradigms:      motion planning (path planning + trajectory optimization; e.g., random search, CEM, MPPI, gradient-based optimization)    policy learning (including model-based RL and goal-conditioned policy training from learned rollouts)    The practical insight is that different representations pair naturally with different control styles. For example:      compact latents/keypoints can support fast iterative control    particle models can offer better physical fidelity for deformables but may be heavier    object-centric models can help planning in multi-object tasks    5. Representative Tasks Covered  The review summarizes how learned dynamics models are used across several task families:      object repositioning    deformable object manipulation (rope, cloth, dough, soft objects)    multi-object manipulation (packing, insertion, rearrangement)    tool-use manipulation    This section is useful because it maps task types to representation choices instead of treating “world model for robotics” as one homogeneous problem.  6. Future Directions (Well-Framed and Worth Reading)  The future-directions section is one of the strongest parts of the review. It is concrete and not just generic “scale more data.”  Some key directions the authors emphasize:      better handling of partial observability and robust state estimation    richer multimodal perception (vision + touch + audio, etc.)    more robust dynamics models under long-horizon planning and model exploitation    foundation dynamics models (and the data bottleneck for action-labeled interaction data)    using foundation-model priors for physical parameter estimation    importing new scene representations from graphics (e.g., NeRF/3DGS-inspired directions)    large-scale scene representations beyond tabletop settings    hierarchical dynamics modeling and planning    planning under imperfect models with stronger robustness / guarantees    I especially like the emphasis on hierarchical abstraction. It matches the review’s core thesis: one representation level is unlikely to be optimal for every decision scale.  7. Strengths of the Review      Clear robotics-centric framing (not just ML taxonomy)    Useful representation-first or..."
    },{
      "title": "BFM-Zero: A Promptable Behavioral Foundation Model for Humanoid Control Using Unsupervised Reinforcement Learning (arXiv 2025)",
      "url": "/posts/2026/02/bfm-zero-paper-notes/",
      "date": "2026-02-23",
      "tags": ["Robotics","Humanoid","Reinforcement Learning","Unsupervised RL","Sim2Real","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  BFM-Zero is a humanoid-control foundation model built with off-policy unsupervised RL (instead of standard PPO-style task-specific training). It learns a shared latent task space that can be prompted for:      zero-shot motion tracking    zero-shot goal reaching    zero-shot reward optimization    few-shot latent-space adaptation (without finetuning network weights)    The paper’s main contribution is not just a single algorithmic trick, but a full sim-to-real recipe combining:      Forward-Backward (FB) unsupervised RL representations    motion-data regularization (FB-CPR lineage)    asymmetric history-based learning    domain randomization    safety/feasibility reward regularization    They demonstrate the system on a real Unitree G1 humanoid, including recovery from large perturbations and promptable behavior composition.  Paper Info      Title: BFM-Zero: A Promptable Behavioral Foundation Model for Humanoid Control Using Unsupervised Reinforcement Learning    Authors: Yitang Li, Zhengyi Luo, Tonghe Zhang, et al.    Affiliations: Carnegie Mellon University, Meta    Venue: arXiv preprint (submitted 2025-11-06)    arXiv: 2511.04131    Project page (paper first page): BFM-Zero Website    1. Motivation and Problem Setting  The paper targets a core gap in humanoid control:      Many strong humanoid systems are task-specific (especially tracking).    Many rely on on-policy RL (e.g., PPO) with explicit rewards.    It is hard to get large-scale humanoid action-label datasets for pure behavior cloning.    The authors ask whether off-policy unsupervised RL can train a reusable, promptable behavioral foundation model for humanoids that supports multiple downstream tasks without retraining.  They formulate real-world humanoid control as a POMDP and train in simulation with:      privileged state available in sim    partial observations for the actor (plus history)    a motion dataset of unlabeled trajectories used as behavioral regularization    2. Core Idea: A Promptable Behavioral Foundation Model  BFM-Zero learns:      a shared latent task space Z    a promptable policy conditioned on latent vector z    Different task types are mapped into the same latent space:      Goal reaching: prompt with a latent derived from a target state    Reward optimization: infer a latent from reward-weighted state embeddings    Tracking: generate a sequence of latent prompts from a motion trajectory    This gives a unified interface for humanoid behaviors instead of training separate policies per task.  3. Method Overview  3.1 Built on Forward-Backward (FB) Unsupervised RL  The method builds on Forward-Backward representations and FB-CPR:      B(s) encodes states into a latent/task-related representation    F(s, a, z) behaves like a latent-conditioned successor-feature-style quantity    z defines a task/objective in latent space    The paper emphasizes that this latent space is structured enough to support:      explainable prompting    zero-shot task execution    interpolation between skills    few-shot optimization in latent space    3.2 Key Sim-to-Real Design Choices (Most Important in Practice)  The paper highlights several design choices needed to make unsupervised RL work on real humanoids:      Asymmetric learning: actor uses observation history, critics use privileged information    Domain randomization: masses, friction, offsets, perturbations, sensor noise    Reward regularization: auxiliary penalties for safety / physically feasible behavior    Large-scale off-policy training: many parallel environments, replay buffer, high UTD ratio    This is the strongest practical lesson from the paper: the success comes from the combination of representation learning and systems-level sim-to-real engineering.  3.3 Training Objective (High Level)  BFM-Zero combines multiple components during training:      an FB objective for learning long-horizon latent dynamics / representations    an auxiliary critic for safety and physical constraints    a discriminator/style critic to bias behaviors toward motion-data realism    a policy objective that balances task value, style, and regularization terms    The result is a policy that is both:      broad enough to support different prompts    constrained enough to remain stable and human-like on hardware    4. Zero-Shot Inference Modes (Why This Paper Is Interesting)  The same pre-trained model supports multiple downstream uses:  4.1 Zero-shot Tracking  Given a motion trajectory, BFM-Zero derives a sequence of latent prompts and tracks the motion without retraining.  4.2 Zero-shot Goal Reaching  A target pose/state is embedded into the latent space and used as a prompt. The paper shows smooth transitions and reasonable behavior even for difficult or partially infeasible targets.  4.3 Zero-shot Reward Optimization  A reward function can be converted into a latent prompt via replay-buffer samples and state embeddings. This enables:      locomotion commands    arm-raise commands    crouching / sitting-like behaviors    combined rewards (composed skills)    This is an unusually clean interface: the same model can be prompted by rewards, goals, or motions.  5. Experiments and Results  5.1 Training / Setup      Robot: Unitree G1 humanoid    Simulation: IsaacLab for training (paper reports simulation at 200 Hz, control at 50 Hz)    Behavior dataset: retargeted LAFAN1 motions (40 several-minute motions)    Also evaluated: Mujoco sim transfer and a Booster T1 humanoid in appendix    5.2 Simulation Validation (Zero-shot)  The paper evaluates:      tracking    reward optimization    pose/goal reaching    Key reported observations:      Domain-randomized deployable BFM-Zero performs somewhat worse than a privileged no-DR version, but remains strong.    The paper reports drops of about 2.47% / 25.86% / 10.65% across tracking / reward / pose-reaching compared with the idealized privileged setting.    Sim-to-sim transfer to Mujoco shows relatively small degradation (reported variations under ~7%).    The model also generalizes to out-of-distribution AMASS motions/poses (evaluated in Mujoco).    5.3 Real-Robot Results (Main Highlight)  The real-robot section demonstrates:      tracking of diverse motions (including dynamic behaviors)    goal reaching with smooth transitions    reward optimization for locomotion / posture / arm control    disturbance rejection and recovery (pushes, kicks, being dragged/falling)    A particularly notable claim is that recovery looks natural/human-like rather than brittle or overly aggressive.  5.4 Few-Shot Adaptation Without Finetuning Weights  The paper adapts behavior by optimizing in latent prompt space:      Single-pose adaptation: with a 4 kg payload, optimized latent prompt improves single-leg standing from failure (<5s collision) to >15s balance.    Trajectory adaptation: under friction shift, latent-sequence optimization improves tracking error by about 29.1%.    This is a strong demonstration of prompt-level adaptation for control.  6. Latent Space Structure (Interpretability / Compositionality)  The authors visualize the latent space (t-SNE) and show:      semantically similar motions cluster together    different task types occupy structured regions    interpolating latent vectors yields meaningful intermediate behaviors    This supports their “promptable BFM” framing: the latent space is not just a hidden internal variable, but a usable interface.  7. Strengths      Clear and ambitious objective: a promptable humanoid behavioral foundation model    Strong real-world focus, not simulation-only    Unified interface across tracking / goals / rewards    Practical sim-to-real recipe (asymmetric learning + DR + regularization)    Compelling prompt-space adaptation results without network finetuning    Good qualitative emphasis on robustness and natural recovery    8. Limitations / Open Questions  The paper explicitly notes several limitations (Discussion):      Behavior scope depends on the training motion data distribution    More work is needed on scaling laws (data size / architecture / performance)    Sim-to-real gap is reduced but not solved; stronger online adaptation may be needed    Fast adaptation and finetuning are only preliminarily explored    My additional practical questions:      How robust is reward-to-latent inference when replay-buffer samples shift under stronger domain randomization?    How much of the real-world quality comes from the discriminator/style prior vs. the FB latent structure itself?    What is the failure boundary for more contact-rich manipulation-like humanoid tasks?    9. Takeaways for Robotics Research      Off-policy unsupervised RL for real humanoids is more viable than many people assume.    A reusable humanoid controller may benefit from a latent prompt interface instead of task-specific policies.    Sim-to-real success here is heavily driven b..."
    },{
      "title": "From Entropy to Epiplexity: Rethinking Information for Computationally Bounded Intelligence",
      "url": "/posts/2026/02/epiplexity-paper-notes/",
      "date": "2026-02-23",
      "tags": ["Machine Learning Theory","Information Theory","Data-Centric AI","Pretraining","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle.  TL;DR  This paper argues that classical information notions (Shannon entropy, Kolmogorov complexity) miss a key ingredient for modern ML:      the observer is computationally bounded    To address this, the authors introduce epiplexity:      a measure of structural information that a computationally bounded learner can extract from data    paired with:      time-bounded entropy: the residual random/unpredictable part for that bounded learner    The paper’s core message is highly relevant to data-centric ML:      data quality is not only about total bits or likelihood    it is also about how much reusable structure a bounded model can absorb    this may better explain OOD transfer and pretraining data value    Paper Info      Title: From Entropy to Epiplexity: Rethinking Information for Computationally Bounded Intelligence    Authors: Marc Finzi, Shikai Qiu, Yiding Jiang, Pavel Izmailov, J. Zico Kolter, Andrew Gordon Wilson    Source: arXiv (2026, based on 2601.03220)    1. What problem is this paper solving?  The paper asks a fundamental question:      How should we define and measure the information value of data for learning systems with limited compute?    Why this matters:      In modern pretraining, data choice/curation often matters as much as architecture.    Classical information theory is excellent for communication and coding, but often does not capture:              structure vs randomness (from the model’s perspective)        compute constraints of the learner        why some data formats/orderings improve transfer without improving in-distribution loss much              The authors position epiplexity as a step toward a theory of data selection, not just model selection.  2. The three “paradoxes” motivating the paper  The paper frames the gap between theory and practice using three apparent paradoxes:  Paradox 1: “Information cannot be increased by deterministic transformations”  Classical view:      deterministic transforms should not create new information    But practice suggests otherwise:      synthetic data can improve models    self-play systems (e.g., AlphaZero-style examples) extract useful knowledge from simple rules    dynamical systems generate emergent structures that models can learn    Paradox 2: “Information is independent of factorization/order”  Classical view:      total information content should not depend on ordering/factorization    But in practice:      left-to-right vs reverse text/chess orderings can change what models learn    one direction can be easy to predict while the reverse is much harder    Paradox 3: “Likelihood modeling is just distribution matching”  Classical intuition:      maximizing likelihood only matches the data-generating distribution    models should not learn “more structure” than what is in the generating process    But in practice:      predictive models often need to learn extra internal structure (induction, emergent shortcuts) to make tractable predictions under limited compute    The paper’s thesis is that these paradoxes weaken once we explicitly model a computationally bounded observer.  3. Core concept: Epiplexity + Time-Bounded Entropy  The paper defines an MDL-style optimization under a runtime constraint T:      find the best time-bounded probabilistic program P* minimizing:              model description length + expected coding loss              Then decompose information into:      Epiplexity S_T(X) = size of the optimal time-bounded program (|P*|)    Time-bounded entropy H_T(X) = residual expected unpredictability under that program    Interpretation:      S_T(X) = structural information visible to the bounded learner    H_T(X) = random/unpredictable information (to that learner)    This decomposition is the main conceptual contribution.  Intuition  Two datasets can have similar training loss / entropy-like behavior but differ in:      how much structure the model had to internalize to get that loss    The paper argues that this “missing component” is often what matters for OOD transfer.  4. Why this is different from Shannon entropy / Kolmogorov complexity  The paper is not merely proposing another complexity metric. It changes the viewpoint:      information becomes observer-dependent    the observer is constrained by computation    Consequences:      CSPRNG outputs can look random to polynomial-time observers (high time-bounded entropy, low epiplexity)    deterministic dynamics can produce data with high structural content for bounded learners    ordering/factorization can matter because forward and inverse computations can have different computational difficulty    This is why the framework naturally connects to cryptography, emergence, and ML training dynamics.  5. How they estimate epiplexity in practice (important)  The theory is abstract, but the paper gives practical estimators using neural network training.  5.1 Prequential coding (fast heuristic)  Key idea:      estimate model information from the area under the loss curve above the final loss    If loss drops slowly and substantially:      the model may be absorbing more structural information from data    Pros:      simple    can reuse standard training runs    Cons:      not fully rigorous as an explicit time-bounded code for the learned model    5.2 Requential coding (more rigorous, slower)  Key idea:      explicitly code the learned model via a teacher-student training process    use cumulative KL terms (teacher vs student) to estimate model code length    The paper presents this as a more principled way to estimate epiplexity, but with extra computational overhead (they note it is typically slower than prequential coding).  Practical recommendation (paper’s stance)      use prequential for quick ranking / rough comparisons    use requential when more accurate estimates are needed    This is a strong part of the paper: it does not stop at theory, it gives a measurement pipeline.  6. Key experiments / evidence (what the paper shows)  6.1 Deterministic computation can create useful information (for bounded observers)  Using cellular automata (ECA rules), the paper shows different deterministic rules produce very different decompositions:      mostly simple/predictable    mostly random (high time-bounded entropy, low epiplexity)    mixed random + structured (higher epiplexity)    This supports the idea that deterministic processes can create learnable structure from the perspective of a bounded learner.  6.2 Factorization/order matters  The paper shows order-dependent effects in:      one-way-function-like setups    chess data formatting/ordering    A particularly interesting result:      a reverse chess ordering can yield higher epiplexity and better OOD transfer on some downstream chess tasks, even if the in-distribution setup is not obviously better by standard metrics    This is a very practical takeaway for data formatting and curriculum design.  6.3 Likelihood modeling can require learning more than the generating process  Through induction and emergence examples (e.g., masked latent information, ECA dynamics), the paper shows that a bounded predictive model may need to learn:      inversion strategies    latent inference procedures    emergent pattern abstractions    that are not explicitly present in the short data-generating program.  This is one of the most important conceptual contributions in my view.  6.4 Epiplexity and natural data / pretraining  The paper estimates epiplexity across natural modalities and data formats and argues:      language tends to exhibit higher epiplexity than images under their estimation setup    tokenization/representation choices (e.g., VQ tokenization) can change epiplexity    epiplexity may explain part of why some pretraining data sources transfer better than others    They also connect this to Adaptive Data Optimization (ADO) and show evidence that data selection strategies that improve downstream/OOD performance can coincide with higher estimated epiplexity.  7. Why this matters for data-centric ML (my take)  This paper gives a useful lens for questions like:      Why does data ordering matter?    Why can synthetic data help even when “no new Shannon information” is added?    Why do some corpora transfer better than others despite similar perplexity?    Why can training loss fail to predict downstream transfer?    The key distinction is:      loss / entropy-like terms capture residual unpredictability    epiplexity captures how much reusable structure the model had to build    That decomposition is not a full theory of transfer, but it is much closer to the phenomena we actually observe in pretraining.  8. Strengths      Ambitious and original conceptual framing    Connects ML practice with cryptography / complexity / MDL in a non-trivial way    Provides both theory and practical estimation procedures    Explains several empirically familiar phenomena (ordering, emerg..."
    },{
      "title": "RLinf → RLinf-USER → RL-Co: A Full-Stack RL Pipeline for Embodied VLA Training",
      "url": "/posts/2026/02/rlinf-series-notes/",
      "date": "2026-02-22",
      "tags": ["Robotics","Reinforcement Learning","Embodied AI","VLA","Systems","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle.  TL;DR  These three papers form a coherent full-stack story for embodied RL:      RLinf: makes large-scale RL workflows efficient and flexible    RLinf-USER: makes real-world online policy learning runnable on heterogeneous robots    RL-Co: makes simulation RL actually improve real-world VLA performance (instead of just sim metrics)    The key idea across the series is:      systems bottlenecks and training-paradigm bottlenecks are coupled    you need both infrastructure and algorithm design to get sim-real RL to work for large models / real robots    The Three Papers (exact titles)            RLinfRLinf: Flexible and Efficient Large-scale Reinforcement Learning via Macro-to-Micro Flow Transformation              RLinf-USERRLinf-USER: A Unified and Extensible System for Real-World Online Policy Learning in Embodied AI              RL-CoBeyond Imitation: Reinforcement Learning-Based Sim-Real Co-Training for VLA Models        1. Unified Problem View  All three papers address one broader question:      How do we scale interactive RL for embodied foundation models (especially VLA policies) efficiently and stably, from simulation to real robots?    This splits into three layers:                    Layer        Paper        Bottleneck                            RL compute engine        RLinf        RL workflows are heterogeneous/dynamic, so fixed execution modes waste hardware                    Real-world learning system        RLinf-USER        Real robots are slow, asynchronous, heterogeneous, and crash-prone                    Sim-real training algorithm        RL-Co        SFT-only co-training underuses simulation interaction; sim-only RL forgets real-world skills              Your summary captures this well. What the papers add is a clear stack decomposition: execution system -> robot infrastructure -> training objective.  2. Paper-by-Paper Notes (with extra details)  2.1 RLinf (compute engine for large-scale RL)  Core claim  RL training inefficiency is largely a system flexibility problem, not just a kernel/runtime problem.  The paper argues that RL workflows are:      heterogeneous (generation/inference/training/simulator/search server have different resource profiles)    dynamic (long-tail rollout durations block synchronized stages)    dependency-heavy (dataflow + weight update barriers + cyclic flows in embodied RL)    Main idea: M2Flow (Macro-to-Micro Flow)  Developers write RL workflows at a macro logical flow level (clean procedural program), and RLinf transforms it into a micro execution flow optimized for the current hardware/workload.  This decouples:      logical workflow semantics    physical execution plan / scheduling mode    Key mechanisms (paper-verified)      Worker abstraction for RL components    Adaptive communication across workers (placement-aware backend selection)    Elastic pipelining (flexible granularity / batch flow)    Automatic context switching for temporal GPU multiplexing (with device-lock-based coordination)    Profiling-guided scheduler that searches execution plans    Useful nuance:      RLinf uses Ray for cluster/process management, but implements its own more flexible device allocation (instead of relying only on Ray’s packed/spread modes).    The paper explicitly notes that non-accelerator devices (including robot arms) can also be abstracted as schedulable devices. This foreshadows RLinf-USER.    Results (as reported)      1.07x – 1.70x speedup vs prior systems (reasoning RL settings)    up to 2.43x speedup in embodied RL settings    scheduling search overhead remains small (reported milliseconds to seconds even at large GPU counts)    Why RLinf matters in the series  Without this layer, algorithmic RL improvements often disappear in practice because rollout/training pipelines are bottlenecked by orchestration and hardware idle time.  2.2 RLinf-USER (real-world online RL system)  Core claim  Real-world policy learning is fundamentally a systems problem, not just an algorithm problem.  Unlike simulation:      you cannot cheaply reset robots    you cannot massively replicate hardware    cloud-edge networks are unstable / bandwidth-limited    long-running experiments need persistence + crash recovery    Main idea: treat robots as first-class hardware resources  USER introduces a unified hardware abstraction layer (HAL) where:      robots and accelerators are both schedulable hardware units    heterogeneous deployments can be discovered, managed, and scheduled under a unified interface    This is a very important conceptual move. It reframes robot learning orchestration as a distributed systems scheduling problem.  Core system design (paper-verified)  (A) Unified Hardware Abstraction Layer      nodes expose typed hardware units (GPU, robot, etc.)    scheduler operates on hardware units as atomic resources    supports heterogeneous node groups (rollout nodes, robot nodes, training nodes)    (B) Adaptive Communication Plane      tunneling-based cloud-edge networking (for NAT / isolated domains)    distributed data channels (sharded FIFO producer-consumer queues to localize traffic)    SM-aware NCCL weight sync (caps NCCL GPU SM footprint to reduce rollout interference)    This communication design is one of the most practically useful contributions in the paper.  (C) Fully Asynchronous Learning Framework  USER decouples:      data generation    data transmission    training    weight synchronization    The robot side no longer blocks on synchronized train/update cycles.  (D) Persistent, Cache-Aware Buffer      recent data in memory    historical data persisted to disk    supports crash recovery and long-running training    avoids the “memory-only replay buffer” limitation in long-horizon real-world experiments    Extensibility (important practical point)  USER supports under one pipeline:      policies: CNN/MLP, flow/generative policies, large VLA models    algorithms: RL + imitation + human-in-the-loop variants    rewards: rule-based, human labels, reward models    This is what makes it more than a one-off system for a single method.  Results and useful quantitative details  From the paper’s experiments/ablations:      distributed channels reduce cross-domain episode generation time by up to about 3x    asynchronous pipeline improves both generation/training periods substantially (reported speedups include 1.20x/1.55x generation and 5.70x/4.61x training in selected settings)    the system demonstrates multi-robot and heterogeneous training setups    Why RLinf-USER matters in the series  RLinf solves RL execution efficiency in general, but USER makes it workable for real robots, where network topology, hardware heterogeneity, and persistence dominate.  2.3 RL-Co (algorithmic sim-real bridge for VLA)  Core claim  Most sim-real co-training for VLA models treats simulation as a static demo dataset (SFT-only), which leaves a lot of performance on the table because simulation’s main advantage is interactive RL.  But doing RL in simulation alone causes a different failure:      catastrophic forgetting of real-world behaviors    poor real-world transfer despite improving sim reward/success    Main idea: RL in sim, anchored by real-world SFT  RL-Co proposes a simple but strong two-stage design:  Stage I: SFT co-training (initialization)  Train on a mixture of simulation and real demos:  L_SFT = alpha * L_SFT(D_sim) + (1 - alpha) * L_SFT(D_real)  Purpose:      inject real-world task knowledge    bootstrap enough simulation competence for RL to start from a non-trivial policy    Stage II: real-regularized RL in simulation  Optimize in simulation with RL, plus a real-data SFT anchor:  L_total = L_RL^sim + beta * L_SFT^real  Purpose:      gain exploration and reward-driven improvement in sim    preserve real-world skills and mitigate forgetting    This is the central algorithmic contribution of the series.  Why this is better than SFT-only co-training  SFT co-training can mix sim and real data, but it still:      depends on fixed trajectories    cannot leverage reward feedback    cannot actively explore failure modes / corrections    RL-Co uses simulation as an interactive optimizer, not just a data augmenter.  Paper details that strengthen the story (from the PDF)      Real/sim tasks are framed as paired POMDPs (digital twins)    Simulation is built in ManiSkill    They do not chase photorealism; they model essential geometry/task structure    Sim data generation uses MimicGen, seeded by replaying real trajectories in ManiSkill    They generate about 1,000 successful simulated trajectories per task for D_sim    Evaluated on 4 tabletop tasks with OpenVLA and π0.5    For π0.5 RL stage, they use ReinFlow and mention using RLinf as the training framework (nice connection back to paper 1)    Main results (as reported)  Real-world success gains over baselines:      OpenVLA: up to +24% (paper reports substantial gains; table avera..."
    },{
      "title": "χ0: Resource-Aware Robust Manipulation via Taming Distributional Inconsistencies (arXiv 2026)",
      "url": "/posts/2026/02/kai0-paper-notes/",
      "date": "2026-02-22",
      "tags": ["Robotics","Manipulation","Imitation Learning","VLA","Paper Notes"],
      "excerpt": "",
      "text": "This post supports English / 中文 switching via the site language toggle in the top navigation.  TL;DR  This paper argues that robust real-world manipulation is often bottlenecked by distribution mismatch rather than just model scale:      P_train: human demonstration distribution    Q_model: policy inductive bias after training    P_test: actual deployment trajectories (including control latency / execution effects)    The proposed χ0 framework improves robustness by aligning these distributions with three practical modules:      Model Arithmetic (MA): merge subset-trained policies in weight space (model soup)    Stage Advantage (SA): directly predict progress/advantage with stage conditioning    Train-Deploy Alignment (TDA): recovery data (heuristic DAgger), spatio-temporal augmentation, and deployment-side temporal smoothing    The result is a highly engineering-driven, data-efficient system for long-horizon garment manipulation on real dual-arm robots.  Paper Info      Title: χ0: Resource-Aware Robust Manipulation via Taming Distributional Inconsistencies    Authors: Checheng Yu et al. (Kinetix AI)    Venue: arXiv preprint (2026)    Project links (from paper first page):              Code (OpenDriveLab/kai0)        Project blog              1. Problem Statement  The paper frames the entire robot learning pipeline as a distribution alignment problem across:      P_train: expert demonstrations used for imitation learning    Q_model: action distribution induced by the learned policy    P_test: executed trajectories during real deployment (after inference-to-control effects)    Three failure modes follow from mismatch:      Coverage deficiency: demonstrations undersample the valid task manifold    Temporal mismatch: long-horizon stages look visually similar but require different actions; latency further shifts execution timing    Failure cascade: no recovery behavior in demos means small perturbations can snowball into unrecoverable failures    This is a useful lens because it explains why simply increasing parameter count or compute may not fix real-world robustness.  2. Core Contributions (χ0)  2.1 Model Arithmetic (MA)  Instead of training one policy on all data and stopping there, the authors:      split data into subsets    train multiple policies/checkpoints    merge them in weight space (model soup / weighted interpolation)    select merging weights using validation on OOD recovery data (DAgger-style data), not only in-domain demos    Why this matters:      It improves mode coverage under limited demos    It is resource-efficient compared with collecting much more expert data    It explicitly targets the gap between training support and deployment states    The paper studies multiple soup strategies (average, inverse-loss, gradient-based, greedy), and reports strong gains with OOD validation for selection.  2.2 Stage Advantage (SA)  The paper’s main post-training idea is to build a more stable advantage signal for long-horizon manipulation.  Prior recipe (e.g., π*0.6-style advantage) uses:      A(s, a) = V(s') - V(s)    The authors argue this is noisy because:      two noisy value predictions are subtracted (variance compounds)    multi-stage tasks create ambiguous progress values for visually similar states    χ0 instead directly models advantage/progress as a pairwise prediction:      A(s, a) = f_theta(s, s')    and then makes it stage-aware:      A_stage(s, a, g) = f_theta(s, s' | g)    where g is a manually annotated stage label (normalized scalar in the paper).  Key effects:      denser and more stable progress signals    less idling / spurious retries during long-horizon execution    improved advantage-weighted regression post-training    2.3 Train-Deploy Alignment (TDA)  This module attacks the P_train vs P_test gap through practical deployment + data tricks:      Heuristic DAgger: manually construct failure states, then collect recovery demonstrations    Spatio-temporal augmentation: left-right flip (with arm swap), frame skipping    Temporal chunk-wise smoothing (deployment-side): smooth action chunk transitions to mitigate inference-control latency    This is one of the strongest parts of the paper: the authors treat deployment as a control systems problem, not only a training objective problem.  3. Training Paradigm (What It Is / What It Is Not)  The paper is explicit that χ0 is not standard online RL post-training.  What χ0 post-training is      Core training: imitation learning / behavior cloning (BC)    Post-training: advantage-weighted regression (AWR-style) on offline data    Advantage source: learned progress/advantage estimator from trajectory data (stage-aware in χ0)    Goal: bias the policy toward higher-progress actions while preserving training stability on real robots    What χ0 post-training is not      no PPO / SAC / policy gradient online optimization    no Bellman backup, no Q-learning loop    no environment-reward-driven exploration policy improvement during deployment    no large-scale online trial-and-error RL rollouts as the main learning engine    χ0 vs “typical RL post-training” (important distinction)  Compared with general RL post-training, χ0 is closer to:      weighted imitation learning / offline policy reweighting than online RL    supervised post-training with advantage labels than closed-loop reward maximization    Practical differences:      Safer on real robots: less exploration risk    More stable optimization: no bootstrapping instability from Bellman updates    More data-efficient in robot-time: reuses demos + DAgger recovery data    Less theoretically optimal in the RL sense: improvement is bounded by demonstration and recovery data quality    The appendix reinforces this point and explicitly asks “Why not online RL such as PPO?” Their answer is mainly about real-world sample inefficiency and reset/parallelization cost.  4. Data, Hardware, and System Setup  From the paper/appendix:      Data scale: about 20 hours of expert demonstrations per task (important nuance)    Tasks: long-horizon collaborative garment manipulation (flattening / folding / hanging, plus retrieval/handover variants)    Robots: two dual-arm systems (ALOHA-style setup; paper details Agilex Piper + ARX X5 bimanual platforms)    Sensors: 3 × Intel RealSense D435i per system (1 head-view + 2 wrist-view), 640x480 RGB    Rates:              vision/data collection/inference around 30 Hz        low-level control around 100-200 Hz              Training compute: 8 x A100 GPUs    Inference compute: RTX 4090 (appendix)    Action chunk length: K = 50 (appendix table)    A notable systems-level claim: they report running the system 24 hours nonstop from arbitrary initial states.  5. Engineering Tricks That Matter (Beyond “Advantage”)  A key takeaway from this paper is that the final gains do not come from a single learning trick.  High-impact engineering choices include:      model soup across subset-trained checkpoints (MA)    OOD validation using recovery data for model/weight selection    heuristic DAgger to front-load recovery experience    spatio-temporal augmentation for train-time coverage    deployment-side temporal chunk smoothing for latency robustness    stage annotation for stable long-horizon progress supervision    This is a strong example of replacing brute-force scaling with distribution alignment + deployment engineering.  6. Strengths      Clear and useful conceptual framing: P_train / Q_model / P_test    Strong engineering realism (latency, control buffer mismatch, recovery behaviors)    Good ablation mindset: modules are evaluated separately and in combination    Data-efficient real-robot focus (relative to foundation-model-scale retraining)    Practical validation insight: OOD validation can be more informative than in-domain validation    7. Limitations / Open Questions      Manual stage labels reduce scalability    Task family is concentrated on deformable garment manipulation; rigid-object generalization remains unclear    Performance is still bounded by demo/recovery data quality    The work does not fully evaluate retention of pre-trained priors during post-training (also acknowledged in the paper/appx discussion)    Some gains depend on high-quality engineering integration, which may be harder to reproduce than a single algorithmic module    8. My Takeaways for Robotics Research      Real-world robustness is often a distribution alignment problem before it is a model-capacity problem.    Validation on recovery / failure-adjacent data is often more useful than clean demo validation.    Deployment-side control engineering (latency mitigation, smoothing) can generate gains as large as training-side changes.    Offline advantage reweighting is a compelling, safer bridge between pure BC and full online RL for real robots.    “Post-training” in robotics should be disambiguated: χ0-style post-training is not the same thing as online RL fine-tuning.    Notes on Terminology      The paper introduce..."
    },{
      "title": "SAM-RL: Sensing-Aware Model-Based Reinforcement Learning via Differentiable Physics-Based Simulation and Rendering - RSS 2023",
      "url": "/posts/2024/02/blog-post-2/",
      "date": "2024-02-20",
      "tags": ["Robotics","Reinforcement Learning","RSS"],
      "excerpt": "Key information  Model-based reinforcement learning (MBRL) is potentially more sample-efficient than model-free RL.  Integration of differentiable physics-based simulation and rendering facilitates the learning process.  Pipeline: Real2Sim -> Learn@Sim -> Sim2Real to produce efficient policies.",
      "text": "Key information  Model-based reinforcement learning (MBRL) is potentially more sample-efficient than model-free RL.  Integration of differentiable physics-based simulation and rendering facilitates the learning process.  Pipeline: Real2Sim -> Learn@Sim -> Sim2Real to produce efficient policies.Real2Sim with Differentiable Simulation and RenderingThe integration of differentiable simulation and rendering forms the core of the model used for model-based reinforcement learning. It allows for the calculation of gradients for the simulation images (\\(I_{sim}\\)) with respect to the camera pose (\\(P_c\\)) and object attributes. This enables the updating of the model by directly comparing simulated images with real-world images and adjusting the model parameters to reduce discrepancies, thereby improving the accuracy of the simulation model for better policy generation.The whole process involves building an initial model that does not have to be precise at the first stage and updating the model with differentiable simulation and rendering. During the second half, differentiable simulation and rendering techniques are used to adjust the model parameters, including object mesh vertices, colors, and poses, to minimize the discrepancies between simulated and real images. This process involves:  Updating camera and robot poses in the simulation to match their real-world counterparts.  Generating and comparing rendered RGB-D images from the simulation with real RGB-D images, including segmentation.  Defining loss functions based on the differences in RGB images and 3D point clouds (using Earth Mover Distance for the latter) between the simulation and the real world.  Using the gradients of these loss functions to iteratively update the model’s parameters, thus reducing the discrepancy between the simulation and real-world observations.Refer to:[1] X. Zhu, et al, “Diff-LfD: Contact-aware Model-based Learning from Visual Demonstration for Robotic Manipulation via Differentiable Physics-based Simulation and Rendering”, CoRL 2023.[2] Y. Xiang, et al, “Diff-Transfer: Model-based Robotic Manipulation Skill Transfer via Differentiable Physics Simulation”Learn@SimThis stage focuses on learning sensing-aware action within a simulated environment. According to this paper, the process includes  Sensing-aware Q-function: A Q-function is adapted to include the camera pose’s impact, optimizing camera adjustments to enhance task-related information capture. This optimization is achieved by calculating gradients that suggest optimal camera positioning for action improvement.  Learning Actor in Simulation: An actor is trained through imitation learning within a simulated environment. The actor learns to perform tasks by mimicking successful actions, optimized through loss minimization that balances action efficiency and goal achievement.  Learning Sensing-aware Q Function: The sensing-aware Q function is developed using simulated trajectories, focusing on predicting the Q value from image-action pairs. This is approached as a supervised learning problem, emphasizing the role of sensing in action effectiveness.  Selecting Perception and Action: The system dynamically adjusts the camera pose to maximize the Q value, ensuring that both perception and action are optimized for task performance. This iterative process seeks the most informative viewpoint for executing the task.Sim2RealThis part starts with the policy learned in simulation, denoted as \\(\\pi_{sim}\\), which takes rendered images (\\(I_{sim}\\)) as inputs and outputs actions (\\(a_{sim}\\)). To bridge the sim-to-real gap, a residual policy \\(\\pi_{res}\\) is trained. This policy takes in real-world images (\\(I_{real}\\)) and simulated actions (\\(a_{sim}\\)), outputting a residual action (\\(\\delta a_{real}\\)). The actual action applied in the real world (\\(a_{real}\\)) is the sum of the simulated action and the residual action.The residual policy is initially set such that it outputs zero adjustments, ensuring it does not worsen the initial policy. Training involves updating this policy based on real-world executions and feedback. The aim is to adjust actions to account for differences between the simulated and real-world environments. This action is then executed in the real world, with outcomes used to further refine the policy.Experiments  Set up: Franka Panda performing the manipulation task, and Flexiv Rizon with the RGB-D RealSense camera for active sensing. PyBullet for real-world simulation, NimblePhysics and Redner as differentiable simulation and renderer.  Tasks: Peg-Insertion, Spatula-Flipping, and Needle-Threading.  Network details: CNN feature extractor and MLP heads to output the action/Q value. Residual policy network to reduce the sim-to-real gap.  Evaluation: sensing-aware Q function and its qualitative visualization, salient map of actor-network, comparison with TD3, SAC, and Dreamer, ablation, etc.My takeawaysThe paper gives a comprehensive approach highlighting an active sensing approach with differentiable physics simulation that closes that gap between simulation and the real world.Practically, the research points a potential pathway for future general-purpose robots, where active sensing enhances real-world perception, enabling robots to make more informed decisions."
    },{
      "title": "DayDreamer: World Models for Physical Robot Learning - CoRL 2022",
      "url": "/posts/2024/02/blog-post-1/",
      "date": "2024-02-06",
      "tags": ["Robotics","Reinforcement Learning","CoRL"],
      "excerpt": "Key information  This paper learns two models: a world model trained on off-policy sequences through supervised learning, and an actor-critic model to learn behaviors from trajectories predicted by the learned model.  The data collection and learning updates are decoupled, enabling fast training without waiting for the environment. A learner thread continuously trains the world model and actor-critic behavior, while an actor thread in parallel computes actions for environment interaction.",
      "text": "Key information  This paper learns two models: a world model trained on off-policy sequences through supervised learning, and an actor-critic model to learn behaviors from trajectories predicted by the learned model.  The data collection and learning updates are decoupled, enabling fast training without waiting for the environment. A learner thread continuously trains the world model and actor-critic behavior, while an actor thread in parallel computes actions for environment interaction.World model learning  The world model can be thought of as a fast simulator of the environment that the robot learns autonomously, despite that the physical robot runs in real environment.  The world model is based on the Recurrent State-Space Model (RSSM) which consists of encoder, decoder, dynamics and reward networks.  The encoder network fuses all sensory inputs \\(x_t\\) together into the stochastic representations \\(z_t\\). The dynamics model learns to predict the sequence of stochastic representations by using its recurrent state \\(h_t\\). The reward network predicts task rewards by letting the robot interact with the real world. (It appears that the decoder network is not in use in this paper.)  All components of the world model are jointly optimized by stochastic backpropagationActor-critic learningThe actor-critic algorithm learns a behavior that is specific to the task at hand, in which the actor-network decides which action to take in a given state by maximizing returns, whereas the critic-network evaluates the action by regressing the returns.ExperimentsExperiments are carried out on four different robots with different tasks.  Unitree A1 Quadruped Walking  UR5 Multi-Object Visual Pick and Place  XArm Visual Pick and Place          Rainbow algorithm in comparison        Sphero NavigationMy takeawaysThe paper gives a method in which RL can be trained in real environments aside from a simulator. A world model is trained and used for quick updates, and the data collection and learning updates are decoupled. These techniques provide insightful ideas for future reinforcement learning architectural design.Reproduction of this paper  Physical robots (May require MuJoCo or Isaac to reproduce due to lack of hardware)  Games (Easier to reproduce)"
    }]