[Paper Notes] OmniContact: Chaining Meta-Skills via Contact Flow for Generalizable Humanoid Loco-Manipulation
Published:
OmniContact proposes a compact interface for long-horizon humanoid loco-manipulation. Instead of asking a planner to generate dense whole-body motion, it represents a future skill using sparse body targets and binary contact schedules. A learned controller executes this contact flow, while a rule-based planner regenerates it online as the object state changes.
My read: the central contribution is the boundary between planning and control. CF-Gen converts object-level goals into contact-aware references through hand-designed phase templates, geometry, IK, and interpolation. CF-Track is a unified reinforcement-learning policy that turns those references into low-level humanoid actions. Contact flow is the shared language that makes carrying, pushing, kicking, recovery, and skill chaining fit into one system.
Paper Info
The paper is “OmniContact: Chaining Meta-Skills via Contact Flow for Generalizable Humanoid Loco-Manipulation” by Runyi Yu, Xiaoyi Lin, Ji Ma, Yinhuai Wang, Koukou Luo, Jiahao Ji, Huayi Wang, Wenjia Wang, Runhan Zhang, Ping Tan, Ting Wu, Ruoli Dai, Qifeng Chen, and Lei Han. The paper, project page, official MuJoCo execution repository, and OmniContact dataset are public.
The Problem: What Should a High-Level Planner Command?
Long-horizon loco-manipulation has two coupled problems. The robot needs a robust controller for individual interactions, and it needs a mechanism for composing those interactions while reacting to failed contacts or displaced objects.
Dense human-object trajectories precisely specify behavior, but they are expensive to synthesize and brittle under online changes. A symbolic command such as carry(box, target) is easy to plan, yet gives the low-level controller little information about where and when contact should happen. Latent skill embeddings are compact, although their semantics are difficult for an external planner to inspect and modify.
OmniContact chooses an intermediate representation: preserve the small set of future body targets required to express motion intent, and explicitly attach the contact timing that distinguishes manipulation from ordinary locomotion.
Contact Flow
At time (t), contact flow is defined as
[ F_t={(b_{t+k},c_{t+k})}_{k\in\mathcal{T}}, ]
with non-uniform future offsets
[ \mathcal{T}={0,1,2,3,4,8,12,16,24,32,50}. ]
Here (b_{t+k}) contains sparse future body targets, primarily torso, wrists, and ankles. The contact vector
[ c_{t+k}\in{0,1}^{4} ]
specifies binary contact states for four robot end effectors. Non-uniform sampling allocates more targets to the immediate future and fewer to longer-term intent.
This representation carries two kinds of information:
- Where the body should move: sparse future torso and end-effector targets.
- When interaction should occur: explicit contact/no-contact state for each end effector.
Contact flow does not contain contact force, pressure, or a dense physics trajectory. It describes intended contact mode and sparse kinematic goals. The low-level policy retains freedom to choose the remaining whole-body motion needed for balance and compliance.
The ablation strongly supports this design. Tracking torso and end effectors without contact reaches only 11.5% success on the CF-Gen Carry Box evaluation. Adding binary contact raises success to 98.7%. Adding dense object, joint-DoF, or full-body targets then hurts generalization because the controller becomes overconstrained. The successful interface is the compact set torso + end effectors + contact.
CF-Track: The Learned Low-Level Executor
CF-Track is one unified policy for locomotion, carrying, pushing, sliding, relocating, kicking, and their transitions. Its input combines the target contact flow with a five-step observation history:
[ x_t=[F_t,H_t], \qquad H_t=[o_t,\ldots,o_{t-4}]. ]
Each observation includes robot proprioception, base orientation, end-effector positions, the previous action, and the object’s relative 6D pose and bounding box. The policy outputs low-level motor actions at 50 Hz. The actor and critic use Transformer networks.
CF-Track is trained with reinforcement learning using three reward families:
[ r_t=lambda_{\text{track}}r_t^{\text{track}} +\lambda_{\text{amp}}r_t^{\text{amp}} +\lambda_{\text{reg}}r_t^{\text{reg}}. ]
The tracking term follows sparse body, object, and contact-flow targets. The AMP adversarial motion prior encourages human-like whole-body motion. The regularization term penalizes abrupt action changes. The selected balance places most weight on tracking while retaining enough motion prior to smooth the artifacts of rule-generated references.
Pure tracking achieves relatively accurate motion but reduces disturbance stability. Giving the motion prior too much weight produces stable-looking behavior with almost no task success. This is a genuine control trade-off: the policy must obey contact intent strongly enough to finish the task while preserving enough behavioral flexibility to absorb imperfect plans and perturbations.
CF-Gen: Rule-Based Contact-Flow Synthesis
CF-Gen is described as a lightweight, heuristic planner. It is not a learned trajectory generator. Each meta-skill has a hand-designed sequence of phases. Carrying, for example, is decomposed into approach, move to a hand-contact pose, lift, walk while maintaining contact, and release.
For each phase, CF-Gen performs four operations.
1. Select object-centric contact anchors
The planner uses object pose and dimensions to choose suitable contact faces and approach directions. It adapts wrist, torso, and ankle targets to object geometry, enabling the same carry or push template to handle different object sizes and initial poses.
2. Generate keyframes with selective IK
Locomotion phases mainly specify ankle goals and a nominal posture. Contact phases specify ankle and wrist targets, then solve constrained IK over pelvis height, pelvis pitch, and robot joints. The solver is limited to 20 iterations for real-time synthesis.
3. Interpolate a dense kinematic reference
Positions and joint values use linear interpolation, while pelvis orientation uses SLERP. This creates a continuous reference between phase keyframes.
4. Convert the reference into sparse contact flow
CF-Gen samples the interpolated trajectory at the non-uniform future offsets and transforms the targets into the current torso-yaw frame. CF-Track receives only the resulting sparse body targets and binary contact schedule.
This sequence is important: CF-Gen temporarily creates a dense reference internally, but the controller does not directly track the full joint trajectory. Contact flow discards unnecessary constraints before execution.
Closed-Loop Chaining and Recovery
CF-Gen maintains a finite stage state containing the active object, selected meta-skill, phase, and target. When a phase finishes, it generates the next contact-flow segment. Multi-stage tasks are built by switching objects and meta-skills at stage boundaries.
During execution, CF-Gen monitors the observed and predicted object state at 50 Hz:
[ \delta_t=d(x^{\text{obj}}{t,\text{obs}},x^{\text{obj}}{t,\text{pred}}). ]
If (\delta_t) crosses a threshold, the planner aborts the current reference and replans from the observed state. A dropped or displaced box can therefore trigger a new approach and contact sequence.
Online replanning raises Push Suitcase from 82.5% to 94.5%, Stack Boxes from 56.6% to 80.5%, and Push-Stack Boxes from 76.5% to 84.5%. The paper also notes an important nuance: most of this gain comes from refreshing future segments with updated object states. Full emergency recovery is triggered relatively rarely.
Under deliberately injected failures, recovery reaches 92.5% after dropping a carried box, 97.5% after offsetting its pose, and 89.5% after offsetting a pushed suitcase, with roughly 1.5-1.8 replans per trial.
OmniContact Dataset
The authors collect a synchronized MoCap human-object interaction dataset tailored to long-horizon loco-manipulation. It contains:
- 1,274 valid sequences and 22.29 hours of motion;
- 7.22 million object frames captured at 90 Hz;
- synchronized BVH human motion and rigid-object 6D trajectories;
- carry, push, relocate, slide, and kick primitives;
- primitive-level contact modes and task-level language descriptions.
The average sequence lasts 62.98 seconds and moves the object 19.76 meters, emphasizing long-range transport. The dataset has fewer objects and sequences than OMOMO, but longer clips, higher temporal resolution, and explicit loco-manipulation structure. Scaling CF-Track training data from 2.2 to 22.3 hours improves both success and object accuracy.
The dataset does not provide measured contact forces. Contact supervision is constructed from synchronized body/object trajectories, binary contact modes, and dynamic/static phase annotations.
Main Results
In the randomized simulation benchmark, OmniContact reaches:
- 98.7% on Carry Box;
- 82.5% on Push Suitcase;
- 56.5% final-stage success on stacking three boxes;
- 76.5% final-stage success on Push-Stack Boxes.
The baselines perform much worse on interaction and chaining. PhysHSI reaches 87% on Carry Box but fails at the final stacking stage. LessMimic reaches 34% on Carry Box and 12.5% on Push Suitcase, with 0% final success on the chained tasks. OmniContact’s main empirical advantage is therefore composition under object-pose variation, not basic locomotion; several baselines already perform well on locomotion-only evaluation.
The additional tasks include Relocate Ball, Slide Box, Kick Ball, Relocate-then-Kick, and Push-then-Relocate. Long-duration experiments show perfect 40-minute survival when repeatedly resampling goals for one object under the easiest protocol. Survival drops to 29.5% at 40 minutes when both initial and final object goals are repeatedly resampled, and becomes harder with multiple interacting objects. These extended results give a more calibrated picture than the headline 40-minute demonstration.
Real-World Deployment
The learned policy is deployed on a Unitree G1. The controller runs at 50 Hz, and a Noitom motion-capture system tracks the robot pelvis and manipulated object at 100 Hz to provide global poses. Before hardware deployment, the policy is checked through Isaac Lab-to-MuJoCo sim-to-sim transfer.
The real-world demonstrations show that the controller can execute the behaviors on hardware, but the main quantitative tables are simulation results. The deployment also relies on external MoCap for robot/object pose; this is not yet a vision-only autonomous loco-manipulation system.
The public repository currently focuses on MuJoCo execution. It includes an ONNX policy, CF-Gen references, scripted skill and skill-chaining runners, and an interactive joystick-based state-switching path that mirrors sim-to-real deployment.
VLM Integration
A VLM can receive a rendered scene and language instruction, then output an ordered list of object-level goals and available meta-skills. For example, it can decompose “arrange the boxes into a heart” into repeated carry/place subgoals. CF-Gen grounds each subgoal into contact anchors and contact flow; CF-Track executes it.
The VLM never predicts joint trajectories, contact timing, or low-level actions. Its role is semantic decomposition. The paper presents this integration qualitatively, including heart-shaped layouts and multi-object arrangements. These examples demonstrate interface compatibility; an end-to-end trained vision-language-control model remains outside the paper’s scope.
What Is Actually New?
OmniContact combines several familiar components: MoCap imitation, AMP-style reinforcement learning, hand-designed state machines, object-centric geometry, IK, interpolation, and online replanning. Its main design contribution is how these components communicate.
Contact flow is compact enough for a planner to synthesize and edit online, explicit enough to represent contact timing, and sparse enough to leave the RL controller freedom over whole-body execution. The system separates:
- What interaction should happen: meta-skill, object goal, contact anchors, and binary contact schedule.
- How the humanoid physically realizes it: balance, gait, compliant adaptation, and low-level motor actions.
The strongest ablation is the jump from 11.5% to 98.7% after adding explicit contact state to torso and end-effector targets. That result supports contact flow as a useful planning-control interface more directly than the headline VLM examples do.
Limitations
CF-Gen depends on manually designed phase templates, contact-anchor rules, thresholds, and task-specific geometry logic. The authors report that it struggles in highly dynamic settings and propose learning contact-anchor generation in future work. The G1’s underactuated grippers restrict fine manipulation, and contact flow has not yet been extended to dexterous hands.
The approach assumes access to accurate object pose and dimensions. Real deployment uses external MoCap, while VLM demonstrations operate on rendered scenes and qualitative task decomposition. Broader visual perception, uncertain geometry, force-aware contact, and unstructured real-world recovery remain open. The baseline comparison also requires task-specific adaptations because several methods do not natively support autonomous approach, release, or chaining.
Takeaway
OmniContact is best understood as a hierarchical humanoid control interface. It does not ask one model to invent long-horizon motion from language. A semantic planner selects object-level goals; CF-Gen turns them into sparse future body targets and contact schedules; CF-Track converts those targets into stable whole-body control; observed object-state errors close the loop through replanning.
The useful idea is simple: for loco-manipulation, sparse pose targets are too ambiguous, while dense whole-body trajectories are too restrictive. Adding an explicit contact timeline creates a middle representation that can be planned, composed, tracked, and repaired online.