AgentChord

Robotics: Science and Systems 2026

From Reaction to Anticipation Proactive Failure Recovery through Agentic Task Graph for Robotic Manipulation

A recovery-aware task-graph system that helps real robots anticipate disturbances, switch to prepared recovery branches, and keep long-horizon manipulation moving.

Paper Code
AgentChord icon AgentChord

From Reaction to Anticipation: Proactive Failure Recovery through Agentic Task Graph for Robotic Manipulation

AgentChord prepares recovery branches before execution. During rollouts, compiled monitors trigger the right transition immediately, avoiding repeated online re-planning when manipulation tasks are disturbed.

Sheng Xu1 Ruixing Jin1 Huayi Zhou1 Bo Yue1 Guanren Qiao1
Yueci Deng1 Yunxin Tai2 Kui Jia1,2 Guiliang Liu1,3,*
The Chinese University of Hong Kong, Shenzhen logo

1 The Chinese University of Hong Kong, Shenzhen

DexForce Technology logo

2 DexForce Technology

Shenzhen Loop Area Institute logo

3 Shenzhen Loop Area Institute

* Corresponding author

Abstract

AgentChord treats a manipulation task as a directed task graph, then enriches that graph before execution with failure-specific recovery branches. A composer builds semantic sub-goals, an arranger anticipates likely failures and inserts corrective routes, and a conductor compiles the graph into executable, interruptible transitions with low-latency monitors.

During execution, the robot evaluates multimodal geometric and gripper-state signals. When a persistent deviation is detected, AgentChord immediately switches to the corresponding pre-compiled recovery branch instead of re-planning from scratch or backtracking through the task.

Contributions

  1. Recovery-augmented task graph. A unified graph representation for task structuring, execution compilation, failure anticipation, and online recovery.
  2. Forward-moving recovery. Anticipated recovery branches are filtered so they preserve task progress and avoid regressive resets.
  3. Real-world evidence. Experiments across simulation and six real-world tasks show higher success rates and shorter execution time than reactive recovery baselines.
AgentChord overview showing nominal actions, disturbances, and proactive recovery actions
AgentChord anticipates recovery actions before execution. Blue trajectories are nominal actions, red marks failures or disturbances, and pink trajectories are proactive recovery actions prepared by the system.

Method

A choreographed agentic task graph

The system separates task reasoning, recovery reasoning, and execution compilation, then brings them back together as a single recovery-aware graph that can be monitored and interrupted online.

AgentChord framework with task structuring, recovery orchestration, and execution compilation agents
The framework constructs a nominal graph, augments it with anticipatory recovery branches, and compiles both nominal and recovery transitions into executable programs with monitors.

Composer

Task Structuring Agent

Builds the nominal directed graph from the task instruction and initial observations.

Arranger

Recovery Orchestration Agent

Predicts likely failure modes, creates online-detectable triggers, and adds recovery branches.

Conductor

Execution Compilation Agent

Compiles node keyframes, edge programs, and monitors into interruptible robot behaviors.

1

Directed task graph

Nodes encode semantic sub-goals. Edges encode constraint-aware motion transitions.

2

Recovery augmentation

Failure modes receive recovery nodes, recovery edges, and feasible downstream merge targets.

3

Hierarchical solvers

Node-level and edge-level constrained solvers realize both nominal and recovery transitions.

4

Compiled monitors

Persistent feature violations trigger immediate switching to a pre-compiled recovery edge.

Results

Higher success with lower recovery latency

AgentChord is evaluated on simulated disturbances, real-world bimanual manipulation, repeated perturbations, and recovery-aware policy learning.

99.2% Average simulation success rate across three disturbance-heavy tasks.
41.5s Average simulation execution time, lower than DRM, ReKep, and CaM.
77.5% Average real-world success rate across six disturbed manipulation tasks.
39/50 Policy success after fine-tuning with AgentChord-generated recovery trajectories.

Simulation

Average over three simulation tasks with external disturbances

Method Success Time Steps
IM 79.2% - -
DRM 92.5% 105.9s 373
ReKep 92.5% 54.4s 391
CaM 97.5% 78.1s 356
AgentChord 99.2% 41.5s 352

Real World

Average over six long-horizon manipulation tasks with external disturbances

Method Success Time
IM 59.2% -
DRM 66.7% 143.5s
ReKep 65.0% 107.1s
CaM 72.5% 130.9s
AgentChord 77.5% 92.2s
Representative AgentChord recovery examples in simulated manipulation tasks
Simulation recovery visualizations with object displacement and task continuation.
AgentChord real-world failure recovery process across six tasks
Real-world recovery examples across pouring, rearrangement, handover, folding, and tray setup tasks.
Object instances for the six evaluation tasks
Object instances and task settings used for generalization tests.
Dual-arm pour water stress test with repeated recovery transitions
Stress test with repeated disturbances in dual-arm pouring.
Initial and final states of AgentChord across real-world tasks with randomized objects and poses
Real-world trials across six tasks with varied object instances, poses, and outcomes.
Dual-arm pouring results in cluttered tabletop scenes
Dual-arm pouring in cluttered scenes with different bottles and target cups.

Paper

Read and cite the work

Use the publication link, visit the repository, or copy the BibTeX entry for citation.

Open Paper Code Repository 
@inproceedings{xu2026agentchord,
  title = {From Reaction to Anticipation: Proactive Failure Recovery through Agentic Task Graph for Robotic Manipulation},
  author = {Xu, Sheng and Jin, Ruixing and Zhou, Huayi and Yue, Bo and Qiao, Guanren and Deng, Yueci and Tai, Yunxin and Jia, Kui and Liu, Guiliang},
  booktitle = {Robotics: Science and Systems (RSS)},
  year = {2026}
}