PICR

Physics-Informed Causal Reasoning for Nonstationary Robot Dynamics

PICR infers physical context from recent robot interactions and conditions a shared Lagrangian model to predict motion under changing payloads, without retraining the network.

Jungjin Lee1, Se Jin Kim1, Sung Kwon On2, Insoon Yang1,2
1Dept. of Electrical and Computer Engineering  ·  2Graduate School of AI  ·  Seoul National University
Under review
Paper · coming soon Code Demos
Isaac Sim · dynamics prediction 1× real time
What changesPayload mass changes during motion
Training rangeTrained on 0.1–2 kg; evaluated up to 4 kg
What we predictMotion 300 ms ahead

How PICR works

Infer the context, keep the physics

A robot that picks up something heavier suddenly obeys different dynamics. Instead of retraining a model every time that happens, PICR looks at the last few moments of state and torque and infers a compact context that describes the current physical condition.

That context conditions a single Lagrangian dynamics network through FiLM. It modulates the learned inertia matrix and potential energy, and the Coriolis and gravitational forces are still derived from them, so the structure of the equations of motion holds for every context.

Recent interactionsshort window of states and torques
Inferred contextre-inferred online
Lagrangian mechanicsshared network, FiLM-conditioned
Predicted motionunder the applied torques

Adapt the inferred context, not the network weights.

The network is trained once, offline. A physically structured contrastive objective organizes the context by payload properties during training, but payload mass is not provided at inference: PICR estimates the dynamics-relevant context from recent interactions alone.

Result 1

Predicting under changing payloads

In the demo at the top of the page, a Franka arm traces a 3D figure-eight while its payload grows from 0.1 to 4 kg, well past the 2 kg it saw in training. At every moment, each model predicts where the cube will be 0.3 s later. The top row overlays predicted cubes (coloured) on the true cube (grey); the bottom row accumulates prediction errors.

Payload mass is not provided at inference. PICR estimates it implicitly from how the arm has been moving and keeps the lowest error both inside and beyond the training range.

Cube ADD (cm) ↓PICRRWMCaDMCaDeLaCDeLaNNeural-Fly
Seen (0.1–2 kg)4.587.096.799.076.827.66
Unseen (2–4 kg)6.9211.6310.9024.7525.6524.78

Selected baselines are shown in the video; the table gives the full comparison.

PICR achieves the lowest prediction error among the evaluated models in both payload ranges. On unseen payloads it reaches 6.92 cm ADD, compared with 10.90 cm for the best baseline.

Result 2

Predicting what happens when you push

A good dynamics model should answer "what if" questions. Here the state and payload are identical, and only joint 2, the shoulder joint that swings the whole arm forward and back (second from the base), receives an extra ±30 N·m of torque. Does the model know which way, and how far, the arm will move?

Franka arm with joint 2, the shoulder joint, and its rotation axis marked
Where the push is applied: joint 2 rotates the whole arm about the dashed axis.
≈13× slow motion
Simulator outcome (opaque) vs. PICR prediction (orange) from three viewpoints, over 0.3 s. Predictions are conditioned on the measured applied torques in each branch.

Across intervention sizes from ±10 to ±40 N·m, PICR keeps a stable acceleration response, while baselines lose response at larger offsets.

Result 3

What the context represents

Payload-based supervision organizes the context during training. At inference, PICR recovers the dynamics-relevant context from interaction history alone. The following experiments show how this context changes and how it influences predictions.

Same robot, same torques, different context

The robot is carrying 3 kg. We keep the initial state and the future torques fixed and change only the context, then predict 0.24 s ahead. With a context from a lighter payload, the predicted arm drifts as if the object were lighter.

Context from a lighter payload (0.1 kg) · error 25.8 cm
Context inferred from the current interaction · error 3.0 cm
Grey: real arm. Purple / orange: predicted arm. The left plot in each video marks where the inserted context lies. Both videos play in sync.

The model applies different dynamics depending on the context, so the same action leads to a different predicted outcome.

The context separates right after a lift

The robot lifts 0.1, 1.19 and 2 kg objects. Before the lift the three contexts overlap; within about 0.4 s after lifting, each moves to the region of its own mass.

Top: robot playback. Bottom left: context plane. Bottom right: first principal component over time.

The context follows a payload that keeps changing

During figure-eight tracking, the payload is slowly changed from 4 to 0.1 kg and back from 0.1 to 4 kg. The context moves along with it in real time.

2× speed
Two episodes; line colour is the true payload mass at each moment. Watch at real time (1×).

Result 4

Looking inside the learned mechanics

Because PICR predicts torque as τ = H(q, z)q̈ + c + g, we can open it up and look at each term. Without direct supervision on individual mechanical components, PICR learns gravity and inertial terms that approximately agree with simulator references, even while carrying an unseen 3 kg payload.

first 1 s at 5× slow motion
Joint-2 gravity, inertial and Coriolis terms of PICR vs. the simulator and structured baselines (CaDeLaC, DeLaN), carrying 3 kg.

When the payload switches mid-episode, PICR's gravity and inertia terms reach 90% of the reference change within 200 ms.

Resources

Paper, code and citation

PaperUnder review. The preprint link will appear here.
Code & trained modelsTraining and evaluation code, the trained PICR model, and the scripts behind the paper results: github.com/jungbbal/PICR
DataThe simulation datasets are not publicly available yet. The repository lists the dataset ids and where each file goes.