A WORLD-MODEL RESEARCH PROJECT

Does LeWorldModel Preserve Local Action Geometry?

If we change a robot’s action just a little, will a world model predict how much the result changes?

Vasil Vasilev · Siddharth Nayak

Read on arXiv ↗ Hugging Face paper page ↗

THE QUESTION

Not all small action changes have the same effect.

One small change might move a robot arm a lot. Another might barely move it. We want the model to know the difference.

We took recorded actions in two simulated tasks—Reacher and Cube—and nudged them in different directions. Then we compared what really happened in the simulator with what LeWorldModel predicted. Both outcomes were measured using the same image encoder.

The paper calls this pattern of responses local action geometry. It describes which action changes matter most, and by how much.

WHAT WE FOUND

The model gets part of the picture, but not all of it.

The strongest action direction in the model differs from the simulator by about 36° on Reacher and 42° on Cube at our smallest tested action change. These are typical values across 458 Reacher and 500 Cube states—not errors for every individual state.

Blue simulator and orange model ellipses point in different directions in example Reacher and Cube states.
Which actions matter most? Blue is the simulator; orange is the model. These examples show the difference in direction. The Cube drawing shows only two of its five action dimensions.

The size of the change matters, too.

When we made the test action change ten times larger, the model and simulator looked closer: their typical geometry difference fell by about 70% on Reacher and 47% on Cube. So there is no single answer that works at every scale.

Plots show that model and simulator action sensitivity looks more alike when the tested action changes are larger.
How the measurements change across three sizes of action change. Lines show typical results; shaded areas show the middle half of states.

Can it predict changes we did not use in the measurement?

We also tested new action directions. At the smallest scale, the simulator-based measurement predicted the size of these changes better than a simple equal-in-all-directions guess. The model-based measurement did worse than that guess: 33% vs 26% error on Reacher and 56% vs 38% on Cube.

These numbers measure response size in the model’s image representation. They do not tell us whether the model picks the right action to reach a goal.

See the held-out test plots
Four plots compare how the action response changes with scale and how well each method predicts the size of responses to new actions.
The left plots compare physical pose, pixels and image embeddings. The right plots show prediction error for action directions not used to build the measurements. The model-granted-true-scale line uses simulator information and is not a deployable method.

THE TAKEAWAY

Knowing how much is not the same as knowing where.

The model captures some useful patterns, but it can still get the strength or direction of an action’s effect wrong. This study checks one pretrained model for each task. It does not show that improving this measurement would improve planning or robot control.

The tests use newly generated, internally consistent simulator runs based on the original dataset starting points. They are not an exact replay of the original runs.

PAPER & LINKS

Read the full study.