Counterfactual Simulation-Guided Physical Reasoning for Physics-Based Video Generation

Anonymous Submission

ECCV 2026 Workshop 3D in the Era of World Models

Motivation

Comparison of single-image, video-guided, and counterfactual-simulator-guided physical reasoning paradigms
Comparison of physical reasoning paradigms. Single-image reasoning relies on static visual evidence, while video-guided reasoning incorporates generated motion as additional dynamic evidence. Our counterfactual-simulator-guided reasoning instead uses controlled physical interventions to provide physically grounded, parameter-specific evidence.

Comparisons with Baselines

Side-by-side comparisons of our counterfactual-refined result with the corresponding baselines. Select an input scene and choose a comparison method below.

RealWonder + Ours
RealWonder + PhyMAGIC

Ablation Study

Input Image
Input image of a multicolored flag
(a)
(b)

Ablation of the Counterfactual Physical Reasoning Module. (a) Ours without the Counterfactual Physical Reasoning Module and (b) Ours. Removing the module leads to unstable dynamics, while the full model produces coherent, instruction-aligned physical behavior.

Quantitative Evaluation

Following the evaluation protocol of PhyMAGIC, we report CLIP Similarity to measure semantic alignment between the generated motion and its text description, Aesthetic Score to assess the visual quality of the generated video, and PhysReal. Higher values indicate better performance for all metrics.

Method CLIP Sim. ↑ Aesthetic ↑ PhysReal ↑
RealWonder 0.269 53.60 0.624
RealWonder + PhyMAGIC 0.273 54.28 0.635
RealWonder + Ours 0.278 55.14 0.647

Bold: best. Higher is better for all metrics.

Human Study (2AFC)

We conducted a two-alternative forced-choice (2AFC) user study with 50 participants to compare our method with physics-based video generation baselines. Values indicate the percentage of participants who preferred RealWonder + Ours in terms of physical plausibility and instruction alignment. Higher values are better.

Ours vs. RealWonder

81.0% Physical Plausibility
85.1% Instruction Alignment

Ours vs. RealWonder + PhyMAGIC

80.0% Physical Plausibility
83.8% Instruction Alignment


Abstract

Physics-based video generation combines physics simulators with generative models to produce visually realistic, physically grounded dynamics. However, the resulting motion critically depends on simulator parameters, whose estimation from a single image remains inherently ambiguous. We introduce counterfactual physical reasoning, which reasons about physical parameters through controlled interventions and their resulting dynamics. Across 30 scenes, our module improves instruction alignment, visual quality, and physical realism over both baselines, and receives 80.0–85.1% preference in a 50-participant 2AFC study.

Overview

Overview of physical scene initialization, physics-based generation, counterfactual rollout, and effect-guided refinement
Overview of our Counterfactual Physical Reasoning framework. When the current dynamics do not satisfy the user instruction, we construct parameter-wise interventions and simulate their counterfactual outcomes. Comparing the factual and counterfactual dynamics provides physically grounded, parameter-specific evidence for effect-guided refinement toward the intended behavior.