NovaPunk LLC
← All services

02 / Physics World Models

Physics World Model Applications

Rehearse physical actions before executing them.

NovaPunk builds simulation environments, outcome models, and planning tools for physical systems. When outcome labels are unavailable, a defined physical environment can generate training examples and provide a separate basis for evaluating a plan.

Discuss this service

Built for your requirements.

The problem

A harvest planner needs to decide which fruit to attempt, in what order, and with which gripper configuration while protecting the stem. Public detections provide geometry but do not label the outcome of a picking attempt.

What we can build with you

  • A simulation environment with explicit physical assumptions
  • Simulation-generated training and validation data
  • Outcome and state-transition models
  • A planning interface with evaluation results

A model of the action and the world it changes.

The harvest example separates physical simulation, learned prediction, and planning. Removing one fruit can change the neighbours, so a sequence must account for state changes as well as individual pick outcomes.

Physics-consistent environment

MuJoCo scenes represent the fruit, segmented stalk, competing separation points, and a force-limited gripper. Approach, grasp, bend, twist, and release are simulated rather than assigned an outcome in advance.

Outcome and transition models

The outcome model predicts six pick classes. A separate transition model estimates whether neighbouring fruit remain tilted after a pick; known gravity behavior resolves the unambiguous cases.

Sequence-aware planning

The planner selects stations, picking order, and gripper configuration. Neighbour state updates help the plan account for a scene that changes after each action.

Independent simulation replay

Selected actions are replayed in MuJoCo with the learned outcome model excluded from scoring. The report records 3,912 validation picks and shows both agreement and prediction errors.

Internal research / prototype

AI Harvest Planner

An internal research prototype for robotic apple harvesting. It combines a MuJoCo environment, learned outcome and transition models, and a planner. The supplied report documents simulation validation; it does not report an orchard field trial.

Recorded videos and figures illustrate the research. No live agent, model, or simulator runs on this page.

Inputs and data

  • FRESH RGB-D detections: position, fruit dimensions, and stem direction
  • 8,133 detections retained after filtering
  • MuJoCo scenes with force-limited grasping and bend/twist actions
  • Four gripper apertures evaluated from the same settled state

Example scope

The report documents separate checks of physical calculations, invariants, pipeline consistency, and plan replay. In the reported operating configuration, predicted success was 0.788 and realised simulation success was 0.730. That gap illustrates why classification accuracy alone is insufficient for evaluating a planner.

01

Reconstruct

Create a physical scene from fruit geometry.

02

Simulate

Try paired actions from the same initial state.

03

Learn

Fit outcome and neighbour-transition models.

04

Plan

Choose stations, sequence, and gripper settings.

05

Validate

Replay the selected plan in MuJoCo.

Geometry → physical scene → simulated actions → labels → planner

A single fruit is evaluated across four apertures from the same saved state. The resulting labels describe simulated outcomes.

Five MuJoCo frames showing approach, grasp, bend and twist, and stem-intact detachment
Recorded MuJoCo frame sequence from Physics World Models, Figure 6 (page 8). This is a simulation example, not footage from a physical robot. Select the figure to view it at full size.

The simulation in motion.

Recorded MuJoCo simulations from the harvest research. Select play to watch each sequence; these recordings do not operate a physical robot.

Single-pick simulation

Annotated MuJoCo recording of one picking attempt. The physical action and resulting state illustrate the outcome-prediction problem described in the report. Open video separately

Two-pick sequence

Annotated recording of successive picks. Watch how the remaining fruit and contact conditions change after the first action, motivating the neighbour-transition model. Open video separately

Neighbour dynamics - prediction vs simulation

Tree 23 comparison: the dynamics model predicts the remaining fruit state on the left; recorded MuJoCo replay appears on the right. The one-survivor case settles vertically, illustrating how known physical behavior and learned transitions are evaluated separately. Open video separately

Final simulation example - a complete harvest plan

Tree 23 harvest-plan visualization shows the simulated robot moving between planned stops and picking fruit in sequence with its gripper. The on-screen counters track stops, picks, and harvested fruit. This is a simulation visualization, not a field trial; the canopy shell and trunk are drawn for orientation and are not part of the physical model. Open video separately

What the study reports.

31,960Simulation-generated pick labels
94.07%Reported held-out fruit classification accuracy
3,912Picks replayed for plan validation

Metrics are reproduced from the supplied research or original project account. They have not been independently rerun for this website.

Boundaries and next work

  • No field trial or calibration against a particular orchard is reported.
  • Leaves and branches are excluded; canopy density includes assumptions.
  • Perception and robot motion control are outside the planning module's scope.
  • Block-level routing, shift scheduling, and hardware integration remain separate work.
  • The static page neither executes MuJoCo nor controls hardware.

Explore the engineering.

Figures and reported results are drawn from Physics World Models - Sungwon Son, September 2026.

What would you like to build?

Tell us about your workflow, available data, and the decision or task you want to improve.

Contact NovaPunk