labelrefinery

Seeing the data and the labels

The scene and every round of labels are published. Both open in Foxglove with no custom panel and nothing to install.

Open it in the browser

The scene and every round of labels open in Foxglove on the demo site — nothing to download, nothing to install. The 86 MB recording is read over HTTP range requests, so playback starts without pulling the whole file.

Open the scene on the demo →

Download

curl -O https://samples.magmalake.org/sitegen/v0.2.0/site_seed1_60s.mcap    #  86 MB
curl -O https://samples.magmalake.org/sitegen/v0.2.0/labels_rounds.mcap     # 471 KB

foxglove site_seed1_60s.mcap labels_rounds.mcap    # or drag both in at once

Foxglove merges multiple local files into a single playback timeline, so the labels line up with the recording frame for frame. Predictions live in their own file on purpose: the recording stays immutable, and one 86 MB scene serves any number of pipeline runs.

What to turn on

Add a 3D panel and set display frame to map. Then:

TopicSchema
/lidar/points foxglove.PointCloud the sweep — set colour by to intensity
/ego/joint_states foxglove.JointStates swing, boom, stick, bucket
/camera/<name>/image foxglove.CompressedImage four cameras at the house corners
/terrain/heightmap foxglove.PointCloud ground and stockpiles, published once
/ground_truth/actors foxglove.SceneUpdate held out — the oracle cuboids
/pred/* foxglove.SceneUpdate one topic per pipeline round, from the labels file

Scrub to t ≈ 25 s: round 0 has objects standing on the stockpile and round 2 does not. That is the 824 → 136 false-positive drop, visible rather than tabulated. Add a Plot panel on /ego/joint_states for the dig cycle — it repeats every 15 s, and the flat stretch at t ≈ 26–34 s is the machine walking to its second station.

/ground_truth/* is the oracle. It is in the file so you can see what a labeler was up against — a labeler must never read it. Keeping the truth and the data in one recording is deliberate: the same pass wrote both, so the truth cannot drift from what it describes.

What is in the scene

A 20-tonne excavator running a dig–swing–dump cycle, loading an articulated hauler that backs in, is loaded and hauls off before a second truck takes its place. Part-way through, the machine walks 6.5 m along the trench to a second dig station — house squared up, boom tucked — and the hauler repositions with it. A spotter holds station; a second worker crosses the swing radius. Two stockpiles sit at the angle of repose, a row of grade stakes runs along the north edge, and dust kicks up when the loaded truck pulls away.

Four things are deliberate:

The excavator is articulated. Base pose plus swing, boom, stick and bucket — four degrees of freedom on top of the body. Every link is its own ground-truth box, so a scorer can ask whether a labeler got the bucket right, which is the part that will actually hit something.

Proprioception is a published topic. The machine's own joint angles are free, exact labels. The LiDAR rides the house, so 15% of every sweep is the ego looking at itself, and forward kinematics is what removes it before anything else runs.

The difficulty is in the sensor, not the scene. Points fall off as 1/r², range noise grows with distance, dropout rises quadratically and dust eats returns over a region. A worker at 14 m gets about 20 points.

Nothing is balanced. Terrain is ~83% of returns; the two workers together are ~0.3%. Real class imbalance, not a curated benchmark.

Reproducing it

The scene is deterministic in its seed — two runs produce byte-identical files, verified by checksum, so two pipelines under comparison consume exactly the same input.

git clone https://github.com/labelrefinery/sitegen
cd sitegen && make install

uv run sitegen generate --out site.mcap --seed 1 --duration 60 --camera-hz 1
# sha256 b3140981...  the published file, byte for byte

sitegen overlay writes a label file from any tracker-schema CSV, so your own pipeline's output can be viewed the same way.