3D ReconstructionResearchNTU CARTINHTXSCDF

Thermal Dense Mapping with Odometry-Guided Foundation Depth Estimation

Real-time dense mapping from thermal imagery, where heat radiation replaces visible texture and standard RGB-based assumptions often fail. The prototype couples a thermal-radar-inertial SLAM backbone with odometry-guided Depth Anything 3 inference, reaching about 0.5 Hz online dense reconstruction on an RTX 5090 laptop.

Overview

Thermal Imaging4D RadarInertial SLAMFoundation Depth EstimationDense Mapping

Details

Why Thermal Dense Mapping

The project targets dense reconstruction when RGB texture is unreliable or unavailable. In smoke, darkness, weak texture, or reflective indoor spaces, the robot still needs geometry that can support navigation, inspection, and operator understanding. Thermal imagery offers a different visual signal: it responds to heat radiation rather than visible appearance, so useful structural cues may remain visible when standard RGB assumptions break down.

The current system uses thermal-radar-inertial SLAM for metric motion and map alignment, then estimates dense geometry from short multi-frame thermal observations. The emphasis is not a standalone foundation-model demo, but an online robotics mapping pipeline whose output stays tied to the robot trajectory.

Odometry-Guided Foundation Depth

Early experiments with foundation geometry showed that thermal images can recover plausible scene structure, but scale and trajectory consistency are fragile if the reconstruction is detached from robot odometry. The current direction therefore guides Depth Anything 3 with calibrated thermal observations and odometry-derived camera motion.

Early VGGT-only thermal mapping prototype — Early thermal foundation-depth prototype. The result shows useful structure, but robot-aligned dense mapping requires metric motion constraints.

This keeps the role of DA3 natural: it is used as a multi-view depth estimator conditioned by thermal image sequences and robot motion, rather than as a loose single-image prior. On an RTX 5090 laptop, the current prototype runs online at about 0.5 Hz for dense thermal mapping.

System View

Simplified system view: robust odometry supplies metric motion, thermal sequences provide the observations, and foundation-depth inference produces dense geometry for map integration.

The implementation is deliberately kept modular: odometry provides motion, DA3 estimates dense depth from thermal views, and the mapping stage integrates confident geometry into a global representation. For the project page, the important point is the result: dense thermal mapping can be made online and robot-aligned, even though the sensing signal is very different from RGB.

Mapping Results

DA3 thermal TSDF reconstruction in an indoor hallway — Indoor thermal dense mapping result. The reconstruction is integrated in the robot odometry frame while retaining the thermal observations for inspection.

DA3 thermal TSDF reconstruction near an indoor stairwell — Thermal reconstruction near an indoor stairwell. More final mapping results will be added as the project media set is cleaned up.

Current Status

The prototype currently demonstrates online thermal dense reconstruction at about 0.5 Hz on an RTX 5090 laptop. The next step is to add cleaner mapping sequences and stronger result visualizations, especially for smoke and fireground-style environments.

Back to Projects