Skip to content

DeepLens Hyperspectral Imaging

DeepLens Hyperspectral Imaging is a simulated snapshot-hyperspectral reconstruction example built on the vendored DeepLens optics code in its source repository. A 31-band CAVE spectral cube (400–700 nm in 10 nm steps) is blurred one band at a time by a DiffractiveLens with a fixed Pixel2D DOE. The configured BFS-U3-200S7C-C response curves collapse those bands to three channels, and NAFNet reconstructs the 31-band cube.

31-band CAVE cube (400–700 nm)
  ↓ fixed Pixel2D spectral PSFs
simulated RGB response (3 channels)
  ↓ NAFNet
31-band reconstructed cube

Current workflow

The released source contains one reconstruction workflow:

  1. 0_hello_deeplens_hsi.py visualizes the configured DOE phase and its PSF at 400, 500, 600, and 700 nm.
  2. 1_deeplens_hsi.py trains NAFNet against captures simulated with that fixed DOE and sensor response.

The optical renderer is differentiable, but the shipped trainer passes only the network parameters to AdamW. It does not optimize the DOE. Sensor noise, Bayer sampling, and an ISP are also not applied in this example, so the three-channel tensor is a simulated response rather than a complete raw-camera capture.

Code structure

DeepLens_Hyperspectral/
├── 0_hello_deeplens_hsi.py       # DOE phase and spectral PSF visualization
├── 1_deeplens_hsi.py             # Fixed-DOE NAFNet training
├── hsi_dataset.py                # CAVE loader and augmentation
├── configs/1_deeplens_hsi.yml    # The single training configuration
├── ckpts/nafnet_hsi.pth          # Included NAFNet checkpoint
├── lenses/paraxiallens/          # Pixel2D DOE lens and phase map
├── sensors/flir/                 # RGB response configuration
└── src/
    ├── hsi_camera.py             # Spectral PSF rendering and RGB response
    ├── deeplens/                 # Vendored optics snapshot used by this project
    ├── sensor/                   # RGBSensor response model
    └── network/                  # NAFNet and training losses

The vendored src/deeplens/ snapshot is the implementation used by this project and can differ from the current standalone DeepLens release.

Set up

From a clone of the repository:

git clone https://github.com/AI4Optics/DeepLens_Hyperspectral.git
cd DeepLens_Hyperspectral
conda env create -f environment.yml -n deeplens
conda activate deeplens

The environment declares Python 3.10, PyTorch 2.7 or newer and torchvision 0.22 or newer from the CUDA 12.8 index, targeting RTX 5090 / sm_120. The scripts select CUDA when available and otherwise use CPU.

Continue