RGB-MS Dataset

Tosi, Fabio ; Zama Ramirez, Pierluigi ; Poggi, Matteo ; Salti, Samuele ; Mattoccia, Stefano ; Di Stefano, Luigi (2022) RGB-MS Dataset. University of Bologna. DOI 10.6092/unibo/amsacta/6877. [Dataset]
Full text available as:
[thumbnail of RGB-MS Dataset Labeled] Archive (RGB-MS Dataset Labeled)
License: Creative Commons: Attribution-Noncommercial 4.0 (CC BY-NC 4.0)

Download (511MB)
[thumbnail of RGB-MS Dataset Unlabeled] Archive (RGB-MS Dataset Unlabeled)
License: Creative Commons: Attribution-Noncommercial 4.0 (CC BY-NC 4.0)

Download (81GB)
[thumbnail of README] Text(rtf) (README)
License: Creative Commons: Attribution-Noncommercial 4.0 (CC BY-NC 4.0)

Download (121kB)


We address the problem of registering synchronized color (RGB) and multi-spectral (MS) images featuring very different resolution by solving stereo matching correspondences. Purposely, we introduce a novel RGB-MS dataset framing 13 different scenes in indoor environments and providing a total of 34 image pairs annotated with semi-dense, high-resolution ground-truth labels in the form of disparity maps. To tackle the task, we propose a deep learning architecture trained in a self-supervised manner by exploiting a further RGB camera required only during training data acquisition. In this setup, we can conveniently learn cross-modal matching in the absence of ground-truth labels by distilling knowledge from an easier RGB-RGB matching task based on a collection of about 11K unlabeled image triplets. Experiments show that the proposed pipeline sets a good performance bar (1.16 pixels average registration error) for future research on this novel, challenging task.

Document type
Tosi, FabioUniversity of Bologna0000-0002-6276-5282
Zama Ramirez, PierluigiUniversity of Bologna0000-0001-7734-5064
Poggi, MatteoUniversity of Bologna0000-0002-3337-2236
Salti, SamueleUniversity of Bologna0000-0001-5609-426X
Mattoccia, StefanoUniversity of Bologna0000-0002-3681-7704
Di Stefano, LuigiUniversity of Bologna0000-0001-6014-6421
Depth Stereo Cross-Spectral Matching RGB-MS Multispectral
Deposit date
25 Mar 2022 14:24
Last modified
31 May 2022 21:00

Other metadata



Staff only: View the document