It is well known that training deep models relies on massive amounts of training data. Thus, a large-scale dataset is required for the un paired video translation between nighttime infrared and daytime visible. Unfortunately, existing infrared-related datasets are either collected for other specific research areas or limited by the small scale. To this end, we capture the infrared and visible videos to build the InfraredCity dataset, by far the largest infrared-related dataset to date, which contains nearly 600,000 frames. Furthermore, we artificially select parts of the InfraredCity dataset to make the InfraredCity-Lite to facilitate comparison with other methods.

Thesis address: https://arxiv.org/abs/2204.12367

Code address: https://github.com/BIT-DA/ROMA

InfraredCity:

The InfraredCity dataset consists of nighttime infrared, nighttime visible, and daytime visible videos. In particular, the nighttime infrared and nighttime visible videos are captured through a binocular infrared color camera (DTC 300 equipment), which is aligned at the hardware level.

Nighttime Infrared and Nighttime Visible Videos. According to the heat-related imaging principle, infrared sensors can be divided into the near-infrared camera and the long-wave infrared camera. Compared with the near-infrared camera, the long-ware infrared camera conforms to the requirements of recognition and vehicle driving scenes at night. Thus, we utilize a type of long-wave equipment, DTC 300 for capturing infrared videos. As for the nighttime visible videos, they are still captured via the DTC 300 equipment. Since we perform the alignment of the infrared and the visible cameras at the hardware level, nighttime infrared and nighttime visible videos are identical in scene and length.

For practical applications at night, we shoot traffic and monitor ing videos to build the InfraredCity dataset. Traffic videos consist of scenes from the city and highway. The more diverse the content of city videos is, the harder it will be for the model to converge. Thus, the city scenes captured on the InfraredCity focus on varied things (e.g., buildings and cars) which could be observed in our daily life when driving. On the other hand, motion shift should be one of the challenges in video-related datasets. Thus, we capture highway scenes for rapid movement changes. Additionally, since infrared sensors are widely used for monitoring, we also collect the monitoring videos as a part of InfraredCity. Instead of fixing the angle, we rotate the camera in all directions for diversity and motion shift.

Daytime Visible Videos. Since we want high-quality translated daytime visible results, we shoot the daytime visible videos on a clear day. The daytime visible videos still contain scenes of the city, highway, and monitoring. Although all scenes in the daytime visible videos correspond to the scenes in the nighttime infrared videos, there are large differences in each scene. For example, trucks are more common at night than during the day because there are fewer vehicles on the nighttime road, which benefits the trucks. The inconsistency of the shooting time between nighttime infrared videos and daytime visible ones brings great challenges to the dataset.

InfraredCity-Lite:

Although the InfraredCity is diverse and large-scale, it may not be appropriate for experimental studies due to repetition of similar frames and mixture of all scenes. In this case, we select parts of InfraredCity to build the InfraredCity-Lite.

Scenario. To be in line with the input requirements of most image/video translation methods, three scenarios: Single, Double and Triplet are designed on InfraredCity-Lite. In particular, The ratio of the three is controlled at about 3 : 2 : 1. Taking Triplet as an example, we select three consecutive frames from ten consecutive frames to form the Monitor part of the InfraredCity-Lite dataset. The strategy reduces the repetition of similar frames on our dataset. It is more suitable in size and structure as an experimental dataset for research.

Structure. Unlike mixing all scenes on InfraredCity, InfraredCity-Lite is divided into City, Highway, and Monitor, which correspond to city scenes, highway scenes, and monitoring scenes on InfraredCity. Notably, nighttime infrared videos on City and Highway are captured under two weather conditions, clearday and overcast. The clear day condition is ideal for infrared sensors while the overcast condition may affect the temperature contrast. Thus, taking over cast weather into account can help InfraredCity-Lite to be more applicable for the actual scene. Additionally, since the main challenges of monitoring scenes are the different angles of the monitor, we rotate the monitor in all directions and collected videos named Monitor. The structure of InfraredCity-Lite can be viewed on paper.

The whole dataset structure is shown below：

The researchers

The study was conducted by:

Qing Liu ( qing.liu@iraytek.com )

Shuigen Wang ( shuigen.wang@iraytek.com )

Copyright Notice

-----------COPYRIGHT NOTICE STARTS WITH THIS LINE------------

Copyright (C) 2021 Yantai IRay Technology Co., Ltd. All Rights Reserved.

All rights reserved.

Permission is hereby granted, without written agreement and without license or royalty fees, to use, copy, modify, and distribute this database (the images, the results and the source files) and its documentation for any purpose, provided that the copyright notice in its entirety appear in all copies of this database, and the original source of this database, Yantai IRay Technology Co., Ltd (IRay, https://www.infiray.com/ ), is acknowledged in any publication that reports research using this database. In no event shall Yantai IRay Technology Co., Ltd be liable to any party for direct, indirect, special, incidental or consequential damages arising out of the use of this database and its documentation,even if Yantai IRay Technology Co., Ltd has been advised of the possibility of such damage.

Yantai IRay Technology Co., Ltd disclaims any warranties, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose. The database provided hereunder is on an "as is" basis, and Yantai IRay Technology Co., Ltd has no obligation to provide maintenance, support, updates, enhancements, or modifications.

-----------COPYRIGHT NOTICE ENDS WITH THIS LINE------------

Download

We are making the InfraredCity Dataset of infrared images available to the public for free. If you use this database in your research, We would appreciate your appreciation for the research done by our company:

Yantai IRay Technology Co., Ltd.

Database application

-The name is 2-10 Chinese characters or letters- -Incorrect email format- -The organization is 4-30 Chinese characters or letters- -Illegal mobile phone number input- -Industry is 4-30 Chinese characters or letters- -The position is 2-20 Chinese characters or letters-

  

You can fill in the above form to submit the application form for using this database to us. After the approval, the download address of this database will be sent to your email.

If you encounter any problems in downloading, viewing or using this database, please contact the maintenance staff of this website:Chunlei Guan( chunlei.guan@iraytek.com ),WeChat( gcl1353 )