goodguy-dev commented 4 years ago

The Hyundai Department Store dataset

This is the public release of the Hyundai Department Store dataset for indoor visual localization challenge in KCCV 2020.

The Hyundai Department Store dataset are generated by using M1X mapping robot which is developed in NaverLabs. The M1X has a capability to build a precise indoor map using high accurate lidia sensors and it provides groundtruth for simultaneously obtained images from equipped six Basler cameras and four Samsung Galaxy S9.

The Hyundai Department Store dataset includes train images, raw lidar data, 3D point cloud map, and high-accurate groundtruth for images and raw lidar data. The groundtruth of the raw lidar sensor data are obtained from high-accurate Lidar SLAM. Due to acquisition time difference between lidar and images, the poses of the image are linearly interpolated according to its timestamps from the computed lidar poses. And, we modified images semi-automatically to blur human’s upper body to cover privacy problem.

License

As described in LICENSE.txt file, basically, the Hyundai Department Store dataset follows the CC-NC-ND license Creative Commons Attribution-NonCommercial-NoDerivatives 4.0. In order to promote usages with research purpose, the license is modified with additional terms to permit research paper works.

Provided Files

The following files are provided with this release of the Hyundai Department Store dataset:

{1f,b1}/: each floor has test and train datasets
- train/{timestamp}/
- images/: contains train images obtained from six Basler cameras or four Samsung Galaxy S9
- pointclouds_data/: contains lidar data in *.pcd format obtrained from two different Velodyne Puck sensors
- map.pcd: global 3D pointcloud map
- camera_parameters.txt: contains the intrinsic parameters for each camera sensor
- groundtruth.hdf5: contains the groundtruth poses for each lidar data and image
- test/{timestamp}/
- images/: contains test images obtained from six Basler cameras or four Samsung Galaxy S9
- camera_parameters.txt: contains the intrinsic parameters for each camera sensor
LICENSE.txt: describes the license of the Hyundai Department Store dataset
README.md: readme file

In the following, we will describe the provided files in detail.

pointclouds_data

This directory contains the lidar sensor data in *.pcd format known as PointCloud format. The name of each pcd file indicates the name of sensor and acquisition time, {lidarname}_{timestamp}.pcd. To read this file, we used PCL library.

map.pcd

This file is a global PointCloud map in *.pcd format. By using pointclouds_data and groundtruth.hdf5, it could be easily generated from PCL library.

images

This directory contains the images obtained from six Basler cameras or four Samsung Galaxy S9. Each image has the name by following format:

{cameraname}_{timestamp}.jpg

The cameraname and timestamp are used to search the intrinsic camera parameters and groundtruth pose. We will discuss next.

camera_parameters.txt

The text file camera_parameters.txt contains the intrinsic camera parameters of the cameras. Each line has the format:

{cameraname} {width} {height} {fx} {fy} {cx} {cy} {k1} {k2} {p1} {p2} {k3}

groundtruth.hdf5

This file contains groundtruth poses for *.pcd files in pointclouds_data/ and *.jpg files in images/. The file format is HDF5 which is well known fast I/O storage. To see contents of the file, we recommend HDFCompass viewer working well in Ubuntu 18.04.

In this file, there are two database labels for each sensor(camera, lidar), {sensorname}_stamp and {sensorname}_pose. The columns of {sensorname}_stamp database label mean the timestamp and the columns of {sensorname}_pose database label mean the pose array by following format:

[tx ty tz qw qx qy qz]^T

For example, to get the pose of 22970285_1555392901779869.jpg image, first, search the index having same timestamp 1555392901779869 in 22970285_stamp database. After then, access the pose array with the searched index in 22970285_pose database.

Coordinate system (IMPORTANT!)

We use the robot coordinate system typically used in robotics. In this robot coordinate system, the sensor is going forward on the x-axis, pointing to the left on the y-axis, and pointing to the top on the z-axis. To directly connect to the camera coordinate system, we rotated the pose at the end. You can compose a pose matrix from the searched pose array as follows:

// pseudo code to convert pose array to matrix
pose = Identity(4)
pose[:3, 3] = p[:3]
pose[:3, :3] = Quaternion(p[3:]).rotation_matrix

And, a global 3D point from a pcd file can be projected into the image plane as follows:

// pseudo code to project a 3D point into image
point3d_local = pose.inverse()*point3d
image_pixel = (K*point3d_local).hnormalized()

SeungyounShin commented 4 years ago

submit 형식은 어떻게 되나요 ?
pose 에서 앞에 3개가 의미하는 바는 무엇인가요? 4개는 쿼터니안인가요?