Bug report: Build hl2ss_ulm.so and main_ulm.cpp

[Secret-Lin]

Thanks for your great work. I am trying to do some image process on what holoLens2 sees. So, I wanted to import hl2ss as shared library(hl2ss_ulm.so). Here are bugs I met and how I solved: For building hl2ss_ulm.so:

1. ‘std’ does not name a type, like mutex; How to fixed: using c++17.
2. ‘wstring_convert’ is not a member of ‘std’; How to fixed: using c++17 and add '#include < l o c a l e >' (delete spaces)in hl2ss_ulm.cpp.
3. Not find defination of hl2ss_mt::source::xxxxx; How to fixed: run 'g++ -fPIC -shared -std=c++17 -o libhl2ss_ulm.so hl2ss_ulm.cpp hl2ss_mt.cpp' to build hl2ss_ulm.so. For building main_ulm:
4. No such file or directory of xxxxx, like opencv or can not find lhl2ss_ulm. Here is the tasks.json of I used: '{ "version": "2.0.0", "tasks": [ { "type": "cppbuild", "label": "C/C++: g++ build active file", "command": "/usr/bin/g++", "args": [ "-fdiagnostics-color=always", "-g", "-std=c++17", "${file}", "hl2ss.cpp", "hl2ss_lnm.cpp", "../../3rdparty/Zdepth/src/.cpp", "../../3rdparty/Zdepth/zstd/src/.c", "-o", "${fileDirname}/${fileBasenameNoExtension}", "-DHL2SS_ENABLE_ZDEPTH", "-I../../3rdparty/Zdepth/include", "-I../../3rdparty/Zdepth/zstd/include", "-I../../3rdparty/Zdepth/zstd/src", "-lavcodec", "-lavutil", "-I/usr/include/opencv4", // OpenCV 4 include path "-L/usr/local/lib", "-L/home/rvc/colcon_ws/utils/hl2ss/extensions/client_cpp", "-lopencv_highgui", "-lopencv_imgcodecs", "-lopencv_imgproc", "-lopencv_core", "-O3", "-lhl2ss_ulm", "-Wl,-rpath,/home/rvc/colcon_ws/utils/hl2ss/extensions/client_cpp", "-lpthread" ], "options": { "cwd": "${fileDirname}" }, "problemMatcher": [ "$gcc" ], "group": { "kind": "build", "isDefault": true }, "detail": "Task generated by Debugger." } ]

}'

2. error: a reinterpret_cast is not a constant expression. In hl2ss_ulm.h, the author uses constexpr to void multidefination, but reinterpret_cast can not be used in the return. How to fixed: here is the code I changed for constexpr functions in hl2ss_ulm.h: 'constexpr map_rm_vlc unpack_rm_vlc(uint8_t payload) { rm_vlc_metadata metadata = reinterpret_cast<rm_vlc_metadata>(payload + parameters_rm_vlc::PIXELS); return { payload, metadata }; // return { payload, (rm_vlc_metadata)(payload + parameters_rm_vlc::PIXELS) }; } constexpr map_rm_depth_ahat unpack_rm_depth_ahat(uint8_t payload) { uint16_t payload_temp = (uint16_t)(payload); uint16_t return_value2 = (uint16_t)(payload + (parameters_rm_depth_ahat::PIXELS sizeof(uint16_t))); rm_depth_ahat_metadata metadata = (rm_depth_ahat_metadata)(payload + (2 parameters_rm_depth_ahat::PIXELS sizeof(uint16_t))); return { payload_temp, return_value2, metadata }; }

constexpr map_rm_depth_longthrow unpack_rm_depth_longthrow(uint8_t payload) { uint16_t payload_temp = (uint16_t)(payload); uint16_t return_value2 = (uint16_t)(payload + (parameters_rm_depth_longthrow::PIXELS sizeof(uint16_t))); rm_depth_longthrow_metadata metadata = (rm_depth_longthrow_metadata)(payload + (2 parameters_rm_depth_longthrow::PIXELS sizeof(uint16_t))); return { payload_temp, return_value2, metadata }; }

constexpr map_rm_imu unpack_rm_imu(uint8_t payload) { rm_imu_sample payload_temp = (rm_imu_sample*)(payload); return { payload_temp }; }

constexpr map_pv unpack_pv(uint8_t payload, uint32_t size) { pv_metadata metadata = (pv_metadata*)(payload + size - sizeof(pv_metadata)); return { payload, metadata }; }

constexpr map_microphone_raw unpack_microphone_raw(uint8_t payload) { int16_t payload_temp = (int16_t*)(payload); return { payload_temp }; }

constexpr map_microphone_aac unpack_microphone_aac(uint8_t payload) { float payload_temp = (float*)(payload); return { payload_temp }; }

constexpr map_microphone_array unpack_microphone_array(uint8_t payload) { float payload_temp = (float*)(payload); return { payload_temp }; }

constexpr map_si unpack_si(uint8_t payload) { si_frame payload_temp = (si_frame*)(payload); return { payload_temp }; }

constexpr map_eet unpack_eet(uint8_t payload) { eet_frame payload_temp = (eet_frame*)(payload); return { payload_temp }; }

constexpr map_extended_audio_raw unpack_extended_audio_raw(uint8_t payload) { int16_t payload_temp = (int16_t*)(payload); return { payload_temp }; }

constexpr map_extended_audio_aac unpack_extended_audio_aac(uint8_t payload) { float payload_temp = (float*)(payload); return { payload_temp }; }' Finally, I successfully built main_ulm and saw the pv. Thanks again for the author's great work. And these bugs may due to my ubuntu20.04 foxy env. Hope this issue can help someone.

[jdibenes]

Very cool. Thank you. Indeed, I only tested the client shared library on Windows and MSVC.

[Secret-Lin]

@jdibenes I figure that you do not write a corresponding hl2ss_3dcv for cpp, I want to transfer the value of longthrow to depth refer to its camera in cpp. By reading your sample_pv_depth_lt.py, I have following questions:

1. Are the world coordinates of pv and lt the same one?
2. Why you do transformation on xy1_o and xy1_d seperately and finally hstack them togather?

(By the way, I cannot run sample_pv_depth_lt.py, it just stuck and cannot be killed. And I can run clientsteam*.py to see the sensors' information)