Hypertools Submodule

[robertCubert]

Open Points for Discussion:

• Where/How to integrate into our repo
• Who will test (Linux & Nvidia GPU required)
• Test with the new sample data
• Add live performance example

[robertCubert]

feature/add-hypertools

[antmedellin]

• Since CUDA can be used with Python and CPP, I do not think it should have its own language folder.
• Nvidia GPU is not required. I have implemented functions for both CPU using multi-threading and in GPU. The code also works with WSL, so a complete Linux OS is not required.
• I have tested with the cornfields and teagarden sample data. However, the exported TIFF files from the Cubert-SDK did not work with the OpenCV function "imreadmulti". I had to export as an ENVI and then convert to TIFF.
• What do you mean by a live performance example? Do you want a video showing the code working?

[nhanson2]

@antmedellin it might be useful to have a tutorial with images to show how a user might be expected to operate the interfaces. I am happy to test this on my Ubuntu machine.

[arndcubert]

@antmedellin The imreadmulti function reads multi-page TIFFS. By default, the cubes are exported as multi-channel-Tiffs. Can exports using the tiff_formatMultiPage option be imported?

[robertCubert]

• Good to know that it works with WSL as well, maybe we can include the necessary steps in the readme/tutorial as well
• I saw that a GPU is not required, however it would be nice to have it tested on one, to get a feeling for the performance
• Regarding the sample data, I have uploaded more data here, which is all in .cu3 format, so there should be no issues with TIFF.
• By live performance example I mean it would be nice to have an example maybe based on Example_06, where a live video feed from the camera can be directly fed to the Hypertools to do some classification in real-time.

[robertCubert]

Regarding the integration of the Hypertools submodule, as mentioned in #26 , I created a new repo Hypertools C++/CUDA Classification. In this repo we would suggest to add the Hypertools as well as the SDK as submodules. In this repo we then want to provide a cmake file that is looking for both of the submodules so that the user can link their project.

@antmedellin from your readme, I've seen that you would like the Hypertools to be agnostic to camera manufactures, which I think is a good idea. We would therefore suggest that cubert specific examples that use the Hypertools could be developed in our repo. What do you think about this approach?

[antmedellin]

• Documentation is definitely a work in progress for my repo. After my UIs are more stable, I can document how to use them. The WSL installation should be pretty similar. If there are any differences, just let me know and I can add it to the readme file. The only difference may be in the cmake command.
• @arndcubert Where can I find an example of exporting as multi-page TIFFs? I was unable to find the option in the SDK example files or in the Cuvis UI export function.
• The live performance may take some time. Since it involves integration with Cubert cameras, would someone be able to help me develop the functionality?
• Is there anything I need to do for adding the submodule to the new repo? I saw it is already added in.
• The idea to keep the cubert specfic examples separate from my repo sounds good to me. A new folder for examples would need to be created in the new repo. A function for loading .cu3 images/video feed into opencv would need to be created and integrated with my work. Everything else should be the same on my side of the code. (It may be best if this function is in cuvis.h/cuvis.hpp)

[robertCubert]

• Example for multipage TIFF can be found here
• I can of course help with the live classification! It should be quite easy to merge your cuda_cubert_example with the 06_recordVideo Example . We have to see weither we can get it to run with CUDA and WSL or I will contact our IT and they will set me up with a dual-boot system.
• I think currently nothing else is needed to integrate the submodule. We will create the function probably in cuvis_auxiliary, because this is were helper functions will be gathered in the near future.

[antmedellin]

• The example link says that it is a multi-channel TIFF not a multi-page TIFF. Do I change this to something different for multi-page TIFF "args.format = cuvis::tiff_format_t::tiff_format_MultiChannel;" ?
• I created a Dockerfile to be able to run the non-GPU code in a Docker container. It installs all of the necessary dependencies for my code. This will allow easy testing on Windows/Mac without WSL or a VM. I still need to make a Dockerfile with CUDA packages. My Docker experience is limited, so it is taking some time to figure it out. The Dockerfile can be found here.
• I think the cuvis_auxiliary is a great idea. We can discuss during our meeting about some initial functions that can be included there.

[robertCubert]

• Sorry my bad, I thought they were the same for a second. But Arnd mentioned here:

@antmedellin The imreadmulti function reads multi-page TIFFS. By default, the cubes are exported as multi-channel-Tiffs. Can exports using the tiff_formatMultiPage option be imported?

that there should be an option for MultiPage as well. I haven't tried it myself though.

• Docker seems like a great solution for the OS interoperability!
• Sounds good to me. Maybe @arndcubert can upload what we currently have, when he is back from his holidays

[antmedellin]

Just tested this example with 'args.format=cuvis::tiff_format_t::tiff_format_MultiPage;' for the multi-channel tiff and it worked. I was able to view the output file in Document Viewer on Linux in the desired format.

As a side note, it did not work on the cornfields dataset. I think the example requires the session_xxx_xxx_cube.tiff to work properly.