Include Example for Working with Pansharpening HSI Data

[nhanson2]

Comments imported from Slack for conciseness

Question:

1) Is there also any documentation on how to use the Viewer from the SDK,? There doesn't seem to be an example that uses it, and I'm unable to get a minimum working example to execute without it silently failing: 2) What is the correct way to read a panimage measurement with the SDK? If I try to load a single .cu3 file with the Python SDK, I end up with a measurement where the 'PANIMAGE' field is an array of (w x h x 1) for a single channel. Loading the same measurement with Cuvis Touch results in a multi-dimensional datacube where I can see spectra. T here seems to be a different structure to the "Data" field of the Measurement type. A normal field has a "cube" field, while the panimage lacks this. Am I missing something that needs to be done to force the cube to load? Steps to load:

path = 'C:/Users/nathaniel/Downloads/cornfields/session_002/session_002_490.cu3'
mesu = cuvis.Measurement(path)
print(mesu.Data)
print('==============')
print(mesu.Data.get('PANIMAGE').array.shape)
Output:
{'white_ref': 'C:/Users/nathaniel/Downloads/cornfields\\Calibration\\white__session_002_001_snapshot16423118797989931.cu3', 'dark_ref': 'C:/Users/nathaniel/Downloads/cornfields\\Calibration\\dark__session_002_003_snapshot16423119279414228.cu3', 'PANIMAGE_info': <cuvis.cuvis_il.cuvis_sensor_info_t; proxy of <Swig Object of type 'cuvis_sensor_info_t *' at 0x000001F3B338B030> >, 'PANIMAGE': <cuvis.Measurement.ImageData object at 0x000001F3B338B940>, 'IMAGE_info': <cuvis.cuvis_il.cuvis_sensor_info_t; proxy of <Swig Object of type 'cuvis_sensor_info_t *' at 0x000001F3B338B8A0> >, '': 'Not Implemented!', 'GPS_data': <cuvis.cuvis_il.cuvis_gps_t; proxy of <Swig Object of type 'cuvis_gps_t *' at 0x000001F3B338B990> >}
==============
(2048, 2448, 1)

Attempts to use viewer

import cuvis
path = 'C:/Users/nathaniel/Downloads/cornfields/session_002/session_002_490.cu3'
mesu = cuvis.Measurement(path)
# Create viewer settings
settings = cuvis.cuvis_il.cuvis_viewer_settings_t()
# Create viewer - Code dies here
viewer = cuvis.Viewer(settings)
view = viewer.apply(mesu)

[arndcubert]

@nhanson2 I'll answer in C++, as I'm not the python expert. I don't have your data, so I try to recreate. I see two issues:

a) the cube might not be processed. If the Error "Cube could not be processed" occurs, the measurement is still unprocessed. In you mesu.Data I don't see the field "cube". You can fix this like this:

cuvis::ProcessingArgs processing_args;
processing_args.processing_mode = cuvis::processing_mode_t::Cube_Raw;
pc.set_processingArgs(processing_args);
pc.apply(m);

b) You did not select any view-option. You need to provide: pan_algorithm, pan_interpolation_type, pan_scale and most importantly userplugin. That latter is the User-plugin XML content (not the file path, but the XML content)

cuvis::ViewArgs viewer_args;

viewer_args.userplugin = buffer.str(); //file content
viewer_args.add_fullscale_pan = true;
viewer_args.pan_algorithm = cuvis::pan_sharpening_algorithm_t::
    pan_sharpening_algorithm_CubertMacroPixel;
viewer_args.pan_interpolation_type =
    cuvis::pan_sharpening_interpolation_type_t::
        pan_sharpening_interpolation_type_Cubic;
viewer_args.pan_scale = 1.0;

auto viewer = cuvis::Viewer(viewer_args);

To read the XML file I used the following code:

//read entire XML from file
std::ifstream t("C:\\Program Files\\Cuvis\\user\\plugin\\ref\\ndvi.xml");
std::stringstream buffer;
buffer << t.rdbuf();

For me, this works. I've attaches the whole main.cpp file for this. main.cpp.txt

Can you confirm this solves it for phyton and if so provide a working python code of this?

[nhanson2]

Hi @arndcubert I translated your C++ into Python

import cuvis # Load the Cubert SDK

path = 'C:/Users/nathaniel/Downloads/cornfields/session_002/session_002_490.cu3'
mesu = cuvis.Measurement(path)
print(mesu.Data)
print('==============')
print(mesu.Data.get('PANIMAGE').array.shape)

pluginFile = 'C:/Program Files/Cuvis/user/plugin/ref/hNDVI.xml'
with open(pluginFile) as f:
    plugin_text = f.read()

print(plugin_text)
viewer_args = cuvis.cuvis_il.cuvis_viewer_settings_t()
viewer_args.user_plugin = plugin_text
viewer_args.add_fullscale_pan = True
viewer_args.pan_algorithm = cuvis.cuvis_il.pan_sharpening_algorithm_CubertMacroPixel
viewer_args.pan_interpolation_type = cuvis.cuvis_il.pan_sharpening_interpolation_type_Cubic
viewer_args.pan_scale = 1.0
viewer = cuvis.Viewer(viewer_args)
view = viewer.apply(mesu)
print('Succeeded!')

The code is still failing on the constructor for the Viewer. Specifically it seems to fail silently on this function in the cuvis_il.py file. The function should throw an SDK exception on failure, but that doesn't seem to occur.

def cuvis_viewer_create(o_pViewer, viewerSettings):
    return _cuvis_pyil.cuvis_viewer_create(o_pViewer, viewerSettings)

@robertCubert do you have any suggestions from the Python side of things?

[arndcubert]

@nhanson2 could you provide the debug log file? (see Program Data/cuvis or /log)

[nhanson2]

@arndcubert this seems to be the only log file that is populated. python.exe.log

[arndcubert]

@BenMGeo can you try to reproduce in python? If c++ works, why does python not work? Pls check the wrappers

[BenMGeo]

I'll look into it

[BenMGeo]

Hi Nathaniel,

Thank you for your patience.

So I edited the code below: The main issue is, that viewer_args (cuvis_viewer_settings_t()) uses the attribute userplugin instead of user_plugin.

Also, there is no add_fullscale_pan'. I assume this is the equivalent tocomplete`.

If you call it that way, it runs correctly into the SDK Error "as expected". Unfortunately, the cube could not be processed. I have a look into the specific issues here.

import cuvis # Load the Cubert SDK

path = 'C:/Users/nathaniel/Downloads/cornfields/session_002/session_002_490.cu3'
mesu = cuvis.Measurement(path)
print(mesu.Data)
print('==============')
print(mesu.Data.get('PANIMAGE').array.shape)

pluginFile = 'C:/Program Files/Cuvis/user/plugin/ref/hNDVI.xml'
with open(pluginFile) as f:
    plugin_text = f.read()

print(plugin_text)
viewer_args = cuvis.cuvis_il.cuvis_viewer_settings_t()
viewer_args.userplugin = plugin_text
viewer_args.complete = True
viewer_args.pan_algorithm = cuvis.cuvis_il.pan_sharpening_algorithm_CubertMacroPixel
viewer_args.pan_interpolation_type = cuvis.cuvis_il.pan_sharpening_interpolation_type_Cubic
viewer_args.pan_scale = 1.0
viewer = cuvis.Viewer(viewer_args)
view = viewer.apply(mesu)
print('Succeeded!')

@arndcubert This rises 1 question and some follow up: This is how swig handles the SDK. The C++ SDK does not return a readable status, when a view definition is missing. Should we fix that in C++? Should I wrap the type, so it checks (or maybe even adapts) the different ways to write userplugin? Do we want to define a simple default, when the plugin definition is missing? Why is the swig naming different?

[arndcubert]

@BenMGeo The swig naming is generated automatically by SWIG, perhaps it's not refreshed? If the userplugin is empty, the viewer is supposed to generate a preview instead.

[nhanson2]

@BenMGeo any update on this bug?

[BenMGeo]

I just came back to the office... Will check the current sdk if issue persists.

[BenMGeo]

@arndcubert I still get the wrong naming in swig interface for viewer settings. I'd shift the flag towards the cpp sdk. I will need to check your main now.