Error When Opening DATX Files: Unpacking Values Issue

[nombac]

Description

When attempting to open .datx files (e.g. test.datx) using the open_topography function, an error occurs related to unpacking values. This issue prevents successful file operations on .datx format data.

Error Message

The following traceback details the specific error encountered:

Traceback (most recent call last):
  File "/Users/shirose/Library/CloudStorage/Dropbox/test.py", line 11, in <module>
    reader = open_topography(filename, format='datx')
  File "/Users/shirose/Library/Python/3.12/lib/python/site-packages/SurfaceTopography/IO/__init__.py", line 240, in open_topography
    return lookup_reader_by_format[format](fobj, **kwargs)
  File "/Users/shirose/Library/Python/3.12/lib/python/site-packages/SurfaceTopography/IO/DATX.py", line 101, in __init__
    category, unit, values = x_converter
ValueError: too many values to unpack (expected 3)

[nombac]

I'm not an expert in DATX format or HDF5 technology. However, I wanted to share a Python code snippet, adapted by ChatGPT from a MATLAB sample provided by Zygo, that appears to successfully read DATX files. I hope this can help address the issue reported above.

import numpy as np
import matplotlib.pyplot as plt
import h5py

def get_z_data_with_units(map_hdf):
    """Convert Z data based on the unit converter attributes."""
    unit_out = 'um'  # Default output unit
    z_converter = map_hdf.attrs['Z Converter'][0]  # Correct access to the tuple inside the list
    category = z_converter[0].decode('utf-8')
    base_unit = z_converter[1].decode('utf-8')  # Correct decoding of base unit
    values = z_converter[2]  # Values array

    # Select the scaling based on the unit
    if base_unit == 'NanoMeters':
        z_scale = 1e-3
    elif base_unit == 'MicroMeters':
        z_scale = 1.0
    elif base_unit == 'MilliMeters':
        z_scale = 1e3
    elif base_unit == 'Meters':
        z_scale = 1e6
    elif base_unit == 'Fringes':
        S = values[2]
        O = values[3]
        W = values[1]
        z_scale = 1e6 * S * O * W
    else:
        unit_out = base_unit
        z_scale = 1

    print(base_unit)
    height_um = map_hdf[:] * z_scale  # Correct division using the scale
    return height_um, unit_out

def read_datx(file_name):
    with h5py.File(file_name, 'r') as file:
        # Find the first dataset under 'Data/Surface'
        base_path = 'Data/Surface'
        if base_path in file:
            dataset_names = list(file[base_path].keys())
            if dataset_names:
                dataset_path = f'{base_path}/{dataset_names[0]}'
                map_hdf = file[dataset_path]
                surface_data, unit_out = get_z_data_with_units(map_hdf)

                # Set visualization range
                vmin = -20e-3
                vmax = 20e-3

                # Visualization
                plt.figure()
                plt.imshow(surface_data, cmap='viridis', interpolation='none', vmin=vmin, vmax=vmax)
                plt.colorbar(label=f'Height [{unit_out}]')
                plt.title('Surface Data Visualization')
                plt.show()
            else:
                print("No datasets found under the specified base path.")
        else:
            print("Specified base path does not exist in the file.")

# Example usage
file_name = './test.datx'
read_datx(file_name)

[pastewka]

@nombac - I am sorry that just now I am looking at this. Would you mind sharing the offending file (and can I add it to the automated testing framework). This will help me fix this. I'll do this asap.

[pastewka]

I am sorry, I see that you actually shared that file