pyAT MAT file V7.3

[lnadolski]

Dear all

the function at.load_lattice does not work for a V7.3 mat lattice file.

Per default, the default matlab is still configured with -V7 (if not changed in the user's preference)

Example to generate a 7.3 version matlab file The matlab format has changed significantly

save("newLattice.mat", "-v7.3")

Best regards,

Laurent.

[lfarv]

Hello @lnadolski,

In Matlab, atloadlattice works as expected with V7.3 lattice files: load (and consequently atloadlattice) automatically detects the format.

However in python, we rely on SciPy which state: "v4 (Level 1.0), v6 and v7 to 7.2 matfiles are supported.". In addition, SciPy mentions:

"You will need an HDF5 Python library to read MATLAB 7.3 format mat files. Because SciPy does not supply one, we do not implement the HDF5 / 7.3 interface here."

Decoding the private Matlab structure starting with a plain HDF5 reader looks difficult. So for file exchange with python, you must stick to the V7 format.

[lnadolski]

Dear @lfarv

This is good to know. Maybe we should add a note in pyAT.

[lfarv]

I found the hdf5storage project which claims to support V7.3 .mat files. The project does not seem to be alive any more, but it may work. I would not like to have PyAT depend on this package, but if it works (still to be checked), we could have PyAT check that if the package is available (the user explicitly installed it), it will use it, otherwise it will stick to SciPy.

[lfarv]

Forget hdf5storage, I could not make it work: it hangs forever…

[T-Nicholls]

Hello both,

When I try and load a v7.3 .mat lattice file using at.load_lattice I get the error: NotImplementedError: Please use HDF reader for matlab v7.3 files, e.g. h5py. To me that's clearly telling the user that the v7.3 file is the issue; however, if we wanted to be more specific we could catch that error from scipy and reraise it with a message telling the user that they need to save the lattice file in Matlab using an earlier version.

Alternatively, if there is significant demand, we could write new functions specifically to support v7.3 files. I briefly tried loading a v7.3 file using h5py and it seemed feasible as it, at least partially, decodes the data but it's significantly more convoluted than using scipy:

>>> import h5py
>>> f = h5py.File("test_73_lattice.mat", "r")
>>> ring = f["RING"][0]
>>> len(ring)
2190
>>> f[ring[0]].keys()
<KeysViewHDF5 ['Class', 'Energy', 'FamName', 'Length', 'Limits', 'PassMethod']>
>>> list(f[ring[1]]['Length'])
[array([4.38])]
>>> "".join(chr(i) for i in numpy.asarray(f[ring[12]]['FamName']).flatten())
'Q3D'
>>> list(f[ring[12]]['PolynomB'])
[array([0.]), array([-0.9750889])]