Closed lfarv closed 1 month ago
Dear @lfarv,
a few tests in matlab:
EBS lattice, no errors
FCC lattice WITH errors:
ALL looks ok here.
I will now try to load in python and save/load json in python and back to maltab
Dear @lfarv,
I tried to load the json file in python.
mat_key does not work anymore
r = at.load_lattice(matfile, mat_key='betamodel')
Traceback (most recent call last):
File "/machfs/liuzzo/accelerator_toolbox_tests/matlab/jason/load_json_test.py", line 10, in
the new keywork use
instead is ok.
I can load the json file and plot it.
The help of save_lattice and load_lattice could be updated to include save_json and load_json
The JSON files saved by matlab and by python are different. In particular the R1-R2 and T1-T2 fields are not in the same order.
Same for the field Energy. At least for cross check (diff for example) it would be nice to maintain the same identical output for the json files produced by matlab and python.
Dear @lfarv
globally I find this new feature very useful and working correctly!
The only issues are the mat_key (for backward compatibility) and a fixed order of the fields in json file produced by atwritejson
in matlab and save_json
in python
let me know and I will further test.
I have seen that many functions are impacted but I did not test that all other methods are working as before (save_lattice, load_lattice, save_mat, load_mat, etc...). May be a new test could verify that all load and save did not change their output?
Hello @simoneliuzzo Thanks a lot for testing this branch ! And nice to see that you find it useful. About your remarks:
mat_key does not work anymore
The mat_key
keyword is repaired. Stupid typo in the code…
The help of save_lattice and load_lattice could be updated to include save_json and load_json
What do you mean ? Here JSON is mentioned in the help:
>>> help(at.load_lattice)
Help on function load_lattice in module at.load.allfiles:
load_lattice(filepath: str, **kwargs) -> at.lattice.lattice_object.Lattice
Load a Lattice object from a file
The file format is indicated by the filepath extension. The file name is stored in
the *in_file* Lattice attribute. The selected variable, if relevant, is stored
in the *use* Lattice attribute.
Parameters:
filepath: Name of the file
Keyword Args:
use (str): Name of the variable containing the desired lattice.
Default: if there is a single variable, use it, otherwise select ``"RING"``
name (str): Name of the lattice.
Default: taken from the file, or ``""``
energy (float): Energy of the lattice
(default: taken from the file)
periodicity (int): Number of periods
(default: taken from the file, or 1)
*: All other keywords will be set as :py:class:`.Lattice`
attributes
Returns:
lattice (Lattice): New :py:class:`.Lattice` object
Check the format-specific function for specific keyword arguments:
.. Admonition:: Known extensions are:
.mat
Matlab binary mat-file. See :py:func:`.load_mat`.
.m
Matlab text m-file. See :py:func:`.load_m`.
.repr
Text representation of a python AT Lattice. See :py:func:`.load_repr`.
.lat
Tracy format. See :py:func:`.load_tracy`.
.lte
Elegant format. See :py:func:`.load_elegant`.
.json
JSON representation of a python AT Lattice. See :py:func:`.load_json`.
(END)
Anyway, I'll try and improve the help.
fixed order of the fields in json file
This is more difficult. The JSON spec states that the order of fields in the structure is irrelevant. In python I may have some control on the encoding, but in Matlab, the jsonencode
function does all the job without any control, except "compact or not".
@simoneliuzzo I added a signature in the JSON file, to identify the files created by AT. This will allow better error messages when trying to load a JSON file not created by AT. Please discard your existing files which are missing this signature
Dear @lfarv,
I think that being able to compare to json files is important. json file bring the advantage of being human readable, but they may be extremely big and users as me, at the very first test will want to use common tools such as diff/meld etc. to compare files and spot potential issues. This is why I think the order of the fields in the file should be deterministic (ex alphabetic order) and identical in the output of matlab and python.
Could you test that the modifications for json files did not harm any of the other possible file load/save? repr, mat, m, lat, lte etc... The test off all these cases starts to be complicated (many options to test) and may deserve an automated test as for other features of AT.
best regards Simone
@simoneliuzzo Forcing the field imposes a constraint which is not in the specification. That would force me to rewrite a pseudo-JSON encoder. This is against the idea of using a standard format with standard encoders. I think it's much safer to use the standard encoders provided by python and Matlab rather than rewriting a new one.
Now to compare JSON files, you can one of the many available 3rd party JSON comparators:
Otherwise, I'll add a few tests for other formats (and accordingly a few data files)
With this PR, AT can save and load lattices in JSON format. JSON is a text-based format with the following properties:
.mat
format.In addition, a few modifications have been made to the global load/save package to prepare future formats:
use=lattice_name
is now used for all formats, while the specific keywords (matkey=
,lattice_key=
etc.) are still understood for compatibility,mat_file
,elegant_file
... which records the input file name is replaced by a single one:in_file
. This avoids the accumulation of different attributes, with the consequence that the last loaded file could not be identified. The file name extension is enough to know which kind of file was loaded.Finally, the file loading structure is improved to allow an easier addition of new formats.