Switching to json?

[lassoan]

Slicer5 is moving towards saving markups as json files. This has many advantages:

• more metadata can be saved (for each point, or for the entire list)
• multiple fiducial lists can be saved in a single file
• various markup types can be saved in a file (not just points but lines, curves, planes, etc.)
• json schema file can be created, which can be used for automatic validation of the file (presence of certain labels, values, etc.)
• unlike csv, json is very well defined file format (no questions about what separators are used - comma, semicolon, tab; or how to escape quotation marks and separator characters; or how to encode international characters, etc.)

We would keep supporting loading of .fcsv files and for now fiducials are still saved in .fcsv file by default, but we would want to transition to .json format as much as possible to take advantage of all the features that it offers.

What do you think about this transition? What tools do you currently use to edit or process fcsv files? How difficult would it be to update your toolchain to work with json files instead of/in addition to fcsv?

We can discuss this here, but maybe the best would be if you could join this discussion: https://discourse.slicer.org/t/markupsline-fcsv-loads-as-markupsfiducials/11545/33

[kaitj]

What do you think about this transition? What tools do you currently use to edit or process fcsv files? How difficult would it be to update your toolchain to work with json files instead of/in addition to fcsv?

This sounds great for us on the validator development end. With the majority of the code in Python, we are currently storing relevant information in an json object after reading in the fcsv file (skipping a few lines as someone in the discussion mentioned) as its easier for us to work with. A change in file format would likely be pretty simple to update once we know what keys to look for and would likely allow us to skip a few steps for users using Slicer5.

We'll be happy to contribute to the discussion!

[jclauneuro]

Hi @lassoan thanks for looping our team in. I agree with @kaitj's response above. At this point, we will adapt to whatever specs are decided on for Slicer5, but ultimately we have interest in easily being able to convert between Slicer's Markup file format and one that would be compatible with the Brain Imaging Data Structure (BIDS) specifications.

This may be a good time to try to determine if there is a point of convergence between the needs of both Slicer and BIDS for handling coordinates in .json files that might eliminate the need for any conversion tool. I will provide more details in the discussion on discourse.

[jclauneuro]

In addition to the original thread on discourse above, I've made the following one: https://discourse.slicer.org/t/switching-to-json-for-markup-files-any-convergence-with-bids/

[jclauneuro]

Please see thread for details. For now, let's assume that BIDS and Slicer5 representations of points in .json will develop independently and require a converter tool to go between formats.

[lassoan]

Thanks for the feedback, this is a reasonable plan.