Closed leewujung closed 4 years ago
@ngkavin and I decided to move forward with:
Is there data in the XML file, or is it just configuration information? If there's no actual data, it seems unlikely including it in the netcdf / zarr would increase file size dramatically.
Is the EK80 XML file fairly flat, or does it have deep hierarchies? If the former, wouldn't it make more sense to convert from XML per se to netcdf/zarr attributes. Do you have an example EK80 XML you could point to?
Is there data in the XML file, or is it just configuration information? If there's no actual data, it seems unlikely including it in the netcdf / zarr would increase file size dramatically.
It depends... (as always!). Most part of the hierarchy is just config information, but for some files (see below), some transducer calibration parameters are also present and those are more like actual data.
The calibration parameters are currently not saved into netCDF/zarr. I don't think the original SONAR-netCDF has fields designed for them, but we can certainly create new data variables in the Vendor group (which is designed for saving anything additional that are specific to manufacturers). BUT if we were to store the XML string into a data variable, that'll get long.
Is the EK80 XML file fairly flat, or does it have deep hierarchies? If the former, wouldn't it make more sense to convert from XML per se to netcdf/zarr attributes. Do you have an example EK80 XML you could point to?
Here are two:
Thanks. How about loading those sample XML files on github? It doesn't have to be this repo; it could be a different repo in this organization, if you'd prefer. It just makes it easier to examine the XML files more quickly in more flexible ways ... Another option that's potentially easier to do but a bit more opaque is loading them as gists.
I think downloading the files and opening them in browser does the job. That's how I debugged the config header parsing problems. :)
Currently only the fields of the EK80 XML configuration datagram that correspond to requirements in the SONAR-netCDF convention is saved into netCDF/zarr. However, there may be other parameters that will be useful for users later. Therefore it seems a good idea to give users access to the entire datagram once the data is parsed.
@ngkavin and I discussed the options we'll move forward with:
@emiliom @lsetiawan @valentina-s : any suggestions welcome.