Shirling-VT commented 3 years ago

Writing of SuperDARN data files

Name: Writing of SuperDARN data files

module: fitacf_convert.py

package: utils

Scope

Create a check list using markdowns checklist syntax on the scope of the feature

What will the feature do?

Convert fitacf files to data formats such as hdf5, netCDF, or csv which have been commonly used in the space science community.

Description

PyDARNio is supposed to incorporate the features of reading and writing of SuperDARN data. The writing part is mostly missing except for the borealis radars. Writing the fitacf (grid and map) files into data formats that are commonly used by our community would significantly improve the usage of SuperDARN data. And it is convenient for users to combine SuperDARN data with data from other instruments. In addition, it would be great if the geolocation information of each backscatter echo could be included in the output files instead of range gates for the fitacf files.

pyDARNio Checklist

[x] Does this fit within pyDARNio's scope?
[ ] Is this a minor change i.e., new small utility function?
[x] Is this a major change i.e., new plotting function?
[x] Do you need help developing it?
[ ] Have you created a pyDARNio GitHub project to show the tasks needed to be done?

Development help

pyDARNio does not have a dedicated development team on hand for full feature development. However, if you are able to provide some time and help, the community will try to aid you in the best way they can to make your feature possible.

If you need help please indicate what expertise you are looking for:

[x] developer help with design and integration of code
[x] scientific help with ensuring what you develop is scientifically correct/accurate
[x] project management to help organize what steps need to be taken to make this possible

Please also provide any other information you may need help with, not including testing/reviewers of code

User Interface

Please provide pseudocode on how you see your new feature interacting with the code or user-interface

Extra Notes

Please provide any other details about this feature

https://github.com/vtsuperdarn/SD_exodus

alexchartier commented 3 years ago

see: https://github.com/alexchartier/superdarn raw_to_nc.py (converts rawACF and fitACF to netCDFs). Feel free to incorporate in pydarnio with appropriate attribution

mts299 commented 3 years ago

Thank you @alexchartier I will look into it. Also feel free to develop in pydarnio or pydarn as I see the repo has similar code.

Just reminder Davitpy is deprecated and pydarn is the new official python data visualization software maintained by Data Visualization Working Group. You can email me if you have any comments or questions. :)

alexchartier commented 3 years ago

@mts299 I found that a lot of useful stuff from DavitPy is not present in pydarn, so that's why I went with the old package (DavitPy). If you or @Shirling-VT want to incorporate my netCDF code into your packages, I would be glad to see that happen, but I wasn't able to use Pydarn for my purposes (as in the geolocation of fitACF-level data, or plotting the solar terminator).

Sorry to cross-post, but the more serious issue is the conversion of Borealis-style rawACF files to DMAP seems to be broken (see issue #27 ) - that is currently standing between us and a Wallops radar upgrade. We could use some testing routines or "standard" files to help debug those routines, or at least tell us when they're broken