Export to IAF

[mlnorthernswe]

Followed the wiki about creating a baseline with gui: https://github.com/geomagpy/magpy/wiki/GUI:-Producing-a-baseline I tried to export the month to IAF format but nothing happens. Export to other formats seems to work.

[leonro]

Sorry for not replying earlier. Will check your issue later this week and come back to you. I had the same problem recently and already added a quick fix, which is commited so far only to the development branch. Will need to check and test it.

[leonro]

The following changes will be contained in version 1.1.2 to correct and simplify the construction of IAF files in comparison to previous versions of MagPy:

• IAF files can already be exported correctly even if less than one year of data is provided.

• IAF contains different analytical steps: beside one-minute, one-hour and daily averages, the export routine will now also calculate K values (FMI method) in case such data is not provided along with the export call. Providing external K values for export is only possible using the backend method. Please note: in case of K value calculation you need to provide data for one extra day at the beginning and at the end of the sequence to correctly calculate such values

• IAF export is testing the validity of some meta parameters like coordinates and previously aborted without notification if such meta data was not provided or invalid (even if the meta data was not essential for generating IAF). Now IAF data will be exported, invalid data will be replaced i.e. by zero and a notification will be issued, only in the backend so far however.

• By default IAF export is also creating a skeleton of a README.IMO file.

• Loading hourly means, daily means or k values from IAF files is currently supported in the backends "read" method. A selection menu for XMagPy is on my TODO list.

• There are new example files included within the testing environment (magpy/test/testdata/baseline). Using xmapy you can load the adopted basevalues "ex_baseval.txt". Then load one-month (plus two extra days) of variometer data "ex_variomin.cdf". Got to panel "Analysis", apply "Baseline", got to panel data and apply "Baseline correction". Then export as IAF etc. If you want to apply the baseline directly when using "Analysis"-"Baseline" then to top menu "Options" -> "Basic ..." and select "Apply baseline" -> "directly" (active after restarting XMagPy).

• The examples files contain coordinates in a Gauss-Krueger system. If pyproj is properly installed on your system these coordinates will be automatically transferred to WGS84 when exporting IAF. If not you should provide WGS84 data within panel "Meta" -> "Data related", otherwise zero values will be exported.

The main changes can be found here: 20dda573a713d838668f89f5172415552b0ceac6 9a260237b3eec666474418a84b64773a8047adc9 2cf65c8573e6840cd1587aeffdc7526ba458962a

[leonro]

Version 1.1.2 will go online soon, most likely later today. Provided I don't get any additional requests I will close this issue 4 weeks after publication of 1.1.2.

[mlnorthernswe]

Hello again, thanks for your effort.

I tested with the example data and it works of course.

However with our data I couldn't get it to work. No IAF-file is produced. Export to pycdf seems to work.

If it can be of any help our DI-data and variometer-data comes below in the links: https://www2.irf.se/maggraphs/absolute_measurements/ https://www2.irf.se/maggraphs/vario_scalar/ When I tested I did the following: Loaded DI-data-files (from https://www2.irf.se/maggraphs/absolute_measurements/): 2023-02-24_08-41-23_MIR2_KIR.txt 2023-03-08_09-42-05_MIR2_KIR.txt 2023-03-10_10-42-05_MIR2_KIR.txt
2023-03-17_10-02-10_MIR2_KIR.txt 2023-03-20_06-42-09_MIR2_KIR.txt 2023-03-23_08-59-09_MIR2_KIR.txt 2023-03-27_08-28-55_MIR2_KIR.txt 2023-04-06_11-30-05_MIR2_KIR.txt The variometerdata (and scalar data) used is from https://www2.irf.se/maggraphs/vario_scalar/ Did the step 1 och 2 in https://github.com/geomagpy/magpy/wiki/GUI:-Producing-a-baseline Did the step 3 and loaded variometer files from kir20230228vsec.sec to kir20230401vsec.sec. Did step 4.

Did step 6 and tested to export to IAF but it didn't succeeed. Export to pycdf as mentioned seems to succeed.

I would appreciate if you have time to do the tests on our data.

/Regards Mats

[leonro]

Hi Mats, I am currently downloading your data, will test it later today and come back to you. Just a quick question: The IAF format is a one-minute data product. Have you filtered your one-second source to a one-minute resolution before step 6? Best wishes, Roman

[mlnorthernswe]

Ok, I thought that was done in the export. There we have the problem then probably. I will test again with filtered one-minute data.

[leonro]

Just tested with your data. After filtering (panel "Analysis" -> "Filter", using IAGA default) export to IAF seems to work. As K values are determined when exporting, you might want to modify the K9 limit on panel "Meta" -> "Station related" before exporting. I will include a notification for failing exports into the next version. Please check whether it is working for you.

[mlnorthernswe]

Yes, it works now! Made a note about that data possibly must be filtered before export in the wiki. Good that you mentioned the Meta-panel. Thank you for your time!

[leonro]

Added a notification to the report panel of xmagpy in case of failing exports. see d46de58e316751986e3450bfd8e6b72fb84a3b85

Will close this issue now.