search

SuperDARN / pyDARNio

Python Library for read in SuperDARN data
GNU Lesser General Public License v3.0
8 stars 2 forks source link

SuperDARN Field Missing Error for Certain Fitacf Files #37

Closed SamRennie closed 2 years ago

SamRennie commented 2 years ago

Hi

I'm currently having an issue where using read_fitacf() with certain files - seems to be those from the Hankasalmi radar for years up to the late 2000s - returns the SuperDARNFieldMissingError, saying that the origin.command, origin.code, origin.time, and bmazm fields in record 0 are missing.

` import pydarnio import bz2 file = "/lustre/ahome3/s/sktr1/data/20060916.0000.28.han.fitacf.bz2"

with bz2.open(file) as fp: fitacf_stream = fp.read()

reader = pydarnio.SDarnRead(fitacf_stream, True) records = reader.read_fitacf() `

I haven't encountered the problem when using more recent files or files from different radars.

mts299 commented 2 years ago

@SamRennie I looked at the files via RST dmapdump and yes they are missing these fields which is very surprising!

However, if I grab the fitacf files from our data mirror they have the correct fields so maybe these files were not generated correctly.

I will email our files, but if you want to read these ones in your can also use read_dmap() instead where it does no checks on the files.

mts299 commented 2 years ago

Going to close this issue I believe it was solved. @SamRennie if you want to re-open it you can if you have more questions.

egthomas commented 2 years ago

@SamRennie the origin fields you indicated were added to the radar operating software (ROS) and radar software toolkit (RST) in mid-2006. So any files generated prior to that time, or before the local copy of the RST at Leicester (or ROS at Hankasalmi) was updated, would not include those three fields:

https://github.com/SuperDARN/ros-archive/blob/163682b43ec5d85219281518922c41b94e6bd75a/qnx/code/src.lib/superdarn/radar.1.09/include/rprm.h#L49-L56