SuperDARN / DDWG

A repository for Data Distribution Working Group information
2 stars 1 forks source link

[Update] Hard Disks and differences in rawacf files [Reopened] #13

Closed mts299 closed 4 years ago

mts299 commented 4 years ago

Update: February 14, 2018


We've got another round of rawacf files updated here, this time it's han files from 200804, 200805, 200812, and 201210. The attached list of files were replaced on the mirror with original files without the 'fixrawacf' string in the dmap records. As always, if you have any questions, ask away. This is an ongoing issue discovered several years ago and you can refer to the VT website's DDWG page for more information as well.

Regards, Kevin Krieger

These files have been added onto the vt_mirror_clash.txt file found in the supplemental materials below.


Update: April 8, 2015


Today, files from 201303 Hokkaido have been updated on the mirror with the original file. Please update your repositories and re-produce secondary data products.

Regards, Kevin K.


Update: April 2, 2015


Again I have updated many files on the mirror with the original file provided by Kevin Sterne rather than the 'fixrawacf' version. Please synchronize and update your secondary data products when you get the new files. The files updated are for April 2013 through August 2013 (there were none after August), 2012, and January to February of 2011.

Regards, Kevin Krieger

Along with those updates, there were some files that were found empty on the mirror, and they have been removed as well as blacklisted. If there ever appears a file from the list that isn't empty, then we will of course place it on the mirror.

Regards, Kevin K.


Update: July 23, 2014


Kevin Sterne has reopened this issue after finding more and more differences here. Proposing that files without "fixrawacf" be posted on the USask data mirror if another more informative files exists. Even if it is just a change in the origin.command and origin.time variables, it may be more informative.

A list of files that Kevin has found that do not have "fixrawacf" was sent to the list and is posted below as vt_mirror_clash.txt.


This issue was raised as a much more general and board discussion of differences between files at various institutions. This followed issue #1 in which it was found that data from a few of the University of Alaska radars was being distributed as a different version than as what had been collected at the radar.

The discussion here got into a few details about special variables being contained in a few radars dmap records and not all of the records. However, some of these values depend on hdw.dat files so it seemed more of a problem of not having the correct hdw.dat file for a particular radar. It was brought up that a new hdw.dat repository had been started up and can be found here:

https://github.com/vtsuperdarn/hdw.dat

Kevin Krieger at University Saskatchewan and Kevin Sterne at Virginia Tech noted they were posting the latest version of their radars hdw.dat files to the github repository.

Action Items

http://vt.superdarn.org/tiki-index.php?page=Hdw.dat%20Updater

Supplemental Material

vt_mirror_clash.txt

emptyfiles_201504.txt

mts299 commented 4 years ago

Email send on: 2015-10-23 Hello DDWG members,

I've updated the following radars and months with non-fixrawacf versions, please update your repositories:

sys: 201209, 201210, 201211, 201212, 201301 hok: 201103* (only one file 20110328.2205) cve: 201103, 201104 cvw: 201103, 201104 fir: 201103, 201104 sto: 201103, 201104 bks: 201103, 201104 fhe: 201103, 201104 fhw: 201103, 201104 gbr: 201103, 201104 kap: 201103, 201104 wal: 201103, 201104

In addition, the following rawacf files have been updated from the PI institute. hok: 20150712.2201, 20150722.2201 pyk: 20150622.1001 sas: 20150927.0401

Regards, Kevin K.

mts299 commented 4 years ago

Email Date: 2015-07-20

Hello all,

I'm updating three months worth of sys files with non-fixrawacf versions. The months are 201302, 201303, and 201304. See attached the list of updated files on the mirror. In total there were 2137 files updated. Please sync and update your secondary data products.

Regards, Kevin K.

mts299 commented 4 years ago

Email: 2018-02-02

Hello DDWG,

We've got another round of rawacf files updated here, this time it's han files from 200804, 200805, 200812, and 201210. The attached list of files were replaced on the mirror with original files without the 'fixrawacf' string in the dmap records. As always, if you have any questions, ask away. This is an ongoing issue discovered several years ago and you can refer to the VT website's DDWG page for more information as well.

Regards, Kevin Krieger

kevinkrieger commented 4 years ago

This issue hasn't come up in a few years. It can be re-opened if more files are found with this 'fixrawacf' issue.