search

SuperDARN / DDWG

A repository for Data Distribution Working Group information
2 stars 1 forks source link

[Update] 20061229.2200.00.san.rawacf.bz2 #14

Open mts299 opened 4 years ago

mts299 commented 4 years ago

Files

20061229.2200.00.san.rawacf.bz2

Mirrors To be Updated

Description

Recently I found some files that made their way onto the mirror with no minutes in the file name. Example: instead of 20061229.2200.00.san.rawacf.bz2 they are 20061229.22.00.san.rawacf.bz2

There are 2083 san files from the beginning of July 2006 to the end of Dec 2006. These files were added to the mirror by me back in September 2016 when I went through all the old CDs and DVDs of data here, looking for files that were not in the distribution already.

Tasks

kevinkrieger commented 4 years ago

These were blocked at USASK. It's likely possible to get them back into the distribution if someone can spend some time and rename these files properly, according to the first record. HELP WANTED

egthomas commented 4 years ago

Were these rawacf files produced on-site, or converted from dat files using the dattorawacf utility? If the latter, then it is understandable why the minute information may be missing from the file names. We have some bash scripts which could probably be adapted to address this issue, using logic along the lines of

# Convert dat file to rawacf format
dattorawacf $basefile.dat > tmp.rawacf

# Get file date and hour from filename
date=`echo $file | cut -c1-8`
hour=`echo $file | cut -c9-10`

# Get minute and second of first record in file
dmapdump tmp.rawacf > rawacf.dump
minutes=`sed -n -e 's/^.*time.mt" =//p' rawacf.dump`
minute=`echo $minutes | cut -d " " -f1`
seconds=`sed -n -e 's/^.*time.sc" =//p' rawacf.dump`
second=`echo $seconds | cut -d " " -f1`

# Add leading zeros to minute and second if necessary
minute=$(printf %02d $minute)
second=$(printf %02d $second)

# Convert old single-letter radar code to 3-letter code
case $site in
    ...

# Construct YYYYMMDD.HHMN.SS.RRR filename format
baseraw="$date.$hour$minute.$second.$newsite"

# Rename rawacf file with correct format
mv tmp.rawacf $baseraw.rawacf
kevinkrieger commented 4 years ago

Hmm, from testing one file it appears they are dattorawacf files that were distributed on optical media:

string "origin.time" = "Tue Sep 25 15:17:32 2007" string "origin.command" = "dattorawacf 2006083114d0.dat"

tjk584 commented 1 year ago

None on NSSC mirror