search

NCAR / WVD-MCSupdate

Addition of NCAR MCS, rewrite to software infrastructure, and expansion of features for WVD operations.
2 stars 1 forks source link

MCS writes lots of extra (unnecessary) bites in 2nd line #217

Closed stillwer closed 5 years ago

stillwer commented 6 years ago

As it stands currently, the MCS writes all data in the following form:

Time Stamp Channel Assignments MCS Hex String

These channel assignments are written for each data channel. Currently...all 6 channels are written for each channel. At bare minimum, this makes the raw files bigger than needed. This might not be important when the files get zipped, but I have not tested that. Also, this unpacking is hardwired into the Python code used to unpack the files, which makes it less flexible than desired.

Propose a fix where only the channel to which the data belongs is written.

stillwer commented 5 years ago

Turns out this is a problem for power files too. Also...it causes the power files to not be made if the numbers are 10, 11, or 12 because the python functions are looking at exact bytes. I think.

stillwer commented 5 years ago

Current plan is to use a single byte x the number of channels to put a map into the data file. Will try to implement this in the coming week.

stillwer commented 5 years ago

Power files are fixed and working with the structure defined in the "NewPowerFileStructure.pdf" document. This solves the inability of the python to handle channels 10 and 11 and also makes the files smaller. Raw files are 57% the original size...zipped raw files are 4% smaller...netcdf files are the same for now.

NewPowerFileStructure.pdf

stillwer commented 5 years ago

Both the power and data files are now writing the shorter header version of the data. For now, both versions are being written but only the shorter version is being converted to netcdf.