search

msr-consulting / exscalabar_server

Repository for the EXSCALABAR server.
http://www.msrconsults.com/ukmet-gh/exscalabar
0 stars 1 forks source link

Data file issues #146

Closed JustinLangridge closed 8 years ago

JustinLangridge commented 8 years ago

Some questions on the data file as it currently stands:

The mirror file is still not mirroring the main file as expected

lo-co commented 8 years ago

can we change to comma delimited instead of space delimited?

The file is tab delimited not space delimited. Still want me to change it?

What are Med and High?

Will look at this but I believe it refers to humidified PID output?

CRD_cell_0_T appears twice for each cell. Where are these coming from?

Don't know. Will look into it.

where are pRH_T, pinlet_T etc coming from?

Will look into it.

lo-co commented 8 years ago

We can unwind the tabs but is it necessary? Every program that can read csv files generally can read tab-delimited also. There are a lot of structures to go into; unfortunately I didn't think this through too well with regards to flexibility.

Also, the mirror file IS writing so I am not certain what the issue is there. It is still not reading the header though. Need to figure out why.

cathrynjfox commented 8 years ago

Variables that can be deleted from the data file: crd_cell_0_Q crd_cell_0_T crd_cell_0_RH crd_cell_0_P crd_cell_0_T crd_cell_0_extdilfac pRH_T pInlet_T pRedDry_T pGreenDry_T pBlueTD_T

cathrynjfox commented 8 years ago

Header writing on data file is a bit flaky - was writing data for MFCs but no header names (see file test_20160801_123218.txt). Now isn't writing any headers at all (see file test_20160802_121948.txt)

cathrynjfox commented 8 years ago

Headers now writing again... but still missing headers for some mfcs towards the end of the file (mfc_O3, mfc_O2, mfc_TAP, mfc_Purge, mfc_RedTD, mfc_BlueTD).

There are now 5 columns of data (all zeroes) before the Common.Time variable that I don't know what they are. Might be linked to the pressure controller. Can we write some header names for these 5 columns?

cathrynjfox commented 8 years ago

laserRMS and IABack seem to always be zero etau seems to always be NaN DenuderBypass flag not working

lo-co commented 8 years ago

OK...looks like I got the header/file writing issue fixed (at least in one place). The header is consistent and the data is not jagged. The problem was with some data emitters that were not registering properly (Humidifier.high and Humidifier.low). All emitters must register before we launch the file actor or the data header may not be written properly.

Still have an issue with the header not being written to both files. That being said, if you don't find the header in the main file, look in the mirror file. We have a race condition - first one who recognizes the Registration Complete flag in the MAP DVR wins. I will fix this next.

lo-co commented 8 years ago

So, I am going to add the tag File to indicate the file related issues. I am closing this now as the major issues associated with the file was the jaggedness and the lack of a header. I would like to open individual issues for the points above that cathrynjfox raised. To answer a couple of the questions above:

etau seems to always be NaN

This is an error that is calculated how I can't remember. It is computationally intensive and therefore is usually turned off by default.

laserRMS and IABack seem to always be zero

Will raise this in another issue.

DenuderBypass flag not working

Will raise this in another issue.

I am not going to worry about the redundant or not used entries for right now. We will take this on as lower priority.