[mView] A point at 0s will ocasionally be logged

[nmGit]

Either the datalogger will log or the grapher will display a data point at 0s UTC, which is very wrong and messes up the data

[roboguy222]

This is something @amopremcak has seen a lot. Is it when it is writing to AFS? We think it may be a problem with that.

[nmGit]

Afs as well as locally

[nmGit]

Note: If this is an AFS issue, then keep the logs locally, otherwise, transfer the Leiden log files to AFS at some point (when we get it fixed).

[nmGit]

Seems to sometimes happen when mView crashes.

Note that it permanently damages the dataset (not just adding a point at 0,0). The dataset will go from four vaiables to over 5000.

[amopremcak]

What is mView?

This all sounds rather fishy. When you say goes from four variables to over 4000 what do you mean? Are you saying that

d.getVariables()
indepVarsList = varsList[0]
depVarsList = varsList[1]
print "Total number of variables =", (len(indepVarsList) + len(depVarsList))

goes from printing 4 to 5000?

From what you were saying a few days ago, it was that the data being returned by getData() was being corrupted, growing in length from the expected length (total number of writes to the dataset at that point in time) to something absurd. If this is the case, then you should use the getNumRows() method to monitor the size of the dataset after each call to getData(). Check that the value returned by getNumRows() is the correct length and that the length of the data returned by getData() is equal to the value returned by getNumRows(). If you observe a discrepancy between these two values, then we are in business.

[nmGit]

mView is the gui-making tool that i've been working on (someone hasnt read the readme haha).

What I'm saying is that yes, independent variables + dependent variables will go from 4 (for example) to >5000.

More specifically

len (dataset.getData () [-1])

goes from four to >5000

EDIT: I will try getNumRows ()

[amopremcak]

Haha, guilty as charged. Just to be certain, you should check that indeed len(indepVarsList) + len(depVarsList) is equal to some number greater than the original quantity as len (dataset.getData () [-1]) being larger is not equivalent to the above condition from the programs standpoint. Also, you should print/store what the values are of indepVarsList + depVarsList (not the total lengths but the actual lists). Do you have any datasets that this happened to that I can look at?

[amopremcak]

Also, you should look at the shape of dataset.getData () in addition to dataset.getData ()[-1]. Maybe even try to save the array using numpy.savez or something like that when this happens again.

[amopremcak]

@nmGit, you mentioned that Nick was able to reproduce this issue. I would like you to get to the bottom of this if possible. I was able to reproduce this behavior when writing to AFS, but I have yet to do so locally. It seems to me that this issue and your GUI are somehow intimately related. I wrote a script for you in order to produce this error (see /dataChest/dataCorruptionTest.py), the only downside is you must look at all 100 files using the grapher to determine if the dataset was corrupted. Alternatively, you could write a script to check each of these files (make sure that the total number of points is correct AND that each value is indeed what we expect).

[nmGit]

Ok, I will take a look. NicK and I replicared the issue when we were still logging to AFS. I have not seen the error since.

[amopremcak]

Hold off on this for a couple of hours. I'm going to edit the script to check all of the files after it finishes the writing portion. I'll ping you @nmGit when it's ready.

[amopremcak]

The deed is done @nmGit. Run this script each day, but run it in a dedicated terminal so that you can look at any/all things that the script outputs once it is finished (don't just double click the file). It checks that each file contains the correct number of rows and that each row of each files has the anticipated value as a function of index. As of now it will take ~13-14 hours to complete. Modify it as needed.

[amopremcak]

Also @roboguy222, can we get your two files out of the \servers\dataChest folder? I wanna clean up this folder to keep it as "library like" as possible.

[amopremcak]

@nmGit Whats going on with this? I would like you to run this script once a day for the next week to see if this occurs.

[nmGit]

I havent gotten a chance yet to sit down and run it, I will do that asap.

[nmGit]

@amopremcak I just finished running the script for both local and remote (afs) data storage. The local test ran without error, but the afs test failed.

[amopremcak]

Can you keep running this for the next week or so. I am interested to see whether or not we can make this happen locally.

[nmGit]

Ya, for sure. mView It has been running and logging locally in the DR lab and in the Leiden room since I left. I havent seen a problem with either of them before that though (I regularly leave them running for several days on end)

[amopremcak]

I would rather have the script running because it tests 100 files as opposed to just a handful.