search

bryanhanson / readJDX

Import spectroscopic data in the JCAMP-DX format
https://bryanhanson.github.io/readJDX/
8 stars 2 forks source link

Is FIRSTY really necessary? #1

Closed rguliev closed 7 years ago

rguliev commented 7 years ago

Hi, I was testing your function for handling ',' as a decimal separator. The file I used for test is attached (NOTE. original extension of the file is '.dx', I just had to change it because I couldn't attach it otherwise). I got an error message:

Error in if (firstY == 0) stop("Couldn't find FIRSTY") : 
  argument is of length zero

After I added manually a row with "##FIRSTY=", it worked fine. The question is FIRSTY really that necessary? For example, read.jdx function from hyperSpec package can read a file without FIRSTY.

Information about the file. The file contains baseline corrected SERS (Surface Enhanced Raman Spectroscopy) spectrum. ',' as a separator, numeric (not integer) values and FIRSTY absence appeared after baseline correction. Baseline correction were performed by Spekwin32 v1.72 software. Location: Moscow, Russia.

example firstY.txt

bryanhanson commented 7 years ago

Just yesterday I tried a file I got from Claudia which had the same issue, and I discovered exactly what you did. I went back to the original publication and it clearly states that FIRSTY is a required field (it should be used as an internal check). But, since there are obviously files out there w/o FIRSTY, I can make the check optional. I’ll look into that in the next few days.

Bryan

On Nov 3, 2016, at 1:05 PM, Rustam notifications@github.com wrote:

Hi, I was testing your function for handling ',' as a decimal separator. The file I used for test is attached (NOTE. original extension of the file is '.dx', I just had to change it because I couldn't attach it otherwise). I got an error message:

Error in if (firstY == 0) stop("Couldn't find FIRSTY") : argument is of length zero After I added manually a row with "##FIRSTY=", it worked fine. The question is FIRSTY really that necessary? For example, read.jdx function from hyperSpec package can read a file without FIRSTY.

Information about the file. The file contains baseline corrected SERS (Surface Enhanced Raman Spectroscopy) spectrum. ',' as a separator, numeric (not integer) values and FIRSTY absence appeared after baseline correction. Baseline correction were performed by Spekwin32 v1.72 software. Location: Moscow, Russia.

example firstY.txt https://github.com/bryanhanson/readJDX/files/569392/example.firstY.txt — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/bryanhanson/readJDX/issues/1, or mute the thread https://github.com/notifications/unsubscribe-auth/AAIkPkwk45t0H1z-dD_5SphlVO6NEXRZks5q6hRUgaJpZM4KoqPC.

bryanhanson commented 7 years ago

FIRSTY is required by the standard and it is used in readJDX to check that the actual first y value extracted from the data table matches the FIRSTY given in the metadata. This is an important check of data integrity and probably shouldn't be overridden w/o a lot of thought. I'm thinking there might be some value in having an ignoreChecks argument which would skip all internal checks (except perhaps that the number of x values found is the same as the number of y values found; this would throw an error even if it wasn't checked since one can't make a data frame with unequal numbers of x and y). But in the hands of a naive user this would be problematic. I'll need to review all the internal checks to see exactly what would be skipped.

rguliev commented 7 years ago

Today I checked how Bruker:OPUS software would handle it. It turned out that OPUS loads such files with no problem. Maybe it would be better if the function just will throw a warning message in such cases?

bryanhanson commented 7 years ago

Ah, that sounds like a better idea. readJDX could warn in certain cases. Let me think on it a bit. Thanks so much for checking your files and software - the more examples we have the better.

bryanhanson commented 7 years ago

As of version 0.1.32 (https://github.com/bryanhanson/readJDX/commit/0ea68070ca573437abd3067bcc5202fe80e52fd2) there is now an argument "SOFC" meaning 'Stop on Failed Check", with default TRUE. If you set SOFC = FALSE the check of FIRSTY will be skipped with a warning. The internal mechanism used will allow some other kinds of checks to be skipped as well (though none currently are implemented).

I'll close for now, please re-open if it doesn't seem to work for you.