Open schraivo opened 2 years ago
flowCore is carrying out checks that FJ does not. The error seems to be located where read.FCS is checking whether the measurements are in their range or not (global min and channel max). This test seems to occur when there is no transformation set (but still unsure).
As a rule, I always set the following when using read.FCS transformation = FALSE, emptyValue = TRUE, min.limit = NULL, truncate_max_range = FALSE
. I think this could help while waiting for a better answer from Mike or Greg.
https://github.com/RGLab/flowCore/blob/ba3b6ffed5310c1c0618487ab163c0142d8cab8f/R/IO.R#L1230-L1258
Hi Sam, the error is gone with truncate_max_range = FALSE
. Thanks! Would be curious to know which values were out of range, in order to understand the error. Let's see what Mike or Greg think.
Your Time channel has a reported range of 9.223372e+18 but data in your time channel go up to 3.378476e+38 and some values are NA. We tend to keep things pretty strict about reading valid FCS files where the data make sense and leave it up to the user to knowingly shoot themselves in the foot. That said the messaging here could be much better, and this should be a warning not an error. The bug arises because we aren't treating the unexpected NA values properly when trying to truncate with truncate_max_range=TRUE. We'll get a fix in.
Noting here until I can check out the repo:
Error is in this secton:
any(idx)
returns NA
when idx
contains NA
.
should likely be:
any(na.omit(idx))
same for the section with min.limit
https://github.com/RGLab/flowCore/blob/14781e79b2774e30933a949664a91a25491ea503/R/IO.R#L1234-L1258
For curiosity and if you have time, how a NA results from a FCS file?
@SamGG a great question:
length(which(is.na(dat[,i])))
[1] 528
for that time channel.
The data already contains NA
after
the readBin
call
parameters passed to that are:
readBin(con = con, what = "numeric", n = 17453527, size = 4, signed = TRUE, endian = "little")
And it's not just that it's trying to read more than it should... the length of the read data is 17453527 and the max index that's NA is 15418653. Probably another instrument maker writing bad files. If this happens with the time channel it's a wonder anybody can trust such data.
It's an FCS file from an instrument prototype. Happy to continue discussing offline, as this will include confidential information. Thanks everyone for your support and quick responses.
@gfinak bingo, you identified a new comer! @schraivo NA are precisely NaN; devel should take care when converting 64 bits integers; please close the issue and remove your link if you feel so.
I still think we should test for NA and error more informatively. Will reopen until I push a fix.
Describe the bug When trying to read an FCS file, read.FCS throws an error:
I can not detect anything wrong with the file. FlowJo opens the file without issues. Already checked this issue, which looks similar, but couldn't reproduce the solution: https://github.com/RGLab/flowCore/issues/123
To Reproduce Download FCS file: https://oc.embl.de/index.php/s/F0rTI92aOkzofWc
Expected behavior There should be no error.
sessionInfo():