flow_auto_qc() error when running on a flowSet

[wudustan]

When I run flow_auto_qc on a flowSet I receive the following error for one of my samples:

Error in read.FCS(tmp) : 
  file/var/folders/4m/g5mydqs14l9gk3f878gft1y0000kmf/T//RtmpsHDeuF/file26d57eb8ff47seems to be corrupted. 
 The actual number of cells in data section (1) is not consistent with keyword '$TOT' (0)
In addition: Warning message:
In readFCSdata(con, offsets, txt, transformation, which.lines, scale,  :
  Error in reading data stream for file '/var/folders/4m/g5mydqs14l9gk3f878gft1y0000kmf/T//RtmpsHDeuF/file26d57eb8ff47'/nData may be truncated!

It seems to be particularly associated with flowAI:::flow_rate_check

I've tried to troubleshoot the issue but I'm getting nowhere.

Looking at that sample specifically:

keyword(grp1.comp.fs[[10]], "$TOT") $$TOT [1] "0000000000000005000"

[DillonHammill]

@wudustan, this is not an issue with flowAI per se. As the error suggests for some reason there are fewer than expected events within this file when it is read into R with read.FCS() - $TOT stores the number of events contained within the file.

If you are able to read the file into a flowFrame/flowSet you take a look at the number of rows in the exprs slot and see if this agrees with the value stored in $TOT.

fr <- read.FCS("filename.fcs")
nrow(exprs(fr))

Another suggestion would be to switch to using the new flowWorkspace data structures which may address this issue. I would recommend using CytoExploreR for this and you will find flowAI is already implemented in the cyto_clean() function.

[JcGKitten]

Hey, maybe a little bit late to the party, but I ran in the same problem, for me it seems like 'flow_auto_qc' tries open a temporay file which is corrupted, cause opening the original file with flowCore worked and I called flowAI with the already opened flowSet:

> blub <- read.FCS("~/Downloads/fail.fcs")
> blub <- flowSet(blub)
> indices <- flow_auto_qc(blub, output = 3, remove_from = "FR_FM", ChExcludeFS = c("FSC", "SSC", "index"), ChExcludeFM = c("FSC", "SSC", "index"), fcs_QC = FALSE, html_report = "", mini_report = FALSE, folder_results = "~/Downloads/")
Quality control for the file: V1
Error in read.FCS(tmp) : 
  file/tmp/RtmpWoCtqu/file24ef02018f0a9seems to be corrupted. 
 The actual number of cells in data section (1) is not consistent with keyword '$TOT' (0)
In addition: Warning message:
In readFCSdata(con, offsets, txt, transformation, which.lines, scale,  :
  Error in reading data stream for file '/tmp/RtmpWoCtqu/file24ef02018f0a9'/nData may be truncated!

I couldn't find the code where this happens, so I don't know how to check my file further. It's an FCS2.0 file if this could be a problem.

Best wishes Max

[giannimonaco]

Hi Max,

you could try using the "which.lines" argument when reading the file with the function read.FCS. Check also this other discussion here: https://github.com/RGLab/flowCore/issues/108

Best wishes, Gianni

On Tue, Jan 10, 2023 at 5:55 PM Maximilian Bertsch @.***> wrote:

Hey, maybe a little bit late to the party, but I ran in the same problem, for me it seems like 'flow_auto_qc' tries open a temporay file which is corrupted, cause opening the original file with flowCore worked and I called flowAI with the already opened flowSet:

blub <- read.FCS("~/Downloads/fail.fcs")> blub <- flowSet(blub)> indices <- flow_auto_qc(blub, output = 3, remove_from = "FR_FM", ChExcludeFS = c("FSC", "SSC", "index"), ChExcludeFM = c("FSC", "SSC", "index"), fcs_QC = FALSE, html_report = "", mini_report = FALSE, folder_results = "~/Downloads/")Quality control for the file: V1Error in read.FCS(tmp) : file/tmp/RtmpWoCtqu/file24ef02018f0a9seems to be corrupted. The actual number of cells in data section (1) is not consistent with keyword '$TOT' (0)In addition: Warning message:In readFCSdata(con, offsets, txt, transformation, which.lines, scale, : Error in reading data stream for file '/tmp/RtmpWoCtqu/file24ef02018f0a9'/nData may be truncated!

I couldn't find the code where this happens, so I don't know how to check my file further. It's an FCS2.0 file if this could be a problem.

Best wishes Max

— Reply to this email directly, view it on GitHub https://github.com/giannimonaco/flowAI/issues/4#issuecomment-1377565137, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC2UTEFDFTY6DWY3PR5YAVDWRWIATANCNFSM5EOTQMXQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[jakesauter]

This worked for me:

sample <- ### Your flowFrame of interest

new_sample <- flowCore::flowFrame( exprs(sample) )

keyword(new_sample)["$TOT"] <- nrow(new_sample) keyword(new_sample)["$TIMESTEP"] <- keyword(sample)["$TIMESTEP"]

new_sample_fcs_file <- paste0(sample_name, ".fcs")

write.FCS( new_sample, new_sample_fcs_file )

flow_auto_qc( new_sample_fcs_file )

file.remove( new_sample_fcs_file )