nolanlab / citrus

Citrus Development Code
GNU General Public License v3.0
31 stars 20 forks source link

Error in Studio after citrus.launchUI() #99

Closed LisaMarieKronstad closed 7 years ago

LisaMarieKronstad commented 7 years ago

After using citrus successfully in 2015, my lab now receives the following output after entering the citrus.launchUI() command and selecting the folder with the .fcs files.

"Launching citrus interface with target directory: /Users/BlishLab/Desktop/LMK_CyTOF_Rawn

Scanning parameters in FCS files Reading parameters in M_H1_24_Ligand_160518_23_normalized.fcs the text section does not end with delimiter: . The last keyword is dropped. the text section does not end with delimiter: . The last keyword is dropped."

and then the following error

"Error in value[3L] : Unexpected Error: Error in pnames[sapply(pnames, nchar) < 3] = parameterNames[sapply(pnames, : NAs are not allowed in subscripted assignments"

Any help would be greatly appreciated. Thank you.

rbruggner commented 7 years ago

This sounds like the parameter metadata in the FCS file is malformed and hence, Citrus cannot read it using the flowCore package. I would suggest trying to see if you can read the FCS file using flowCore directly. Something like:

library("flowCore")
file = read.fcs("/PATH/TO/FILE.FCS")
pData(parameters(file))

If that fails, then you will need to fix the FCS file before flowCore (and Citrus) will be able to read it.

LisaMarieKronstad commented 7 years ago

Hi Rob,

Thank you for your response,

Lisa Kronstad, Ph.D. Postdoctoral Scholar Blish Laboratory | Grant Building S164 Department of Medicine | Division of Infectious Diseases Stanford University

On Jul 25, 2016, at 5:00 PM, Robert Bruggner notifications@github.com<mailto:notifications@github.com> wrote:

This sounds like the parameter metadata in the FCS file is malformed and hence, Citrus cannot read it using the flowCore package. I would suggest trying to see if you can read the FCS file using flowCore directly. Something like:

library("flowCore") file = read.fcs("/PATH/TO/FILE.FCS") pData(parameters(file))

If that fails, then you will need to fix the FCS file before flowCore (and Citrus) will be able to read it.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/nolanlab/citrus/issues/99#issuecomment-235122867, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AQPx_gCRjzAAxRj6RV3HB1fsNwA6bcjkks5qZU4AgaJpZM4JUqET.

LisaMarieKronstad commented 7 years ago

I ended up renaming my files from .fcs --> .FCS.

Then ran this code:

library("flowCore") file = read.FCS("/Users/BlishLab/Desktop/Folder") pData(parameters(file))

Now it is working - thanks :)

benostendorf commented 7 years ago

Unfortunately I ran into the same problem. FCS-files exported either directly from FACSDiva or re-exported from Flojo (either 8.7 or 10.0.7) cannot be read into Citrus and give this error message:

Error in value[3L] : Unexpected Error: Error in pnames[sapply(pnames, nchar) < 3] = parameterNames[sapply(pnames, : NAs are not allowed in subscripted assignments

The files from FACSDiva can be read by the flowcore package and the pData command also works on them. Any help is greatly appreciated!

SamGG commented 7 years ago

Hi, The code is at https://github.com/nolanlab/citrus/blob/0d308ab77719aaba0eed3a4983300ec72396cf31/R/citrus.launchUI.R#L56-L58 Once you read your FCS file using flowCore, investigate

(parameterNames = colnames(fcsFile))
(pnames = as.vector(pData(flowCore::parameters(fcsFile))$desc))
sapply(pnames,nchar)<3  # is there any NA here?
benostendorf commented 7 years ago

Hi Sam, thanks a lot for your prompt reply! Yes, in fact there's a couple of NA's there. I guess I need to get rid of all the non-defined parameters there?

SamGG commented 7 years ago

Well it depends if you want to build the analysis on those parameters. If not confidential, could you post the output of the command?

benostendorf commented 7 years ago

Sure:

sapply(pnames,nchar)<3 # is there any NA here?

FITC_F4_80 PerCP-Cy5_5_Ly6G NA NA NA NA NA FALSE NA FALSE APC MHCII BV605 BV711 BV785 FALSE NA FALSE FALSE NA FALSE FALSE FALSE BUV395 PE mCherry PE-Cy7 FALSE NA NA FALSE FALSE NA FALSE
SamGG commented 7 years ago

Without a fixed font, the resulting list is not crystal clear. I think that if you insert before line 58 something like, it should work. It is intended to replace NA description by original channel name.

pnames[is.na(pnames,nchar)] = parameterNames[is.na(pnames,nchar)]

So, please do run the following code on your FCS flowFrame. There is no error reported, and pnames should sounds great.

  parameterNames = flowCore::colnames(fcsFile)
  pnames = as.vector(pData(flowCore::parameters(fcsFile))$desc)
  pnames[is.na(pnames,nchar)] = parameterNames[is.na(pnames,nchar)]
  pnames[sapply(pnames,nchar)<3] = parameterNames[sapply(pnames,nchar)<3]
  # Check
  pnames

Once checked, I think the quickest workaround is to start a new copy of Citrus (just fork it), then insert the new line in the code, and finally install Citrus on your computer using your new repository. I will be off for one hour...

benostendorf commented 7 years ago

Great, after inserting

pnames[is.na(pnames)] = parameterNames[is.na(names)]

before line 58 the files are properly read in (I deleted the ",nchar", since is.na complained about passing on two arguments). One strange thing is that now all parameters are displayed twice (some even three times) in the GUI. Thanks a lot and I'll post how it goes!

benostendorf commented 7 years ago

Currently the clustering fails with this error message

Error in if ((!is.null(fileSampleSize)) && (fileSampleSize < nrow(fcsData))) { : missing value where TRUE/FALSE needed

Any pointers to what I should look at? I was also wondering what the "Scale parameters" settings do? Thanks a lot!

SamGG commented 7 years ago

Sorry for the extra ",nchar". Bad copy/paste. Concerning the last error, either fileSampleSize is NA, either nrow(fcsData) is NA. Both are weird. May be the problem arises from the previous change. I think we should try to avoid any replacement in the getFileParamters function. To do so, insert the following line

  parameterNames = flowCore::colnames(fcsFile)
  return(parameterNames)  # line to insert
  pnames = as.vector(pData(flowCore::parameters(fcsFile))$desc)

For the scaling, I think Robert is the right person. HTH

benostendorf commented 7 years ago

I'm afraid the same error message persists after adding the line above..

Error in if ((!is.null(fileSampleSize)) && (fileSampleSize < nrow(fcsData))) { : missing value where TRUE/FALSE needed

SamGG commented 7 years ago

At https://github.com/bostendorf/citrus/blob/master/R/citrus.util.R#L82 put

browser()

And start debugging... ie when the Citrus stops because the debugger starts, try to guess if any of fileSampleSize or nrow(fcsData) is NA. If both variables look alright, then try piece of the code that raises an error. Alternatively, you could attach a sampling of your FCS file to this thread. The mentioned scaling consists in centering and dividing by standard deviation. If each column (aka channel) of your fcs files covers the same scale, I don't think you need this scaling. Depending on the amount of positive and negative events, scaling could lead to very different results although the original data cover the same scale.

rbruggner commented 7 years ago

If there’s a problematic FCS file that you can share with me, I can potentially take a crack at fixing.

On Sep 19, 2016, at 6:17 AM, bostendorf notifications@github.com wrote:

Unfortunately I ran into the same problem. FCS-files exported either directly from FACSDiva or re-exported from Flojo (either 8.7 or 10.0.7) cannot be read into Citrus and give this error message:

Error in value[3L] x-msg://22/cond : Unexpected Error: Error in pnames[sapply(pnames, nchar) < 3] = parameterNames[sapply(pnames, : NAs are not allowed in subscripted assignments

The files from FACSDiva can be read by the flowcore package and the pData command also works on them. Any help is greatly appreciated!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/nolanlab/citrus/issues/99#issuecomment-247990116, or mute the thread https://github.com/notifications/unsubscribe-auth/ABFhGvjhBKdXvdVdKOS0GrLofbRmPwPvks5qrotngaJpZM4JUqET.

benostendorf commented 7 years ago

Thanks a lot for the offer, Robert. I'm sending you a file as a PM.

rbruggner commented 7 years ago

I've made a new branch called ndDscrpt (bad name, I know) but it attempts to handle NA's in the FCS file parameter descriptions. You should be able to install it with:

install_github("nolanlab/citrus",ref="naDscrpt")

Please give it a try and let me know if it works. If so, will merge with master branch.

benostendorf commented 7 years ago

Thanks so much for looking into this, Robert! Unfortunately, now the clustering aborts with R encountering a fatal error. I attached links to one sample .fcs file as well as the R-script generated by Citrus. Please let me know if there is anything else I should provide.

Thanks again! Ben

Link to example fcs-file https://dl.dropboxusercontent.com/u/4270380/tumor_A_1_004.fcs

On Oct 5, 2016, at 1:31 AM, Robert Bruggner notifications@github.com wrote:

I've made a new branch called ndDscrpt (bad name, I know) but it attempts to handle NA's in the FCS file parameter descriptions. You should be able to install it with:

install_github("nolanlab/citrus",ref="naDscrpt")

Please give it a try and let me know if it works. If so, will merge with master branch.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/nolanlab/citrus/issues/99#issuecomment-251585682, or mute the thread https://github.com/notifications/unsubscribe-auth/AVQ2mtwz-PxuDuWdexJFZotoLHCjHWCXks5qwzY1gaJpZM4JUqET.

rbruggner commented 7 years ago

Can you attach an error log (the output of the citrus run) as well please?

zhanxw commented 7 years ago

Same problems as @bostendorf. I installed citrus from source codes from branch naDscrpt. R crashed when I started to run example 1.

The console output is:

Reading Condition defaultCondition Reading file Patient01_healthy.fcs Reading file Patient02_healthy.fcs Reading file Patient03_healthy.fcs Reading file Patient04_healthy.fcs Reading file Patient05_healthy.fcs Reading file Patient06_healthy.fcs Reading file Patient07_healthy.fcs Reading file Patient08_healthy.fcs Reading file Patient09_healthy.fcs Reading file Patient10_healthy.fcs Reading file Patient11_diseased.fcs Reading file Patient12_diseased.fcs Reading file Patient13_diseased.fcs Reading file Patient14_diseased.fcs Reading file Patient15_diseased.fcs Reading file Patient16_diseased.fcs Reading file Patient17_diseased.fcs Reading file Patient18_diseased.fcs Reading file Patient19_diseased.fcs Reading file Patient20_diseased.fcs Clustering 20000 events

R crashed at citrus.full() function.

zhanxw commented 7 years ago

I also tried to install citrus from github master branch. Citrus also crashed when analyzing example 1. How to fix the crash problem?

rbruggner commented 7 years ago

@zhanxw This sounds like a different problem. Can you please open a new issue and provide details of the operating system, R session info, screenshots, etc etc?

rbruggner commented 7 years ago

Thread, AFAICT, the primary issue in this thread was the handling of NAs in parameter descriptions and I believe the naDscrpt branch fixes that problem, but would love confirmation.

I'm going to close this for the moment, but if it turns out that it's related or the NA problem isn't fixed, can re-open.

benostendorf commented 7 years ago

I'm sorry for the late reply - I can confirm that the naDscrpt branch seems to fix the problem, thanks very much for the quick fix!

zhanxw commented 7 years ago

It also fixed the problem in my analysis. Thanks.

On Mon, Oct 17, 2016 at 8:59 AM, bostendorf notifications@github.com wrote:

I'm sorry for the late reply - I can confirm that the naDscrpt branch seems to fix the problem, thanks very much for the quick fix!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nolanlab/citrus/issues/99#issuecomment-254215290, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJoiHyKAo_abADTGKkI20sSSE3VOqVuks5q038rgaJpZM4JUqET .

rbruggner commented 7 years ago

Ok - will merge with master. If you're still having trouble with citrus crashing, please feel free to open a new issue.