search

Nesvilab / FragPipe

A cross-platform proteomics data analysis suite
http://fragpipe.nesvilab.org
Other
195 stars 38 forks source link

Changing where temp files are written #1785

Closed JadeyMac closed 3 weeks ago

JadeyMac commented 1 month ago

Hi! I have an older computer with minimal local drive space. Everything else runs perfectly fine, but my runs are starting to crash because temp files are not able to be accessed because they are being written on the local drive, which doesn't have enough space. I have deleted everything I can from the local drive, and still don't have enough space for these temp files when I do a big run. Is there a way in the user interface to tell MSFragger/Fragpipe to write the temporary files on my data drive instead?

fcyu commented 1 month ago

Yes.

For Windows: https://github.com/Nesvilab/FragPipe/issues/306#issuecomment-777693128

For Linux: https://github.com/Nesvilab/FragPipe/issues/755#issuecomment-1180889147

Best,

Fengchao

JadeyMac commented 4 weeks ago

Thank you so much for your response!

I was able to change where temp files are written,but now I'm getting a different issue with the temp file.

Here is the tail end of my log with the error at the bottom. It is now giving an error saying that it can't delete a temp file because it is in use.

Do you have any suggestions?

Thanks!

NFO: Processing standard MixtureModel ... Initialising statistical models ... Iterations: .........10.........20..... WARNING: Mixture model quality test failed for charge (6+). WARNING: Mixture model quality test failed for charge (7+). model complete after 26 iterations INFO[13:16:42] Done
Process 'PeptideProphet' finished, exit code: 0 INFO[13:15:10] Executing PeptideProphet v5.0.0
Unknown file type. No file loaded.E:\Users\Taylor\FraggerOut\20240918_DCIS\AA_04/20240717_DCIS_TSH_C3_04_3_Slot1-16_1_3639.mzBIN_calibrated Unknown file type. No file loaded.E:\Users\Taylor\FraggerOut\20240918_DCIS\AA_04/20240717_DCIS_TSH_C3_04_3_Slot1-16_1_3639.mzBIN_calibrated file 1: E:\Users\Taylor\FraggerOut\20240918_DCIS\AA_04\20240717_DCIS_TSH_C3_04_3_Slot1-16_1_3639.pepXML processed altogether 45937 results INFO: Results written to file: E:\Users\Taylor\FraggerOut\20240918_DCIS\AA_04\interact-20240717_DCIS_TSH_C3_04_3_Slot1-16_1_3639.pep.xml

using Accurate Mass Bins using PPM mass difference Using Decoy Label "rev_". Decoy Probabilities will be reported. Not using ntt model Not using nmc model Using non-parametric distributions (X! Tandem) (using Tandem's expectation score for modeling) adding ACCMASS mixture distribution using search_offsets in ACCMASS mixture distr: 0.0 init with X! Tandem nonspecific PeptideProphet (TPP v5.2.1-dev Flammagenitus, Build 201906281613-exported (Windows_NT-x86_64)) AKeller@ISB read in 7028 1+, 24711 2+, 11555 3+, 2366 4+, 277 5+, 0 6+, and 0 7+ spectra. Found 17321 Decoys, and 28616 Non-Decoys MS Instrument info: Manufacturer: UNKNOWN, Model: UNKNOWN, Ionization: UNKNOWN, Analyzer: UNKNOWN, Detector: UNKNOWN

INFO: Processing standard MixtureModel ... Initialising statistical models ... Iterations: .........10.........20..... WARNING: Mixture model quality test failed for charge (6+). WARNING: Mixture model quality test failed for charge (7+). model complete after 26 iterations INFO[13:16:46] Done
Process 'PeptideProphet' finished, exit code: 0 INFO[13:15:10] Executing PeptideProphet v5.0.0
Unknown file type. No file loaded.E:\Users\Taylor\FraggerOut\20240918_DCIS\AA_01/20240717_DCIS_TSH_C3_01_2_Slot1-12_1_3620.mzBIN_calibrated Unknown file type. No file loaded.E:\Users\Taylor\FraggerOut\20240918_DCIS\AA_01/20240717_DCIS_TSH_C3_01_2_Slot1-12_1_3620.mzBIN_calibrated file 1: E:\Users\Taylor\FraggerOut\20240918_DCIS\AA_01\20240717_DCIS_TSH_C3_01_2_Slot1-12_1_3620.pepXML processed altogether 46829 results INFO: Results written to file: E:\Users\Taylor\FraggerOut\20240918_DCIS\AA_01\interact-20240717_DCIS_TSH_C3_01_2_Slot1-12_1_3620.pep.xml

using Accurate Mass Bins using PPM mass difference Using Decoy Label "rev_". Decoy Probabilities will be reported. Not using ntt model Not using nmc model Using non-parametric distributions (X! Tandem) (using Tandem's expectation score for modeling) adding ACCMASS mixture distribution using search_offsets in ACCMASS mixture distr: 0.0 init with X! Tandem nonspecific PeptideProphet (TPP v5.2.1-dev Flammagenitus, Build 201906281613-exported (Windows_NT-x86_64)) AKeller@ISB read in 6501 1+, 24175 2+, 12378 3+, 3160 4+, 615 5+, 0 6+, and 0 7+ spectra. Found 17544 Decoys, and 29285 Non-Decoys MS Instrument info: Manufacturer: UNKNOWN, Model: UNKNOWN, Ionization: UNKNOWN, Analyzer: UNKNOWN, Detector: UNKNOWN

INFO: Processing standard MixtureModel ... Initialising statistical models ... Iterations: .........10.........20..... WARNING: Mixture model quality test failed for charge (6+). WARNING: Mixture model quality test failed for charge (7+). model complete after 26 iterations INFO[13:16:50] Done
Process 'PeptideProphet' finished, exit code: 0 INFO[13:15:10] Executing PeptideProphet v5.0.0
Unknown file type. No file loaded.E:\Users\Taylor\FraggerOut\20240918_DCIS\AA_01/20240717_DCIS_TSH_C3_01_3_Slot1-12_1_3621.mzBIN_calibrated Unknown file type. No file loaded.E:\Users\Taylor\FraggerOut\20240918_DCIS\AA_01/20240717_DCIS_TSH_C3_01_3_Slot1-12_1_3621.mzBIN_calibrated file 1: E:\Users\Taylor\FraggerOut\20240918_DCIS\AA_01\20240717_DCIS_TSH_C3_01_3_Slot1-12_1_3621.pepXML processed altogether 46987 results INFO: Results written to file: E:\Users\Taylor\FraggerOut\20240918_DCIS\AA_01\interact-20240717_DCIS_TSH_C3_01_3_Slot1-12_1_3621.pep.xml

using Accurate Mass Bins using PPM mass difference Using Decoy Label "rev_". Decoy Probabilities will be reported. Not using ntt model Not using nmc model Using non-parametric distributions (X! Tandem) (using Tandem's expectation score for modeling) adding ACCMASS mixture distribution using search_offsets in ACCMASS mixture distr: 0.0 init with X! Tandem nonspecific PeptideProphet (TPP v5.2.1-dev Flammagenitus, Build 201906281613-exported (Windows_NT-x86_64)) AKeller@ISB read in 6583 1+, 24329 2+, 12445 3+, 3034 4+, 596 5+, 0 6+, and 0 7+ spectra. Found 17610 Decoys, and 29377 Non-Decoys MS Instrument info: Manufacturer: UNKNOWN, Model: UNKNOWN, Ionization: UNKNOWN, Analyzer: UNKNOWN, Detector: UNKNOWN

INFO: Processing standard MixtureModel ... Initialising statistical models ... Iterations: .........10.........20..... WARNING: Mixture model quality test failed for charge (6+). WARNING: Mixture model quality test failed for charge (7+). model complete after 26 iterations panic: remove E:\TempFiles\277e9256-eaab-45fe-99dd-895a653dc7fa\PeptideProphetParser.exe: Access is denied.

goroutine 1 [running]: github.com/Nesvilab/philosopher/lib/msg.callLogrus({0xc0024bc0e0, 0x64}, {0xf64557?, 0x1?})       D:/Projects/philosopher/philosopher/lib/msg/msg.go:327 +0x2ed github.com/Nesvilab/philosopher/lib/msg.Custom({0x10d1f00?, 0xc0000e4120?}, {0xf64557, 0x5})       D:/Projects/philosopher/philosopher/lib/msg/msg.go:15 +0x6e github.com/Nesvilab/philosopher/lib/met.CleanTemp({0xc000020800?, 0x24?})       D:/Projects/philosopher/philosopher/lib/met/met.go:484 +0x36 github.com/Nesvilab/philosopher/cmd.glob..func12(0x2b756c0?, {0xc000000300, 0x1, 0xc})       D:/Projects/philosopher/philosopher/cmd/peproph.go:30 +0xe7 github.com/spf13/cobra.(Command).execute(0x2b756c0, {0xc000000240, 0xc, 0xc})       D:/Projects/philosopher/philosopher/vendor/github.com/spf13/cobra/command.go:920 +0x847 github.com/spf13/cobra.(Command).ExecuteC(0x2b76520)       D:/Projects/philosopher/philosopher/vendor/github.com/spf13/cobra/command.go:1044 +0x3bd github.com/spf13/cobra.(*Command).Execute(...)       D:/Projects/philosopher/philosopher/vendor/github.com/spf13/cobra/command.go:968 github.com/Nesvilab/philosopher/cmd.Execute()       D:/Projects/philosopher/philosopher/cmd/root.go:35 +0x25 main.main()       D:/Projects/philosopher/philosopher/main.go:23 +0x85 ERRO[13:16:50] remove E:\TempFiles\277e9256-eaab-45fe-99dd-895a653dc7fa\PeptideProphetParser.exe: Access is denied. Process 'PeptideProphet' finished, exit code: 2 Process returned non-zero exit code, stopping Processing interrupted, stopping PeptideProphet Processing interrupted, stopping PeptideProphet Processing interrupted, stopping PeptideProphet Processing interrupted, stopping PeptideProphet Processing interrupted, stopping PeptideProphet Processing interrupted, stopping PeptideProphet Processing interrupted, stopping PeptideProphet Processing interrupted, stopping PeptideProphet Processing interrupted, stopping PeptideProphet Processing interrupted, stopping PeptideProphet Processing interrupted, stopping PeptideProphet Processing interrupted, stopping PeptideProphet Processing interrupted, stopping PeptideProphet Processing interrupted, stopping PeptideProphet Processing interrupted, stopping PeptideProphet Processing interrupted, stopping PeptideProphet Processing interrupted, stopping PeptideProphet Processing interrupted, stopping PeptideProphet 13:16:50 WARN : Attempt #1 to delete file we need to delete is being used by another process: E:\Users\Taylor\FraggerOut\20240918_DCIS\AA_03\fragpipe-20240717_DCIS_TSH_C3_03_3_Slot1-15_1_3635.pepXML-temp


Cancelling 279 remaining tasks
fcyu commented 4 weeks ago

It seems that there was antivirus software or file system issue preventing the files from being deleted. But I think this issue has been fixed in the latest FragPipe, but you are using an old version.

Best,

Fengchao

JadeyMac commented 4 weeks ago

Is there a way I can fix this issue in version 20? We already published half of the cohort using this version and my PI is adverse to updating.

Is there a reason why we weren't getting this issue before? Does this error occur with specific settings? Can I change antivirus settings? Or do you have any other suggestions I could try?

fcyu commented 4 weeks ago

It was due to your computer or operating system, and probably happens at random. You will need to troubleshoot locally.

Best,

Fengchao