Orthofinder stops at Calculating gene distances

[zephyris]

Hi, I'm having a problem running orthofinder, it seems to quit after starting to calculating gene distances without giving the normal output. I'm certain all programs are installed and running correctly, I can run orthofinder on another proteome set, it's just on this one where it seems to stop before giving an output.

An example console output, starting from pre-calculated BLASTs, is:

OrthoFinder version 2.2.7 Copyright (C) 2014 David Emms

2018-11-04 10:17:05 : Starting OrthoFinder
11 thread(s) for highly parallel tasks (BLAST searches etc.)
11 thread(s) for OrthoFinder algorithm

Checking required programs are installed
----------------------------------------
Test can run "mcl -h" - ok
Test can run "fastme -i [path]/Results_Nov02/WorkingDirectory/SimpleTest.phy -o [path]/Results_Nov02/WorkingDirectory/SimpleTest.tre" - ok
Using previously calculated BLAST results in [path]/Results_Nov02/WorkingDirectory/

Running OrthoFinder algorithm
-----------------------------
2018-11-04 10:17:10 : Initial processing of each species
...
2018-11-04 10:30:36 : Initial processing of species 41 complete
2018-11-04 10:32:16 : Connected putatitive homologs
2018-11-04 10:32:25 : Written final scores for species 10 to graph file
...
2018-11-04 10:36:49 : Written final scores for species 41 to graph file
[mcl] cut <226> instances of overlap
2018-11-04 22:31:00 : Ran MCL

Writing orthogroups to file
---------------------------
A duplicate accession was found using just first part: Chlre4_382136
Tried to use only the first part of the accession in order to list the sequences in each orthogroup
more concisely but these were not unique. The full accession line will be used instead.

Orthogroups have been written to tab-delimited files:
[path]/Results_Nov02/WorkingDirectory/Orthogroups.csv
[path]/eukaryotesFasta/Results_Nov02/WorkingDirectory/Orthogroups.txt (OrthoMCL format)
[path]/eukaryotesFasta/Results_Nov02/WorkingDirectory/Orthogroups_UnassignedGenes.csv
2018-11-04 22:31:29 : Done orthogroups

Analysing Orthogroups
=====================

Calculating gene distances

Do you think this is a silent error calculating gene distances due to the duplicate accession, or do you think it could be something else?

Thanks!

[davidemms]

Hi

There can be some issues with RAM using the '-a' option but the version you are using should alert you to any issues like that so that seems unlikely (you'd get a message starting, "ERROR: The computer ran out of RAM and killed OrthoFinder processes"). A couple of questions, are you using the source code version of the binary version of orthofinder? And what happens if you run it without the '-a 11' command line option? I'm wondering if there's an issue lurking in the binary version that I'm not aware of.

Thanks David

[zephyris]

Hey, I'm running the binary version. I didn't notice any RAM issues so I agree that's unlikely, I'll run it with without -a 11 and let you know though... If I get a chance I'll run the source code version on a parallel copy of the BLAST results too. Thanks again!

[zephyris]

The run without the -a parameter (still using the binary version of orthofinder) just failed identically: Same console output and same (lack) of output.

Time to try the python version?

[davidemms]

Yes, I'd try that. If it works or if it fails then let me know, it sounds like there's something I need to look into.

Thanks David

[zephyris]

I may have found an alternative explanation for my problems! I had a stray copy of clusters_OrthoFinder_v2.2.7_I1.5 in the Working directory, I guess from a run which crashed/failed somehow. Could that've been causing issues?

[davidemms]

Probably not, I don't think that would cause a problem.

[zephyris]

Hmm. Ok, this is definitely a peculiar one then - I can't get it to replicate reliably. The python version has consistently run with no problems. The compiled version has failed for this particular proteome set on all times I've tried it, except one.

[davidemms]

Would you be able to let me know the details of the machine it's running on and I will see if I can investigate:

• operating system and version
• kernel version
• libc version (run "ldd --version")
• How much RAM does it have

Thanks David

[zephyris]

Of course: Ubuntu 16.04.5 LTS running as a Linux subsystem for Windows 10 4.4.0-17134-Microsoft ldd (Ubuntu GLIBC 2.23-0ubuntu10) 2.23 32 Gb RAM

[YingyingYang2019]

Hi, I'm having a similar problem running orthofinder2. My OrthoFinder-2.3.3_source.tar.gz were downloaded from github, and my command are set like this: orthofinder.py -f pep_faa/ -t 90 -a 6 -S diamond My program was stuck for five days. 2019-06-28 00:58:00 : Done 72000 of 100048 2019-06-28 00:58:16 : Done 76000 of 100048 2019-06-28 00:58:20 : Done 77000 of 100048 2019-06-28 00:58:24 : Done 78000 of 100048 2019-06-28 00:58:28 : Done 79000 of 100048 2019-06-28 00:58:40 : Done 82000 of 100048 2019-06-28 00:58:44 : Done 83000 of 100048 2019-06-28 00:58:52 : Done 85000 of 100048 2019-06-28 00:59:11 : Done 90000 of 100048 2019-06-28 00:59:20 : Done 92000 of 100048 I donot know if it is normal because I have 185 species in my dataset. Should I wait for the next process or restart it with -fg?

[davidemms]

Hi

The best way to see is to look if anything is running using a command such as 'top' or 'pstree'. The most likely thing is that a particular command that has been launched by OrthoFinder is taking disproportionately long, most likely because the largest orthogroup is significantly bigger than all the other ones. If something is running you will see it using close to 100% cpu using top. If this is the case then I would definitely leave it running since restarting will probably take just put you 5 days behind where you currently are. Another thing to check is whether all the RAM has been used up on the computer, forcing it to use swap space. If this has happened it would slow things down considerably.

There is the nuclear option if you really can't wait. If you kill the process for inferring the tree that has been running for ages (i.e. not the orthofinder process itself!) then OrthoFinder will carry on with the remaining trees and then move onto the later steps of the algorithm. There is a big consequence of this though--no orthologues will be inferred for any of the genes in this orthogroup as OrthoFinder won't have the tree to examine in order to identify the orthologues. Taking this step isn't recommended.

All the best David

[YingyingYang2019]

Hi David,

Thank you very much for your reply! I have checked the cpu, and they were nearly 100% used. I would leave it running and keep waiting. Thank you very much! And since I didnot find your email address, I sent an email to Kelly Steven whose email address was provided by the article yesterday earlier. I am very very sorry for disturbing!

Best wishes! Yang

At 2019-07-03 17:34:06, "David Emms" notifications@github.com wrote:

Hi

The best way to see is to look if anything is running using a command such as 'top' or 'pstree'. The most likely thing is that a particular command that has been launched by OrthoFinder is taking disproportionately long, most likely because the largest orthogroup is significantly bigger than all the other ones. If something is running you will see it using close to 100% cpu using top. If this is the case then I would definitely leave it running since restarting will probably take just put you 5 days behind where you currently are. Another thing to check is whether all the RAM has been used up on the computer, forcing it to use swap space. If this has happened it would slow things down considerably.

There is the nuclear option if you really can't wait. If you kill the process for inferring the tree that has been running for ages (i.e. not the orthofinder process itself!) then OrthoFinder will carry on with the remaining trees and then move onto the later steps of the algorithm. There is a big consequence of this though--no orthologues will be inferred for any of the genes in this orthogroup as OrthoFinder won't have the tree to examine in order to identify the orthologues. Taking this step isn't recommended.

All the best David

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.