Closed sujaikumar closed 7 years ago
davidemms
commented
7 years ago Hi Sujai
There's a line in the file orthofinder/scripts/get_orthologues.py which says,
# print(dlcCommands[0])
Could you uncomment this line and post the line of text that it prints out. It'll be somewhere in the output and should start, "dlcpar_search -s".
Thanks David
sujaikumar
commented
7 years ago print(dlcCommands[0])resulted in:
dlcpar_search -s /scratch/skumar/maker_test/ExampleDataset/Results_Sep27/Orthologues_Sep27/WorkingDirectory/Trees_ids/SpeciesTree_ids_0_rooted.txt -S /scratch/skumar/maker_test/ExampleDataset/Results_Sep27/Orthologues_Sep27/WorkingDirectory/Trees_ids_arbitraryRoot/GeneMap.smap -D 1 -C 0.125 /scratch/skumar/maker_test/ExampleDataset/Results_Sep27/Orthologues_Sep27/WorkingDirectory/Trees_ids_arbitraryRoot/OG0000000_tree_id.txt -O /scratch/skumar/maker_test/ExampleDataset/Results_Sep27/Orthologues_Sep27/WorkingDirectory/dlcpar/OG0000000_tree_idOn running this (from the same location as where I ran orthofinder.py from, with the same env:
Traceback (most recent call last):
File "/exports/virt_env/python/orthofinder/bin/dlcpar_search", line 209, in <module>
sys.exit(main())
File "/exports/virt_env/python/orthofinder/bin/dlcpar_search", line 137, in main
gene2species = phylo.read_gene2species(options.smap)
File "/exports/virt_env/python/orthofinder/lib/python2.7/site-packages/dlcpar/deps/compbio/phylo.py", line 94, in read_gene2species
util.open_stream(filename))))
File "/exports/virt_env/python/orthofinder/lib/python2.7/site-packages/dlcpar/deps/rasmus/util.py", line 1170, in open_stream
stream = open(filename, mode)
IOError: [Errno 2] No such file or directory: '/scratch/skumar/maker_test/ExampleDataset/Results_Sep27/Orthologues_Sep27/WorkingDirectory/Trees_ids_arbitraryRoot/GeneMap.smap'I checked, and only this file exists:
/scratch/skumar/maker_test/ExampleDataset/Results_Sep27/Orthologues_Sep27/WorkingDirectory/Trees_ids/SpeciesTree_ids_0_rooted.txtand none of these files exist:
/scratch/skumar/maker_test/ExampleDataset/Results_Sep27/Orthologues_Sep27/WorkingDirectory/Trees_ids_arbitraryRoot/GeneMap.smap
/scratch/skumar/maker_test/ExampleDataset/Results_Sep27/Orthologues_Sep27/WorkingDirectory/Trees_ids_arbitraryRoot/OG0000000_tree_id.txt
/scratch/skumar/maker_test/ExampleDataset/Results_Sep27/Orthologues_Sep27/WorkingDirectory/dlcpar/OG0000000_tree_idYes, it's a good idea to try running the command. The command line looks exactly right so I don't think the mangled path is coming from OrthoFinder: '/scratch/skumar/Tests/Input/ExampleDataset/Results_Sep26/Orthologues_Sep26/WorkingDirectory/Trees_ids_arbitraryRoot/OG0000010_tree_id.txt/scratch/skumar/Tests/Input/ExampleDataset/Results_Sep26/Orthologues_Sep26/WorkingDirectory/dlcpar/OG0000010_tree_id.coal.tree'
Those files you were looking for are deleted by OrthoFinder at the end of a run so as to clean up after itself.
I think the problem might be that you have two conflicting parts of dlcpar from two different versions. I've checked the line in dlcpar that is generating the error above:
File "/exports/virt_env/python/orthofinder/lib/python2.7/site-packages/dlcpar/deps/rasmus/util.py", line 1170, in open_stream stream = open(filename, mode)
and this version of the util.py file comes from the 0.9.7 version that is on github whereas I believe you're now using the 0.9.1 version of dlcpar_search that is available from the webpage (http://compbio.mit.edu/dlcpar/). I think that when you installed the 0.9.1 version it didn't delete or overwrite the 0.9.7 version and this is causing the conflict! Can you remove them and install 0.9.1 again from a clean state? It's possible that the issue with 0.9.7 was also an installation problem, I'll check whether that version works for me.
davidemms
commented
7 years ago Note, stream = open(filename, mode) is on line 1170 in dlcpar version 0.9.7 (and in the error message above) whereas it is on line 1209 in version 0.9.1. If I give dlcpar a non-existent output path then the error gets thrown from line 1209, hence why I think there might be an dlcpar installation problem.
sujaikumar
commented
7 years ago Thanks very much for the detective work. Yes, that's exactly what the problem was. For future reference, if anyone wants to uninstall dlcpar (or another python package) COMPLETELY, you need to do this:
I had uninstalled dlcpar using
pip uninstall dlcparBut got an error msg saying
DEPRECATION: Uninstalling a distutils installed project (dlcpar) has been deprecated and will be removed in a future version. This is due to the fact that uninstalling a distutils project will only partially uninstall the project.which I had ignored.
This time, I uninstalled everything using:
python setup.py install --record files.txtAnd then deleting all the files/directories in files.txt using:
parallel rm -rf :::: files.txt # gnu parallel is wonderful for things like thisThen I reinstalled it:
wget http://compbio.mit.edu/dlcpar/pub/sw/dlcpar-0.9.1.tar.gz
tar xzf dlcpar-0.9.1.tar.gz
python setup.py installAnd orthofinder worked perfectly, generating all the genetrees!
Thanks again!
Hi David
I tried running orthofinder (checked out today at 5pm), with the new functionality, and am getting the errors below even with your Tests/Input/ExampleDataset of 4 Mycoplasma protein files.
Any thoughts on what could be causing the errors? The program gives errors during Running Orthologue Prediction, Step 5, but then seems to finish correctly.
However, when I then look at Species-by-species orthologues: .../Tests/Input/ExampleDataset/Results_Sep26/Orthologues_Sep26/Orthologues/Orthologues_Mycoplasma_/ - the files there just have headers and nothing else in them.
From the error messages below, it looks like it is trying to create an output directory which is misnamed: eg
(which has the /scratch/skumar/.... bit in it twice)
Thanks,
Sujai
ps. A dump of the full ExampleDataset folder as run below is at: ftp://ftp.ed.ac.uk/edupload/ExampleDataset.tar.gz
(error messages continue for each Orthogroup)