sgt_constructor.py not working, FileNotFoundError: [Errno 2] No such file or directory

[pyongheng]

Hi, I am getting an error when running sgt_constructor.py on Linux x64. I can start the run with an M1 mac but with the RAxML problem, it cannot complete correctly. I am also using a larger data set now. I am using phylofisher Version: 1.2.0, trying to update using mamba with 'mamba update fisher' returns "All requested packages already installed"

Log of the run. phylofisher2.log

config.ini

[PATHS] database_folder = /flash/HusnikU/databases/PhyloFisherDatabase_v1.0/database/ input_file = /flash/HusnikU/Phua_2/haplozoon/phylofisher/input_metadata.tsv orthomcl = /flash/HusnikU/databases/PhyloFisherDatabase_v1.0/database/orthomcl color_conf = /flash/HusnikU/databases/PhyloFisherDatabase_v1.0/database/tree_colors.tsv

input_metadata.tsv first 5 lines, everything else has the same format

Location    File Name   Unique ID   Higher Taxonomy Lower Taxonomy  Blast Seed  Long Name   Data Type   Source
.   transdecoder.fas    Haplo   Alveolata   Dinoflagellata  none    Haplozoonsp.    Transcriptomic  HaploSmartSeq
.   Abedfoli.fasta  Abedfoli    Alveolata   Dinoflagellata  none    AbediniumfoliumDICHO1703TRINITYpep  Transcriptomic  DICHO1703TRINITYpep
.   Alexande.fasta  Alexande    Alveolata   Dinoflagellata  none    AlexandriumandersoniiCCMP2222EP00969pep Transcriptomic  Dinodata
.   Alexmarg.fasta  Alexmarg    Alveolata   Dinoflagellata  none    AlexandriummargalefiiCS322MMETSP0661TRINITYpep  Transcriptomic  Dinodata

[robert-ervin-jones]

Hi @pyongheng,

Let's start with updating phylofisher. The command to do so is mamba update phylofisher. Once you get phylofisher updated to the latest version, please try sgt_constructor.py again and let us know if it worked.

Best, Robert

[pyongheng]

Hi Robert,

I was able to update phylofisher. sgt_constructor.py still gives the same error

Thank you

[atice]

Could you please send us the command you are using to invoke sgt_constructor.py.

Alex

[pyongheng]

The code I used is below

sgt_constructor.py -i working_dataset_constructor_out_Oct.30.2022 -t 64

before that were these

conda activate fisher
fisher.py -t 64 -o fisher_out_Oct.30.2022
informant.py -i fisher_out_Oct.30.2022
working_dataset_constructor.py -i fisher_out_Oct.30.2022 -o working_dataset_constructor_out_Oct.30.2022

[atice]

Could you clarify if you started sgt_constructor.py on a MacOS and moved all data over to Linux64 and restarted it or if you started the script on Linux64 and got this error.

[pyongheng]

I tried it both ways, working daataset files generated from the mac moved to the linux and when that did not work, I restarted the whole process using the linux.

[atice]

Could you send us a .tar.gz file of the logs directory in sgt_constructor_out_Oct.30.2022 from the run you started on linux please.

[pyongheng]

A stg output folder was not created when running stg_constructor.py.

i have attached the logs from .snakemake folder if this helps

logs.tar.gz

[atice]

Could you delete all out directories produced by sgt_constructor.py you have on your linux64 system and rerun the script using the command you provided above and report to us what happens.

[pyongheng]

I am getting the same error after removing the folder. Logs attached.

2022-11-03T123239.644151.snakemake.log phylofisher2.log

[robert-ervin-jones]

Can you please compress the directory /flash/HusnikU/Phua_2/haplozoon/phylofisher/sgt_constructor_out_Nov.03.2022/logs and send it to us?

[atice]

@pyongheng

If no out directories were created, would you be willing to meet with us virtually and share your screen while you run the program to better help us help you. We think this would be the most efficient way for us to diagnose the problem and get you moving forward on your project. Let us know.

Alex and Robert

[pyongheng]

Yes, that would be great. May I know when you be a good time for you? I am in Japan time zone (GMT+9) but I am flexible about timing. (I'm usually up till 2am)

[robert-ervin-jones]

Would 10PM your time (8AM our time) on Tuesday, Nov 8th work for you? Our emails are rej110@msstate.edu and at1422@msstate.edu. If you don't mind, please send us an email with a meeting link (any platform is fine with us). Please let us know ahead of time which platform so we can install the necessary software if need be.

[pyongheng]

Traceback (most recent call last): File "/home/y/yong-phua/mambaforge-pypy3/envs/fisher/lib/python3.7/site-packages/snakemake/init.py", line 780, in snakemake keepmetadata=keep_metadata, File "/home/y/yong-phua/mambaforge-pypy3/envs/fisher/lib/python3.7/site-packages/snakemake/workflow.py", line 1037, in execute logger.run_info("\n".join(dag.stats())) File "/home/y/yong-phua/mambaforge-pypy3/envs/fisher/lib/python3.7/site-packages/snakemake/dag.py", line 2159, in stats yield tabulate(rows, headers="keys") File "/home/y/yong-phua/mambaforge-pypy3/envs/fisher/lib/python3.7/site-packages/tabulate/init.py", line 2049, in tabulate tabular_data, headers, showindex=showindex File "/home/y/yong-phua/mambaforge-pypy3/envs/fisher/lib/python3.7/site-packages/tabulate/init.py", line 1471, in _normalize_tabular_data rows = list(map(lambda r: r if _is_separating_line(r) else list(r), rows)) File "/home/y/yong-phua/mambaforge-pypy3/envs/fisher/lib/python3.7/site-packages/tabulate/init.py", line 1471, in rows = list(map(lambda r: r if _is_separating_line(r) else list(r), rows)) File "/home/y/yong-phua/mambaforge-pypy3/envs/fisher/lib/python3.7/site-packages/tabulate/init.py", line 107, in _is_separating_line (len(row) >= 1 and row[0] == SEPARATING_LINE) File "/home/y/yong-phua/mambaforge-pypy3/envs/fisher/lib/python3.7/site-packages/snakemake/rules.py", line 1120, in eq return self.name == other.name and self.output == other.output AttributeError: 'str' object has no attribute 'name'

[michoug]

Hi got the same issue as above with the final error as AttributeError: 'str' object has no attribute 'name' I'm using ubuntu 64 and the latest version of your software 1.2.2 Best Greg

[robert-ervin-jones]

Hi Greg,

We were able to figure out that the problem is with the latest version of Snakemake (v6.5.3). I am currently working on downgrading the version packaged with phylofisher to v6.1.0.

I will let you know as soon as v1.2.3 of PhyloFisher is available. It will also include a few other updates alongside the downgraded Snakemake.

Best, Robert

[robert-ervin-jones]

Hi all. You should now be able to upgrade to PhyloFisher v1.2.3