Python TypeError in refgenome catalog step

[claumer]

Hi Matthew, Eoghan,

I'd like to try to use pore-C on some PCR amplified pore-C reads I have collected following (more or less) the protocol, to try to scaffold a reference genome I have which is good (contig N50 0.7 Mbp, ~500 contigs) but not great. Install seems to have worked as expected, no error messages to speak of. However, when I go to actually use poreC within its conda env, I get python errors:

(poreC) [claumer@noah-login-03 pore-c]$ pore_c refgenome catalog --genome-id DalyG /nfs/research1/marioni/claumer/DalyG_HiFi/DalyG_HiFi_q20_Flye/assembly.fasta DalyG_flye 2020-01-14 18:42:29,456 - pore_c - INFO - Adding reference genome under prefix: DalyG_flye 2020-01-14 18:42:29,489 - pore_c - INFO - Creating bgzipped reference: cat /nfs/research1/marioni/claumer/DalyG_HiFi/DalyG_HiFi_q20_Flye/assembly.fasta | bgzip > DalyG_flye.fa.gz Traceback (most recent call last): File "/nfs/research1/marioni/claumer/miniconda3/envs/poreC/bin/pore_c", line 11, in load_entry_point('pore-c', 'console_scripts', 'pore_c')() File "/nfs/research1/marioni/claumer/miniconda3/envs/poreC/lib/python3.7/site-packages/click/core.py", line 764, in call return self.main(args, kwargs) File "/nfs/research1/marioni/claumer/miniconda3/envs/poreC/lib/python3.7/site-packages/click/core.py", line 717, in main rv = self.invoke(ctx) File "/nfs/research1/marioni/claumer/miniconda3/envs/poreC/lib/python3.7/site-packages/click/core.py", line 1137, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/nfs/research1/marioni/claumer/miniconda3/envs/poreC/lib/python3.7/site-packages/click/core.py", line 1137, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/nfs/research1/marioni/claumer/miniconda3/envs/poreC/lib/python3.7/site-packages/click/core.py", line 956, in invoke return ctx.invoke(self.callback, ctx.params) File "/nfs/research1/marioni/claumer/miniconda3/envs/poreC/lib/python3.7/site-packages/click/core.py", line 555, in invoke return callback(args, kwargs) File "/nfs/research1/marioni/claumer/pore-c/src/pore_c/cli.py", line 114, in catalog rg_cat = ReferenceGenomeCatalog.create(file_paths, metadata, {}) File "/nfs/research1/marioni/claumer/pore-c/src/pore_c/catalogs.py", line 238, in create fh.write(yaml.dump(catalog_data, default_flow_style=False, sort_keys=False)) File "/homes/claumer/.local/lib/python3.7/site-packages/yaml/init.py", line 200, in dump return dump_all([data], stream, Dumper=Dumper, kwds) TypeError: dump_all() got an unexpected keyword argument 'sort_keys'

This command makes the following files:

-rw-r--r-- 1 claumer marioni 18967 Jan 14 18:42 DalyG_flye.chromsizes -rw-r--r-- 1 claumer marioni 0 Jan 14 18:42 DalyG_flye.catalog.yaml -rw-r--r-- 1 claumer marioni 18980 Jan 14 18:42 DalyG_flye.metadata.csv -rw-r--r-- 1 claumer marioni 35651 Jan 14 18:42 DalyG_flye.fa.gz.fai -rw-r--r-- 1 claumer marioni 58776 Jan 14 18:42 DalyG_flye.fa.gz.gzi -rw-r--r-- 1 claumer marioni 73650527 Jan 14 18:42 DalyG_flye.fa.gz

And you can see the catalog.yaml file is empty.

Please advise? I saw similar problems when I tried to run this through snakemake.

Regards, Chris L

[eharr]

Hi Chris, it looks like there might be something amiss with environment setup.

I think that the proximal problem is that an old version of pyyaml is being picked up by the python interpreter, however the version it's finding seems to be coming not from the conda environment but from a different location: /homes/claumer/.local/lib/python3.7/. Have you set the PYTHONPATH variable in your login shell by any chance?

[claumer]

Hi Eoghan,

Thanks for the quick response. Yes, I guess I do have some fossils from previous pythons in my home .local - but no $PYTHONPATH in the .bashrc, nor set anywhere else. But your intuition was right - the error was removed when moving those previous python files away from .local. Clearly it's time to clean house. I guess this issue can be considered fixed.

That said, I wonder if there's a way the poreC conda environment could be more robustly isolated from other python stuff that may be on other end users' systems? Speaking abstractly here because my conda-fu is somewhat basic...

Grazie mille, Chris L

[eharr]

Thanks Chris, I agree that the current process isn't ideal. I'm in the process of cleaning up some of the rough edges in the current version and the plan is to get it onto bioconda after that.