quaeler
commented
2 years ago Perhaps adding comments to sites definitions is not valid; a number of the additions in the 0.0.28 tag have these now. ?
Closed quaeler closed 2 years ago
quaeler
commented
2 years ago Perhaps adding comments to sites definitions is not valid; a number of the additions in the 0.0.28 tag have these now. ?
rmcolq
commented
2 years ago Could you try updating scorpio? Using 'pangolin --update' should do it.
Sent from my Galaxy
-------- Original message -------- From: loki der quaeler @.> Date: 08/12/2021 18:44 (GMT+00:00) To: cov-lineages/constellations @.> Cc: Subscribed @.***> Subject: [cov-lineages/constellations] Latest constellations and/or designations creates pangolin failure (Issue #44)
This email was sent to you by someone outside the University. You should only click on links or attachments if you are certain that the email is genuine and the content is safe.
Unfortunately i cannot provide the source FASTA file, but using a non-qc-failing FASTA generated by ARTIC with Pangolin supplied by the staphb/pangolin:3.1.16-pangolearn-2021-11-25 image, and then database-updated to:
2021-12-08T10:17:46.991-08:00 Updating pangolin database...
2021-12-08T10:18:17.687-08:00 pangolearn already latest release (2021-11-25)
2021-12-08T10:18:17.687-08:00 constellations updated to v0.0.28
2021-12-08T10:18:17.687-08:00 pango-designation updated to v1.2.108Produces the following errors when running Pangolin:
2021-12-08T10:18:18.820-08:00 Traceback (most recent call last):
2021-12-08T10:18:18.820-08:00 File "/miniconda/envs/pangolin/bin/scorpio", line 8, in <module>
2021-12-08T10:18:18.820-08:00 sys.exit(main())
2021-12-08T10:18:18.820-08:00 File "/miniconda/envs/pangolin/lib/python3.8/site-packages/scorpio/__main__.py", line 273, in main
2021-12-08T10:18:18.820-08:00 args.func(args)
2021-12-08T10:18:18.820-08:00 File "/miniconda/envs/pangolin/lib/python3.8/site-packages/scorpio/subcommands/list.py", line 7, in run
2021-12-08T10:18:18.820-08:00 list_constellations(options.constellations,
2021-12-08T10:18:18.820-08:00 File "/miniconda/envs/pangolin/lib/python3.8/site-packages/scorpio/scripts/type_constellations.py", line 1080, in list_constellations
2021-12-08T10:18:18.820-08:00 parse_variants_in(reference_seq, features_dict, constellation_file, constellation_names, label=label)
2021-12-08T10:18:18.820-08:00 File "/miniconda/envs/pangolin/lib/python3.8/site-packages/scorpio/scripts/type_constellations.py", line 403, in parse_variants_in
2021-12-08T10:18:18.820-08:00 variant_list, name, output_name, rule_dict, mrca_lineage, incompatible_lineage_calls, parent_lineage, lineage_name = parse_json_in(refseq, features_dict, variants_file, constellation_names, include_ancestral=include_ancestral,label=label,ignore_fails=ignore_fails)
2021-12-08T10:18:18.820-08:00 File "/miniconda/envs/pangolin/lib/python3.8/site-packages/scorpio/scripts/type_constellations.py", line 283, in parse_json_in
2021-12-08T10:18:18.820-08:00 record = variant_to_variant_record(site, refseq, features_dict, ignore_fails=ignore_fails)
2021-12-08T10:18:18.820-08:00 File "/miniconda/envs/pangolin/lib/python3.8/site-packages/scorpio/scripts/type_constellations.py", line 175, in variant_to_variant_record
2021-12-08T10:18:18.820-08:00 length = int(lsplit[2])
2021-12-08T10:18:18.820-08:00 ValueError: invalid literal for int() with base 10: '9 # spike'
2021-12-08T10:18:18.842-08:00 [Wed Dec 8 18:18:18 2021]
2021-12-08T10:18:18.842-08:00 Error in rule get_constellations:
2021-12-08T10:18:18.842-08:00 jobid: 7
2021-12-08T10:18:18.842-08:00 output: /tmp/tmpainen2l4/get_constellations.txt
2021-12-08T10:18:18.842-08:00 shell:
2021-12-08T10:18:18.842-08:00
2021-12-08T10:18:18.842-08:00
scorpio list --constellations /miniconda/envs/pangolin/lib/python3.8/site-packages/constellations/definitions/cP.3.json /miniconda/envs/pangolin/lib/python3.8/site-packages/constellations/definitions/cC.37.json /miniconda/envs/pangolin/lib/python3.8/site-packages/constellations/definitions/cBA.2.json /miniconda/envs/pangolin/lib/python3.8/site-packages/constellations/definitions/cB.1.617.2.json /miniconda/envs/pangolin/lib/python3.8/site-packages/constellations/definitions/cB.1.617.3.json /miniconda/envs/pangolin/lib/python3.8/site-packages/constellations/definitions/cBA.1_probable.json /miniconda/envs/pangolin/lib/python3.8/site-packages/constellations/definitions/cB.1.525.json /miniconda/envs/pangolin/lib/python3.8/site-packages/constellations/definitions/cA.23.1.json /miniconda/envs/pangolin/lib/python3.8/site-packages/constellations/definitions/cB.1.617.2+K417N.json /miniconda/envs/pangolin/lib/python3.8/site-packages/constellations/definitions/cA.23.1+E484K.json /miniconda/envs/pangolin/lib/python3.8/site-packages/constellations/definitions/cB.1.429.json /miniconda/envs/pangolin/lib/python3.8/site-packages/constellations/definitions/cB.1.1.7+E484K.json /miniconda/envs/pangolin/lib/python3.8/site-packages/constellations/definitions/cB.1.617.1.json /miniconda/envs/pangolin/lib/python3.8/site-packages/constellations/definitions/cP.1.json /miniconda/envs/pangolin/lib/python3.8/site-packages/constellations/definitions/cAY.4.json /miniconda/envs/pangolin/lib/python3.8/site-packages/constellations/definitions/cB.1.526.json /miniconda/envs/pangolin/lib/python3.8/site-packages/constellations/definitions/cB.1.1.318.json /miniconda/envs/pangolin/lib/python3.8/site-packages/constellations/definitions/cB.1.1.7.json /miniconda/envs/pangolin/lib/python3.8/site-packages/constellations/definitions/cB.1.427.json /miniconda/envs/pangolin/lib/python3.8/site-packages/constellations/definitions/cB.1.351.json /miniconda/envs/pangolin/lib/python3.8/site-packages/constellations/definitions/cP.2.json /miniconda/envs/pangolin/lib/python3.8/site-packages/constellations/definitions/cB.1.621.json /miniconda/envs/pangolin/lib/python3.8/site-packages/constellations/definitions/cAV1.json /miniconda/envs/pangolin/lib/python3.8/site-packages/constellations/definitions/cBA.1.json /miniconda/envs/pangolin/lib/python3.8/site-packages/constellations/definitions/cB.1.1.529.json /miniconda/envs/pangolin/lib/python3.8/site-packages/constellations/definitions/cAY.4.2.json --pangolin > /tmp/tmpainen2l4/get_constellations.txt
2021-12-08T10:18:18.842-08:00
2021-12-08T10:18:18.842-08:00 (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
2021-12-08T10:18:18.843-08:00 Exiting because a job execution failed. Look above for error messageIf i do not do the database update, Pangolin runs without error.
I can say that the working run judged the lineage to be AY.39 if that is at all useful.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/cov-lineages/constellations/issues/44, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACLIWO55PT5L7BILT4KOUYTUP6RRNANCNFSM5JUQIJHQ. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. Is e buidheann carthannais a th’ ann an Oilthigh Dhùn Èideann, clàraichte an Alba, àireamh clàraidh SC005336.
quaeler
commented
2 years ago I will try this later today (UTC-8) and let you know.
quaeler
commented
2 years ago Though even if that works, could you take an alternate path for your changes so that it doesn't break backwards compatibility?
Some deployment environments require a validation and verification, and perhaps security review, component(s) to their deployment which would be re-triggered by updating software versions.
quaeler
commented
2 years ago Updating scorpio did fix the issue, and i see why that would be.
I would again like to ask you to reconsider this change; this change breaks Pangolin for all users everywhere who can not upgrade scorpio due to whatever reason - and this change is ostensibly to "support in-line comments".
I understand that comments are helpful; perhaps you could include them in a separate data structure in the JSON, or in cases where you really do want to do something that totally breaks backwards compatibility, you could introduce versioning to the JSON format (e.g is scorpio finds "sites_v2", preferentially use that structure from the JSON.)
Thank you for your consideration in addressing this problem.
rmcolq
commented
2 years ago I'm glad the failure is resolved. We will postpone this (breaking) change to some point in the future, but I am reverting it for now. In general we only support classifications made with latest versions of all software/data because we have to be able to be flexible with the software to handle new situations.
quaeler
commented
2 years ago @rmcolq I appreciate the roll-back - thank you; i understand that software is a living thing — is there an announcement list which i can subscribe to in order to hear about impending 'major releases'? It would be beneficial in order to prevent future unexpected breakages, and plan for remediation.
Unfortunately i cannot provide the source FASTA file, but using a non-qc-failing FASTA generated by ARTIC with Pangolin supplied by the
staphb/pangolin:3.1.16-pangolearn-2021-11-25image, and then database-updated to:Produces the following errors when running Pangolin:
If i do not do the database update, Pangolin runs without error.
I can say that the working run judged the lineage to be
AY.39if that is at all useful.