Release for v2.5.0

[johnlees]

Todo:

• [x] Update sketchlib to v2. Use kwargs in all calls.
• [x] Increase recursion depth.
• [x] Replace dendropy #203
• [x] New logo #202
• [x] Fix the cytoscape output #196
• [x] Fix tree = None in cytoscape visualisations. combined_seq not set here
• [x] Documentation update #191, #185, #189
• [x] Reference names #190
• [x] Tweak manual start #174
• [x] Remove vestigal poppunk web code
• [x] Replace t-SNE with mandrake
• [x] Change this line to

  if type(top_value) == str:
      sketch_dict = json.loads(top_value)
  else:
      sketch_dict = top_value

• [x] Change to new microreact api, add --api-key option to visualisation
• [x] Output .microreact file
• [x] Update t-SNE -> mandrake images in docs (after microreact updated)
• [x] For cytoscape visualisation add a loop which writes a graphml for each component
• [x] Update assign so ~high stress/betweenness nodes can be removed (or many zeros, or members of many clusters)~ anything which links >2 connected components
  • [x] Fix QC distance prune https://github.com/bacpop/PopPUNK/issues/194
  • [x] Possible bug with assignment (4,13,19,53 or 139 in 2.4.0; 4_19_53_139 or 4_13_19 or 4_13_53 in 2.4.6)
  • [x] Add a 'one-by-one' option
• [x] Iterative PopPUNK
  • [x] Distances to treeswift part
  • [x] 'Cut tree' (see fig 1D from Bin)
  • [x] Check for boundaries lower than zero
  • [x] Expand the search range (as an option)
• [x] Run some example data through
  • [x] Lots of distances calculated on create db (this was from plot-fit, now capturing stderr via a complicated method!)
  • [x] Network summaries for lineage fit?
• [x] ~Stalls on non-pneumo with GPSCs (e.g. 10050_2#1)~ couldn't replicate
• [x] Potential issue with sketchlib version string #213 Fix this line https://github.com/bacpop/PopPUNK/blob/master/PopPUNK/sketchlib.py#L31
• [x] Potential issue with cytoscape & graph tool #210
• [x] Check Ns in reads #207
• [x] Check the indiv refine test output, stderr looks wrong
• [x] Merge in #209
• [x] Test search range for boundary iterate is fixed & correct
• [x] Check cutoff clusters actually contains some output
• [x] Add documentation for new QC mode and options

[muppi1993]

• [x] Change this line to

  if type(top_value) == str:
      sketch_dict = json.loads(top_value)
  else:
      sketch_dict = top_value

[johnlees]

@nickjcroucher Would you be able to confirm the MST -> phylogeny is still working as expected here?

[johnlees]

Example for the merged clusters

Reference database: /media/mirrored-hdd/shorsfield/jobs/poppunk-models/Salmonella/sal_sketch40k Model fit: /media/mirrored-hdd/shorsfield/jobs/poppunk-models/dbscan_indiv_refine_both query file: /media/mirrored-hdd/shorsfield/jobs/poppunk-models/Salmonella_query_popPUNK_ids_split.txt.acaaae command: poppunk_assign --db sal_sketch40k --query Salmonella_query_popPUNK_ids_split.txt.acaaae --model-dir dbscan_indiv_refine_both --core --threads 20 --output query-assignments_acaaae --qc-filter prune

[nickjcroucher]

@nickjcroucher Would you be able to confirm the MST -> phylogeny is still working as expected here?

Apologies for the delay - for a bacterial example, ape says they are identical, consistent with visual inspection: v2.4.6:

v2.5.0:

[johnlees]

Great, thanks Nick!

[johnlees]

wrt cytoscape issues

When trying to just create cytoscape output for a subset of the data, instead of not subsetting at all like happened on the command line, this time I get "IndexError: list index out of range" for this isolate_names[n]. I checked what these objects there store, and actually isolate_names and viz_subset are holding the same list of samples (the subset I am looking for). If I understand the code correctly, isolate_names should rather have all isolate names rather that just the subset, right? Since isolate_names is coming from combined_seq again, is the cause of this problem that I run it with rank_fit = None/tree = "nj"?

[johnlees]

Just need to have a look at that stall, which may be a sketchlib issue

[johnlees]

🤯

[nickjcroucher]

Fastastic!