convert demographic models to Demes YAML files

[grahamgower]

https://popsim-consortium.github.io/demes-spec-docs/main/introduction.html

[aabiddanda]

Hi @grahamgower, I was wondering about the status of this issue currently and whether it might be possible to add the catalog using even currently available modules? For example, I've found it of great value to generate the following catalog graphic using:

import msprime 
import demes
import demesdraw
import stdpopsim
import matplotlib
import matplotlib.pyplot as plt

print(f"stdpopsim version: {stdpopsim.__version__}")
print(f"msprime version: {msprime.__version__}")
print(f"demes version:  {demes.__version__}")
print(f"demesdraw version: {demesdraw.__version__}")
print(f"matplotlib version: {matplotlib.__version__}")

species = stdpopsim.get_species("HomSap")
n_models = len(species.demographic_models)
fig, ax = plt.subplots(1, n_models, figsize=(n_models*4, 4), sharey=True)
for i,model in enumerate(species.demographic_models):
    # Convert to an msprime v1.0 demography object ... 
    demography = msprime.Demography.from_old_style(model.population_configurations, 
                                           migration_matrix=model.migration_matrix, 
                                           demographic_events=model.demographic_events)
    graph = demography.to_demes()
    demesdraw.tubes(graph, ax=ax[i], log_time=True);
    ax[i].set_title(model.id, fontsize=14)
    ax[i].set_xticklabels([p.id for p in model.populations], 
                          rotation=45, ha='right', rotation_mode="anchor", fontsize=14)
    if i > 0:
        ax[i].set_ylabel('')

plt.tight_layout()
plt.savefig('demography_gallery.png', dpi=300)

where the versions are:

stdpopsim version: 0.1.2
msprime version: 1.0.2
demes version:  0.1.2
demesdraw version: 0.1.4
matplotlib version: 3.4.3

and generate the following:

I presume that the conversion of msprime.Demography.from_old_style() at some point might become obsolete for stdpopsim, but for the current models it is quite useful. Let me know if adding something like this is of interest for the documentation of the various models here.

[grahamgower]

Hi @aabiddanda! As you seem to have figured out, the demes conversion can be automated and is actually quite straightforward. What we're waiting on, is to finalise a couple of details about the demes specification/format itself, and propagate these changes into the demes software ecosystem. There's one particular change we've made to the "pulses" field that is backwards-incompatible. In addition, we have decided to include a "info" field in the demes spec for including an arbitrary dictionary of additional information, which stdposim will then use for things like demography-specific mutation rates. But we haven't implemented this "info" field yet.

If you have an idea of how to automate the inclusion of demographic model plots in the catalog documentation, then a pull request would be happily accepted! This of course would need some small changes after we convert to the demes format for the catalog, but your code above can easily be used in the interim.

[aabiddanda]

Cool sounds like a good idea - I'll try to put a PR together as an intermediary that might have to be edited later on. Thanks!

[apragsdale]

Reviving this issue: I think demes is pretty much stable and it would be safe to start converting the catalog to demes format. This will take some effort, but I think the process itself is straightforward.

@grahamgower what do you think? any ideas for order of operations here?

[grahamgower]

Here's a rough process that might work.

1. Convert existing models. Hopefully we've done all the hard work elsewhere, making this trivial. Load each stdpopsim model in turn, convert the msprime demography to a demes object, write the demes object to a file.
2. Organise the demes yaml files in a sensible way.
3. Change the model registration to instead query which yaml files are present for a given species. Or maybe we don't even need model registration anymore?
4. Change the model loading code to load the relevant yaml file.
5. Delete the code implementing models using the msprime API.
6. Update the tests that break.
7. Update the docs on how to write a demographic model.

[apragsdale]

Great! I think we’ve done most the heavy lifting for conversion already, so it’ll just be a matter of reorganization and testing within stdpopsim. My opinion is that the catalog should exist as data/yaml files in a separate repository with minimal dependencies that stdpopsim (or any other software) can import. You’re right that docs are going to need a major overhaul after this transition.

On Apr 21, 2022, at 12:58 PM, Graham Gower @.***> wrote:



Here's a rough process that might work.

1. Convert existing models. Hopefully we've done all the hard work elsewhere, making this trivial. Load each stdpopsim model in turn, convert the msprime demography to a demes object, write the demes object to a file.
2. Organise the demes yaml files in a sensible way.
3. Change the model registration to instead query which yaml files are present for a given species. Or maybe we don't even need model registration anymore?
4. Change the model loading code to load the relevant yaml file.
5. Delete the code implementing models using the msprime API.
6. Update the tests that break.
7. Update the docs on how to write a demographic model.

— Reply to this email directly, view it on GitHubhttps://github.com/popsim-consortium/stdpopsim/issues/962#issuecomment-1105543139, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIBL46X6XISL2LBOPC3SZ33VGGJNTANCNFSM463VRTWA. You are receiving this because you commented.Message ID: @.***>