Magma: one-to-many link within the same model

[coleshaw]

To support sc_seq, Magma needs to be able to construct one-to-many (collection) relationships within a single model. This would mimic the reference_patient (one-to-one, child) attribute in IPI, and would be used for constructing a "bundle" of related sc_seq records.

[graft]

Isn't this just the same as collection?

[coleshaw]

Except I think we weren't sure that would actually work within the same model? i.e. can a sc_seq record have a collection of sc_seq records?

[coleshaw]

If you've tested that and are sure it works, then we can close this issue and add that attribute to the sc_seq model in coprojects_template, since I think the DS were waiting on engineering to confirm that feature is available.

[coleshaw]

I don't recall the exact (biology) terms that folks agreed on...but it was something like "associated assays", i.e. the primary entry point would be a GEX sc_seq record, that would then have as a collection the associated CITE / ATAC seq records? Sorry, bad memory for the final, agreed upon attribute names :disappointed:

Am I mis-remembering that discussion? Probably from several modelling meetings ago.

[graft]

I am not sure this works, but the attribute behavior seems consistent with collection here. There are examples of collection collecting links e.g. rna_seq_plate though not in the same model.

[dtm2451]

Can confirm @coleshaw! This is what I get if I try to apply a csv with the attributes discussed in modeling meetings:

dan@pop-os:~/GitHub/monoetna$ etna administrate models apply_template coprojects_template --environment development --file ~/Downloads/coprojects_template_models_project_tree_TEST.csv 
Traceback (most recent call last):
    9: from /home/dan/.rbenv/versions/2.7.6/bin/etna:23:in `<main>'
    8: from /home/dan/.rbenv/versions/2.7.6/bin/etna:23:in `load'
    7: from /home/dan/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/etna-0.1.45/bin/etna:29:in `<top (required)>'
    6: from /home/dan/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/etna-0.1.45/lib/etna/application.rb:190:in `run_command'
    5: from /home/dan/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/etna-0.1.45/lib/commands.rb:318:in `execute'
    4: from /home/dan/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/etna-0.1.45/lib/commands.rb:376:in `load_models_from_csv'
    3: from /home/dan/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/etna-0.1.45/lib/commands.rb:376:in `open'
    2: from /home/dan/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/etna-0.1.45/lib/commands.rb:377:in `block in load_models_from_csv'
    1: from /home/dan/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/etna-0.1.45/lib/etna/clients/magma/workflows/add_project_models_workflow.rb:42:in `prepare_changeset_from_csv'
/home/dan/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/etna-0.1.45/lib/etna/clients/magma/workflows/add_project_models_workflow.rb:47:in `validate_changeset': undefined method `models' for #<Array:0x0000565400620db8> (NoMethodError)

The csv: coprojects_template_models_project_tree_TEST.csv

The attributes:

• parent_modality of 'type' link, 'link_model_name' sc_seq, and 'link_attribute_name' (<-- a new column specifically added for this purpose, but we can adjust terminology/etc. here as needed) of associated_modalities
• associated_modalities of 'type' collection, and 'link_model_name' sc_seq

[coleshaw]

Yes, I agree that it is consistent with collection, but I think it was up to us to verify if that behavior worked or didn't work within the same model ... i.e. would it break Magma if we added it to the model, right now? :shrug:

[dtm2451]

*confirmed specifically that the etna path fails to interpret what was wanted to happen, not whether or not magma could handle the link or not. But hopefully my message gives more full context of the desired attributes.

[coleshaw]

Thanks @dtm2451 , I think that concrete example helps!

So I think there are two related tasks for this issue, then:

• [ ] Verify if Magma can handle link and collection within the same model. If not, build the feature.
• [ ] Provide the capability in the etna gem to add such attributes.

[coleshaw]

Tasks:

• [x] Add link_attribute_name. Backfill data. (Migration sketched out already)
• [x] Prevent two link actions (only allow collection, child on the other end link). Update the validations in add_link.rb
• [x] Write link_attribute_name and link_model_name in the add_link action.
• [x] Write link_attribute_name and link_model_name in add_model action.
• [ ] Update queries to use link_attribute_name to construct the query. Inspect uses of .foreign_id to verify what might be affected and deprecate to use link_attribute_name + link_model from the parent (record predicate), column_name from the child / collection. For ::has and ::lacks, change foreign_id to column_name? Same for attribute_join.
• [ ] Update the etna gem to allow / validate above behavior.