Backend to support grouping of observations into "poses"

[mwinokan]

To support the selection of "main" observations #1261 and offer more control of observations grouping in the LHS hit navigator, Frank suggests a new DB concept: Pose.

• A pose is a group of observations belonging to the same compound and CanonSite.
• The pose can have any number of observations, but one of them is the main one
• The pose will have a display_name that is by default the main observation's shortcode, but can potentially be changed by the F/E
• At first poses should be populated by the same rules as the F/E is now doing (by compound and CanonSite), but I will mockup UI to create and modify these groupings in #1322

[mwinokan]

We need to think about whether having Observation and Pose level tags causes complexity in the database schema. Pose-level tags will only create extra visual noise in the observation panel.

[mwinokan]

What is the impact on snapshots? If the grouping changes are snapshots with the old grouping broken? Potentially if state of the frontend is inferred from the observations selected in the snapshot that may not require the grouping to be compatible.

[mwinokan]

@kaliif says this is at least a couple days work to write and test it

[Waztom]

@kaliif and @boriskovar-m2ms need to spec out the endpoint and will update next Thursday 4th April.

[boriskovar-m2ms]

@kaliif from the frontend point of view all we need in the endpoint api/poses is:

• target id
  • display name
  • list of associated site observations

I think we will need these http methods GET, POST, PATCH, DELETE.

I guess we need to preserve poses between target reuploads so probably we will need something similar to what we have for tags (display name and name that identifies the tag). What in case that we have a pose and all of its observations are removed by the reupload? Should also the pose be deleted? Because in #1322 it's suggested by me that pose without observations should be deleted.

@mwinokan currently we are only tagging observations and the compound tags (after implementing this it will be pose tags) are inferred from observation tags. To be able to tag poses would be whole another significant undertaking from frontend (and I guess for the backend too) point of view.

Frontend will take care of the issue that only compatible observations are grouped by the pose but question is if this should be also handled by the backend because api is accessible and someone can create this inconsistency (whether on purpose or by accident) which is something that frontend will not be able to handle (e.g. pose belonging to two canonical sites etc).

[mwinokan]

@kaliif's work should be finished early next week @boriskovar-m2ms estimates the F/E work will also take a week

[kaliif]

A couple of questions: 1) Boris brought up upload versions (https://github.com/m2ms/fragalysis-frontend/issues/1392#issuecomment-2033933534), can somebody (@mwinokan, @phraenquex ) comment on the expected behavior on version 2 upload? 2) Based on Max's comment above (https://github.com/m2ms/fragalysis-frontend/issues/1392#issuecomment-2007035033), implementing pose tags isn't necessary? 3) Here https://github.com/m2ms/fragalysis-frontend/issues/1322#issuecomment-2034030838 it is said that empty poses can be deleted. What about the opposite situation, should an observation always belong to a pose?

[mwinokan]

Thanks @kaliif

1. For the upload hopefully there will be no changes visible to the user. The observations should be grouped into poses just like the front end currently does (By compound and canonical site).
2. I don't see a reason for pose's to need separate tags to the observation. ( @waztom @phraenquex ?)
3. Yes, observations should always have a pose, and most observations will start off being in a pose on their own.

[kaliif]

@mwinokan, in the backend, the second upload can bring some complications. If the user changes, creates, or deletes any of the poses, on the second upload the original poses will be restored. I'm assuming that's not the expected behavior?

[mwinokan]

@kaliif thank you for pointing that out. Is it possible to have separate behaviour for observations that were also part of upload-1 and the ones added in upload-2?

Ideally whatever poses the curators make in the front end should stay even after further uploads, with any additional observations being added to their own new poses (if that compound/CanonSite was not present before) or added to the 'most appropriate' existing pose.

Could the new observations be assigned a tag 'New' as a tag (category: Other) so that curators can easily find and sort them.

[kaliif]

@mwinokan separate treatment for subsequent uploads can be done. New poses are not at risk of deletion, but automatically adding them to an existing pose may not be straightforward. Suppose a new observation coming in with a canon site-compound combination as an existing pose, but the curator has previously split the pose into 2 (if I remember, correctly that was exactly the case when the need for poses was demoed). Those poses have the same canon site-compound description and the loader cannot choose which pose the observation should be assigned to. Create a new pose with this combination?

Could the new observations be assigned a tag 'New' as a tag (category: Other) so that curators can easily find and sort them.

Yes, no problem

[mwinokan]

@kaliif thanks for explaining.

Yes, in the case where it is ambiguous which existing pose should be chosen for new observations, creating a new pose is a good solution.

[kaliif]

@boriskovar-m2ms, based on the conversation with Max above, I suspect the DELETE method in API is not necessary - observation has to belong to a pose and there's no point for empty poses. Deleting a pose with observations should then throw an error, the curator must first reassign observations, and removing the last observation from a pose should delete the pose automatically. Is this correct or do you still see a use case?

[boriskovar-m2ms]

@kaliif If backend can delete empty poses automatically then it's perfect for me and one less thing to worry about

[kaliif]

@mwinokan if the curator moves observations between poses, is it important that the new pose also has the same canon site and compound? Or can they move it to a completely arbitrary pose?

[mwinokan]

@kaliif yes. The front end should only show 'compatible' poses that an observation can be moved to. These would be poses with the same CanonSite and Compound.

[mwinokan]

@kaliif this is with @boriskovar-m2ms for testing

@phraenquex says that poses need their own separate tags. @kaliif says it should not be complicated to have tags for poses.

[mwinokan]

@kaliif has included the tagging and @boriskovar-m2ms is testing as part of #1261

[mwinokan]

@kaliif is working on the fix to assign main observations for #1261. Then @boriskovar-m2ms will verify that all backend functionality for #1261 is working to close this ticket

[mwinokan]

@kaliif please revert the changes to add tags in the 'Pose' category for each pose