Automated handling of pointcloud registration

[BDemps]

Is there any way we can have a "download data from multiple locations and open" type function implemented?

This would be most useful for synspy-register, I would think, since synspy-launcher already can open files and load synapses in "curator mode". It would be great if you could very easily download the .csv file for synapses (intensity and position), the .csv file for nuclei positions, and the images that will be registered and have the program open them once the files are downloaded. It is okay if this only works on the Linux version. It just takes quite a while to collect and open the files at the moment, and this would save a lot of headache, if it is easy? (E.g., have a little window that asks which two synapse IDs you want to view synaptic data for an which region of the brain and then it opens a synspy-register for all of that stuff...)

[karlcz]

This sounds like it would be a nice feature for "curator" mode in the launcher... @mikedarcy could you look into this and at least estimate the feasability?

I think the most confined use case might be something like:

1. Pick the first synaptic Image Region via the current task list interface
2. Have a new menu/action button for "register"
3. Pick another synaptic Image Region from a list filtered as
   • same fish
   • opposite time point
   • same anatomy as first region
   • same human classifier? or should you be able to register w/ mixtures?
4. Launcher automatically finds corresponding nucleic Image Regions for each synaptic region
   • regions of same source Image
   • nucleic segmentation mode instead of synaptic
   • same human classifier?

Ideally, the launcher could automatically find the corresponding nucleic Image Region for each timepoint. But, I think we'd have to test out the concept first.

[BDemps]

Any chance you guys have made headway on this? I'm not exactly sure how it would work, but it seems even more useful now because of a thing Carl asked for today. He wanted me to give him all of the datasets so far, and so I needed to look around for data management system links for him one at a time. If there was a more central way to get everything (Including the definition of the rotation matrices!), it would be really nice.

[carlkesselman]

I think we should explore how to use our existing tools for identifying data collections and BDBag extract to do this in a somewhat general way.

Issue of rotation matrixes is somewhat orthogonal, as they don’t exist in the catalog right now.

Carl

---

Dr. Carl Kesselman Dean’s Professor, Epstein Department of Industrial and Systems Engineering Fellow, Information Sciences Institute Viterbi School of Engineering

Professor, Preventive Medicine Keck School of Medicine

University of Southern California 4676 Admiralty Way, Suite 1001, Marina del Rey, CA 90292-6695 Phone: +1 (310) 448-9338 Email: carl@isi.edumailto:carl@isi.edu Web: http://www.isi.edu/~carl

On Aug 29, 2017, at 12:28 PM, BDemps notifications@github.com<mailto:notifications@github.com> wrote:

Any chance you guys have made headway on this? I'm not exactly sure how it would work, but it seems even more useful now because of a thing Carl asked for today. He wanted me to give him all of the datasets so far, and so I needed to look around for data management system links for him one at a time. If there was a more central way to get everything (Including the definition of the rotation matrices!), it would be really nice.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/informatics-isi-edu/synspy/issues/31#issuecomment-325772204, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ADbjXvuZwgzgkcaLbMgX5EUyq_AzbqaTks5sdGZPgaJpZM4OyS4k.

{"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/informatics-isi-edu/synspy","title":"informatics-isi-edu/synspy","subtitle":"GitHub repository","main_image_url":"https://cloud.githubusercontent.com/assets/143418/17495839/a5054eac-5d88-11e6-95fc-7290892c7bb5.png","avatar_image_url":"https://cloud.githubusercontent.com/assets/143418/15842166/7c72db34-2c0b-11e6-9aed-b52498112777.png","action":{"name":"Open in GitHub","url":"https://github.com/informatics-isi-edu/synspy"}},"updates":{"snippets":[{"icon":"PERSON","message":"@BDemps in #31: Any chance you guys have made headway on this? I'm not exactly sure how it would work, but it seems even more useful now because of a thing Carl asked for today. He wanted me to give him all of the datasets so far, and so I needed to look around for data management system links for him one at a time. If there was a more central way to get everything (Including the definition of the rotation matrices!), it would be really nice."}],"action":{"name":"View Issue","url":"https://github.com/informatics-isi-edu/synspy/issues/31#issuecomment-325772204"}}}

[karlcz]

This is not really a "collections" problem.

This is a modeling problem that the desired vector or 4-tuple of region CSVs is not well defined in the catalog but requires human intervention.

The current catalog model is "too flexible" because it wasn't designed with this one very specific protocol in mind. To fix this, we'd need a new entity type like "study" which constrains the user to choose exactly one before and one after image and exactly one nucleic and one synaptic region from each to populate the 4 slots in the vector.

This could either be a new layer on top of the current model (continued flexible data collection followed by post-hoc structuring) or by changing the model to enforce a more rigid protocol workflow to define the study structure in advance and not support unstructured image and region acquisition...

[carlkesselman]

I understand what you are saying, but even then, we still need to identify the list of studies we want and provide an interface to download in bulk.

[karlcz]

I think we plan to make some modeling changes in the catalog and also automate a bit more data processing. We should not work on changes to the launcher until after these catalog changes are sorted out...

[karlcz]

This issue was originally about making it easier to do pointcloud registration. The new release under test allows the study image pairs and cropped region pairs to be linked explicitly and automatically computes alignment and generates the registered pointcloud CSV files on the server. It embeds a revised pointcloud plot in the browser showing the aligned pair of data. It will also have basic download links for these processed CSV files. I think this covers the original issue.

Other enhancements to allow the definition or download sets of these new studies are out of scope for this issue and this release. They should be filed as separate enhancement requests which can be addressed in the future.

[karlcz]

This is in production.