part display

[dustymc]

Given:

• some part
  • some subsample
    • some subsample attribute
    • some subsample attribute
    • some subsample attribute
    • some subsample attribute
      • some subsample subsample
    • some subsample subsample attribute
    • some subsample subsample attribute
    • some subsample subsample attribute
    • some subsample subsample attribute
  • another subsample
    • another subsample attribute

How do we display that alongside all the other part stuff that users have requested in specimendetail?

Possibilities:

• be more explicit in "subsampledness" - reintroduce the "subsample" prefix (but what to do of subsamples of subsamples?)
• popups: display bare/minimal information in the default view, click a button for details (but need to keep subsample-of-subsample loan data visible??)

To-Do: Make some mockups to play with

@campmlc @ccicero

[dustymc]

I dropped the trigger preventing subsampling subsamples, rebuilt the forms to deal with infinite recursion, added a "subsample" shortcut from parts, and added some parts and attributes to http://arctos-test.tacc.utexas.edu/guid/CHAS:Bird:17187.

The forms are terrifying. I'm up for better ideas, but it seems we need this functionality for GGBN so I don't see how we can simplify too much.

Help?!?

SpecimenDetail parts grid:

Edit Parts:

[campmlc]

I used the part edit tool to create a subsample of http://arctos-test.tacc.utexas.edu/guid/MSB:Mamm:210933 Then I tried to add this to a loan, but the subsample did not show up on the add specimens to loan results. I used the loan form to add another subsample to the loan. Both subsamples show up on the specimen record. But when I went back to the loan to add additional parts, I did not see either subsample.

We need a way to be able to see the subsamples from the loan form - and have them clearly distinguished from the parent parts, in order subsample the subsamples for further loans. However, this runs the risk of people choosing the wrong part for a loan, and it is difficult to make this correction - no way to do so from the add specimens to loan form. Any way to add "subsample" to the dispostion in the loan and specimen record part form, e.g. "liver (frozen) (subsample, unchecked)"?

On Mon, Dec 18, 2017 at 6:38 PM, dustymc notifications@github.com wrote:

I dropped the trigger preventing subsampling subsamples, rebuilt the forms to deal with infinite recursion, added a "subsample" shortcut from parts, and added some parts and attributes to http://arctos-test.tacc. utexas.edu/guid/CHAS:Bird:17187.

The forms are terrifying. I'm up for better ideas, but it seems we need this functionality for GGBN so I don't see how we can simplify too much.

Help?!?

SpecimenDetail parts grid:

[image: chas-bird-17187-dendroica pensylvanica 2017-12-18 17-15-27] https://user-images.githubusercontent.com/5720791/34135792-97fcba8a-e417-11e7-9dcb-1d4a873bc0f0.png

Edit Parts:

[image: chas-bird-17187-dendroica pensylvanica 2017-12-18 17-15-54] https://user-images.githubusercontent.com/5720791/34135799-a0d929ea-e417-11e7-8272-ccd693274cf2.png

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ArctosDB/arctos/issues/1367#issuecomment-352612883, or mute the thread https://github.com/notifications/unsubscribe-auth/AOH0hBvqbfPhBAD81jsu7dXfKi3W8PUWks5tBxODgaJpZM4RGAwP .

[dustymc]

If you can't do something with subsamples that you can do with anything else, it's because someone has explicitly asked to hide subsamples there. I can un-do that, but I'd rather not end up back in whatever situation prompted the changes.

"subsample" to the dispostion

I will strongly resist this. Subsamples are structurally defined. Denormalizing that data and mucking up another concept while we're at it is not good data design.

Subsamples are parts that can be traced back to another part. That's it. The model isn't limited to or designed for tissue samples, although the UI has mostly locked us into that up to this point. Everything that's not "whole organism" is only not a subsample because our data aren't complete. If we're opening the model and UI up so that subsamples aren't constrained to "things created for loans" - and that process is underway - then I don't think I understand any need for "clearly distinguished from the parent parts."

If you do need that distinction, the obvious way is to display part data hierarchically.

For existing subsamples, you may want to avoid them because there's a small "remaining volume," which probably means displaying part attributes with parts.

I would like more input on this before we start doing anything drastic outside of the two forms I'm modifying now. @KyndallH @amgunderson ??

[campmlc]

One reason we may have needed to hide subsamples is because they were impossible to distinguish from parent tissues and ended up being inadvertently added to loans. We do need to avoid that at all costs. We need a way to visually and textually distinguish parent-child relationships in the loan form, to allow users to choose to subsample existing subsamples. This is an interface/display issue, not a data structure issue. The visual display should not have to completely overlap and reflect the underlying data structure - that is one of our overall problems with Arctos usability.

Here is a draft of a possible scenario that we should try to resolve.

Liver (frozen) (in collection) Liver (frozen) (subsample_x) (on loan) (loan 1) DNA extraction (frozen) (from subsample_x) (in collection) (associated metatdata on type of extraction, DNA quantity, Genbank sequences etc) (return of loan from loan 1) DNA extraction subsample (frozen) (subsample of extraction of subsample_x ) (on loan) (loan 2) DNA extraction subsample (frozen) (subsample of extraction of subsample x) (on loan) (loan 3) Liver (frozen) (subsample_y) (on loan) (loan 4) DNA extraction (frozen) (from subsample_y) (in collection) (associated metadata for different type of extraction etc) (returned of loan from loan 4) DNA extraction subsample (frozen) (subsample of extraction of subsample y) (on loan) (loan 5)

To address Teresa's question, how would the single JSON field in the bulkloader work for part attributes? I think this merits more discussion, but I don't understand it yet.

On Tue, Dec 19, 2017 at 8:53 AM, dustymc notifications@github.com wrote:

If you can't do something with subsamples that you can do with anything else, it's because someone has explicitly asked to hide subsamples there. I can un-do that, but I'd rather not end up back in whatever situation prompted the changes.

"subsample" to the dispostion

I will strongly resist this. Subsamples are structurally defined. Denormalizing that data and mucking up another concept while we're at it is not good data design.

Subsamples are parts that can be traced back to another part. That's it. The model isn't limited to or designed for tissue samples, although the UI has mostly locked us into that up to this point. Everything that's not "whole organism" is only not a subsample because our data aren't complete. If we're opening the model and UI up so that subsamples aren't constrained to "things created for loans" - and that process is underway - then I don't think I understand any need for "clearly distinguished from the parent parts."

If you do need that distinction, the obvious way is to display part data hierarchically.

For existing subsamples, you may want to avoid them because there's a small "remaining volume," which probably means displaying part attributes with parts.

I would like more input on this before we start doing anything drastic outside of the two forms I'm modifying now. @KyndallH https://github.com/kyndallh @amgunderson https://github.com/amgunderson ??

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ArctosDB/arctos/issues/1367#issuecomment-352799976, or mute the thread https://github.com/notifications/unsubscribe-auth/AOH0hMq7Zv9f-SvbO6vuzyruLTHL4BEaks5tB9wFgaJpZM4RGAwP .

[dustymc]

inadvertently added to loans

You've got a tube of important stuff, you put half of it in a new tube so you can spread the risk across two freezers. I don't think "don't loan subsamples" can be the full use case. (And most current subsamples were created for loans.)

should not have to completely overlap and reflect the underlying data structure

I don't understand this comment. The data structure is "parts can come from parts." You're asking to see parts which have come from parts. ????

return of loan from loan 1

Are you asking for shipments or disposition or something else?

associated metatdata

part attributes, correct?

Possibly a useful approach: http://arctos-test.tacc.utexas.edu/guid/CHAS:Bird:17187, click "Parts" - what of that do you NOT need to see when selecting a part for a loan? Surely we'll never see actual data that messy, so if we can deal with that we can probably deal with anything....

JSON

JSON is a transport mechanism with a flexible structure.

some part

can be made into JSON

some part
- some attribute
- 600 more attributes
600 more parts
- 600 more attributes for each of the 600 parts

can also be made into JSON, and will fit into the same "container" (text file/field/string) as "some part."

From what I think I know right now, I'd suggest we concentrate on https://github.com/ArctosDB/arctos/issues/1294. JSON and normalization are about the same things in this context. The experiment in partdetail as JSON makes me think we'd need viewers. The data entry screen is already crowded - I'm probably not going to add 600 "fields" to it, even though you can stuff 600 "fields" into a JSON object - same as the separate bulkloaders. A JSON approach would be a specialized tool - it would let you enter new specimens with complex data. Better integrating the parts (and attributes, etc.) bulkloader(s) into the specimen bulkloader would let you enter new specimens with complex data, but also load extraction metadata that you get back from loans to existing specimens and etc. - the bulkloaders are much more powerful tools.

[campmlc]

My comment on data structure is just that we can customize the display to make it easier for users without affecting the underlying structure of the data. So, they might see "liver (frozen) (subsample, on loan)" where the word "subsample" is not actually part of the disposition table. Just a means to try to simplify how users see and interact with the data. This is something we can discuss in the context of a grant.

I can't see the CHAS example in my test login.

I still don't understand #1294 - I still don't know what the three clicks are to deal with " UUID: ac15bafc-e22a-45d5-868a-56450b947c07" and I'd hate to have to add more of these. Can you explain?

On Tue, Dec 19, 2017 at 3:24 PM, dustymc notifications@github.com wrote:

inadvertently added to loans

You've got a tube of important stuff, you put half of it in a new tube so you can spread the risk across two freezers. I don't think "don't loan subsamples" can be the full use case. (And most current subsamples were created for loans.)

should not have to completely overlap and reflect the underlying data structure

I don't understand this comment. The data structure is "parts can come from parts." You're asking to see parts which have come from parts. ????

return of loan from loan 1

Are you asking for shipments or disposition or something else?

associated metatdata

part attributes, correct?

Possibly a useful approach: http://arctos-test.tacc. utexas.edu/guid/CHAS:Bird:17187, click "Parts" - what of that do you NOT need to see when selecting a part for a loan? Surely we'll never see actual data that messy, so if we can deal with that we can probably deal with anything....

JSON

JSON is a transport mechanism with a flexible structure.

some part

can be made into JSON

some part

• some attribute
• 600 more attributes 600 more parts
• 600 more attributes for each of the 600 parts

can also be made into JSON, and will fit into the same "container" (text file/field/string) as "some part."

From what I think I know right now, I'd suggest we concentrate on #1294 https://github.com/ArctosDB/arctos/issues/1294. JSON and normalization are about the same things in this context. The experiment in partdetail as JSON makes me think we'd need viewers. The data entry screen is already crowded - I'm probably not going to add 600 "fields" to it, even though you can stuff 600 "fields" into a JSON object - same as the separate bulkloaders. A JSON approach would be a specialized tool - it would let you enter new specimens with complex data. Better integrating the parts (and attributes, etc.) bulkloader(s) into the specimen bulkloader would let you enter new specimens with complex data, but also load extraction metadata that you get back from loans to existing specimens and etc. - the bulkloaders are much more powerful tools.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ArctosDB/arctos/issues/1367#issuecomment-352905156, or mute the thread https://github.com/notifications/unsubscribe-auth/AOH0hK6dMtRMa_Ya2wwWMQIJ9UyoKNarks5tCDergaJpZM4RGAwP .

[dustymc]

customize the display

Gotcha, and absolutely. Ya'll just need to work out what you want to see and where.

CHAS example

campmlc should be able to get there now - if that's not what you're using let me know and I'll adjust.

UUID

http://handbook.arctosdb.org/how_to/How-to-load-Data-Entry-linked-Attributes.html exists but it's just some screenshots. Parts and such are similar. I'll try to work up something a little more instructive.

[campmlc]

This is interesting and potentially workable - especially if we don't show all the messy stuff but just add an "expand" tab to each part. http://arctos-test.tacc.utexas.edu/guid/CHAS:Bird:17187

How would JSON be a part of this? Still trying to sort this out . . .

On Tue, Dec 19, 2017 at 5:16 PM, dustymc notifications@github.com wrote:

customize the display

Gotcha, and absolutely. Ya'll just need to work out what you want to see and where.

CHAS example

campmlc should be able to get there now - if that's not what you're using let me know and I'll adjust.

UUID

http://handbook.arctosdb.org/how_to/How-to-load-Data-Entry- linked-Attributes.html exists but it's just some screenshots. Parts and such are similar. I'll try to work up something a little more instructive.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ArctosDB/arctos/issues/1367#issuecomment-352926231, or mute the thread https://github.com/notifications/unsubscribe-auth/AOH0hKsWYsnfe8BAv6W1d_8GNsbyNy4aks5tCFHFgaJpZM4RGAwP .

[dustymc]

don't show all the messy stuff but just add an "expand" tab to each part

That's probably it's own Issue. I sorta suspect people want those data to be obvious, but really have no idea. Technically that would not be a problem.

JSON

JSON isn't used for that - I have more powerful options once the data are in Arctos proper.

Each "row" of that parts table consists of one part, potentially some relationships to other parts, zero or more attributes, zero or more loans, maybe some containers, etc. There's no set structure or scope to the data, and I need a flexible mechanism so I can add new things as ya'll find a need for them (loans were recently added, for example). There's also some multi-level ordering - some parts have "children" and some of them have children and siblings and etc. It's complex data - you couldn't stuff it into a row in an Excel table without losing information. (You could stuff those 18 parts into ~100 rows, as long as you don't mind repeating a bunch of stuff - that's how I get it out of Oracle, just because that's what performs best in that particular case.) I generally write code that writes code, so instead of directly building that table I send "part chunks" off to a function which returns table rows, and to do that I need to transport multidimensional data. JSON would be a suitable tool for that transport. If we had some sort of reason to pass those data in or out of Arctos (eg, from the bulkloader or to some LIMS app or etc.) or to move them around after your browser has loaded the initial page, JSON would likely be the tool of choice. (XML and some other stuff would work as well.) JSON is just a transport mechanism - think of it as super-fancy CSV.

[KyndallH]

Alright, here is my input as I attempted to sort through all of this.

I have no problem with the way subsamples are currently displayed.

Mariel brought up the problem of inadvertently adding them to loans. There is not an issue for me since my students and I only add things to loans using "add parts by barcodes". Subsamples don't have a barcode. We also never receive subsamples back. We are starting to do insect DNA extractions but that usually isn't tied to a part. So we may be unique in that everything is barcoded and we don't take subsamples/loans of tissues back.

I also agree that I don't want more stuff added to the specimen screen but I hate having to click 15 times to find the data I need (2-3 doable). Though I'm happy with the status quo when it comes to subsamples. I do like expanding and hiding stuff features.

So to tell them apart, I simply look at the "on loan" in the disposition. I, however, am horrible about closing out loans (hard when you don't ask for them back and you wait years for the publication). So what would happen to the disposition when I close a loan? Would I be able to select a disposition for all those parts or would it continue to say "on loan"? Not sure what disposition I would pick after I close out a loan. Hmm.

[dustymc]

what would happen to the disposition when I close a loan

Nothing, although there's a "change all disposition" widget (review loan items).

That should be linked to barcodes, somehow. Scanning a barcode auto-updates disposition to "in collection" or things with barcodes (=strong object tracking) don't display disposition (=weak object tracking, sorta) at all, or ??????????

I think that's all irrelevant for your subsamples - they should probably be some flavor of "we don't have them."

[dustymc]

Terrifying forms are in prod, although they're not really scary until someone adds terrifying data. We can re-visit this when/if it becomes a problem - closing for now.