Recording Preservation History in Arctos

[KyndallH]

I often have samples that have had several preparation/storage methods throughout their life. I know I'm not alone in this. I (we) would like a way to track this beyond some notes in the part remark field.

For example, they are collected in ethanol or DMSO before being put in the freezer or they have been in a chest freezer for 20 years before getting into liquid nitrogen. Our current method of tracking condition (a simple text box) is not enough to incorporate all the data I have on a sample.

I propose creating a Preservation History table.

Under the specimen record, the Part table would be something like:

I suggest getting rid of preservation method grouped with part name (or keep it if it is that big of deal to people).

If you were to click on the details it would open the preservation history details similar to this:

Date: Date action happened [need some way to start/retroactive to date of collection] Barcode: Barcode affixed to that sample or blank if no barcode User: Person that is entering the information [pulls from login info] Preservation methods: DMSO, frozen, ethanol, isopropyl, RNAlater, dehydrated, TE buffer, DNA hydration solution, unknown [code table] Concentration: 70%, 80%, 90% 92.8% [text field] Temperature: RT, -80C, -70C, -20C, -196 [text field] Storage Buffer: [same as preservation method code table] [This indicts if it is still in the storage buffer or if has been decanted. For example, a subsample from a whole organism in Ethanol would no longer in EtOH when it goes in the freezer. A fin clip in ethanol will stay in Ethanol while in the freezer.] Container Type: [indicate the procession from whirl-paks to cryovials to microwell plates – shows how it has progressed/changed on its archival journey]

A few other examples of how the table would/could look with data in it:

It would be AWESOME if this table could pull in information elsewhere. For example, I subsample a tissue for a loan and it shows on this history chart that is was subsampled (loan data in Remarks). Also, information from environment report such as testing the ethanol in a jar of pickled specimens.

Another big must have would be a way to bulk/batch enter this data. For example, I’m moving the herbarium silica samples into the freezers. Bulk enter/edit their preservation history.

Comments, suggests, interest, doability?

[dustymc]

There are two possible approaches here:

1) Do something weird and text-based on a part-by-part basis. We're sorta doing that in part name now, and it sounds like people are adding more weirdness in remarks and such. Using part attributes instead of remarks would help find things where SOMETHING is weird - it's a bit more standardization. Developing better part attribute control/vocabulary might even facilitate finding WHAT is weird. That's still a part-by-part solution - you have to update the attribute (or remark or whatever) for every single one of the 100K tubes from "frozen" to "sorta stinky, but frozen again" after the freezer dies.

2) Use containers. Everything above can be represented in the current container model. It's a spleen, we hucked it into a container which has an "environment" of RNA later, then we hucked that container into the freezer (also a container). Freezer dies, all 800 parts in the RNA later container (and all other containers in the freezer) inherit that history, etc. Or it was in the range (which is a container and could have a humidity logger - we can do better than "dry") then it got plunked into a container of silica then that went into the freezer (or it came out of silica and went into the freezer or WHATEVER).

Both are capable of maintaining history.

History is unavoidable with containers - only SYS has delete on that table, and they will not be using it. (And it's easy to create fine-grained container environment history - set your data logger up to talk to Arctos and save freezer temp every hour if you want.)

Part attributes have a determiner and date, so...

date	user	attribute	value
2017-01-01	someUser	someAttribute	someValue
2016-01-01	someUser	someAttribute	someValue
2015-01-01	someUser	someAttribute	someValue
2014-01-01	someUser	someAttribute	someValue
2013-01-01	someUser	someAttribute	someValue

fits.

There is a part bulkloader which will handle a few part attributes. It could be altered to do WHATEVER. (But you probably don't need it if you're not using barcodes, and without barcodes it can't tell which of ABC:ZYZ:123's 17 "liver" samples you're trying to update - I'm not sure if we have the data to support some tools.)

A container environment bulkloader is possible, but I'm not sure it's needed. Update the "humidity" environment for the range and you've just "updated" all bajillion specimens in the range-container.

See also #1020

[Jegelewicz]

This was the effectively part of the discussion at today's Darwin Core Hour. I encourage you all to watch the recording. It should eventually be posted here: https://vimeo.com/album/4407185

As for containers, I get the impression they shouldn't be used without barcodes and I can't go there right now. Am I mistaken?

[KyndallH]

I want it searchable and some standardization to it with room for weird hence a mix of code tables and text fields.

I want it both part by part basis but ALSO bulk.

I want to be able to pull up a detailed list with dates when something happened to this one part.

I want to be able to access and edit it from the specimen record page. And enter it through the specimen record entering page.

Yeah, similar to history detail under container but tied to the specimen record and more fields.

I know, I want a lot. Come on, don't you just type a bunch of 1 and 0s and magic happens? :)

[dustymc]

searchable

That's an interface problem - we have the data. (="easy" to fix, given the resources to work on it)

part by part

Parts are generally in a "part container" - NUNC etc. (And parts ARE containers, but I don't think we want to mess with that one directly.)

detailed list with dates

That's in the model

one part

Interface problem - from the one part, you don't have to know it also happened to a million more.

access and edit it from the specimen record page

I don't see why not, as long as you know what you're doing. (Eg, updating the nunc or maybe even the freezer box is OK, but you probably don't want to update the range because you found a rotten liver in the corner.)

And enter it through the specimen record entering page.

You lost me there...

similar to history detail under container but tied to the specimen record

Container history IS tied to specimen records. The interfaces may not much reflect that, but the data do.

more fields

UAM@ARCTOS> desc container_environment Name Null? Type

---

CONTAINER_ENVIRONMENT_ID NOT NULL NUMBER CONTAINER_ID NOT NULL NUMBER CHECK_DATE NOT NULL DATE CHECKED_BY_AGENT_ID NOT NULL NUMBER PARAMETER_TYPE NOT NULL VARCHAR2(60) PARAMETER_VALUE NOT NULL NUMBER REMARK VARCHAR2(4000)

hard to imagine something you can do to a dead-rat-bit that doesn't fit in there.

bunch of 1 and 0s and magic

01011001 01110101 01110000 00101100 00100000 01110100 01101000 01100001 01110100 00100111 01110011 00100000 01101000 01101111 01110111 00100000 01101001 01110100 00100000 01110111 01101111 01110010 01101011 01110011 00101110 00101110 00101110 00101110

[dustymc]

containers...shouldn't be used without barcodes

http://handbook.arctosdb.org/documentation/container.html#object-tracking-without-barcodes

Any sort of machine-readable labels work - barcodes, RFID tags, punchcards....

I can't stop you from using the object tracking capabilities of Arctos through your keyboard, but I strongly suspect it would just be a lot of work for not much benefit. Arctos object tracking is based around uniquely-identifiable well-defined objects (part, box, freezer) and I don't think that's possible without involving machines. Part attributes are a sufficient replacement, and are linked to the thing that people are good at dealing with.

[Jegelewicz]

How can part attributes help me with the ten snakes in a jar problem? I'd like to print a label for them from Arctos (and have Arctos know that they are in the same jar), but I can't buy any equipment right now to give the jar a barcode. :-(

[dustymc]

ten snakes in a jar problem

Give them all the same value in part attribute "location."

print a label

I can help with the SQL, shouldn't be much of a problem.

Arctos know that they are in the same jar

There's where human-based object tracking gets dicey. For all Arctos knows, you have 500 snakes labeled "snake 1" and 50 jars labeled "jar 1" (or anything else - people are very creative!). All Arctos can "know" is that some strings are associated with some other strings (if you happen to type them consistently and correctly). If you really want to know what's where, you need unique identifiers, a device that can't read them incorrectly, and some useful protocols.

buy any equipment

It's not ideal, but you can print a bunch of barcodes on a sheet of paper and ebay claims to have 842 USB 1-D scanners under $20.

[campmlc]

So if we are using containers to track parts, parts that share a container could be listed independently with a shared barcode, correct? So we could eliminate the part: "heart, lung, kidney, spleen (frozen)", and instead have the option to choose each tissue separately from a dropdown and add the same barcode?

That way we wouldn't have to create every possible combination of those parts separately in Arctos to deal with missing lung, for example. That would be helpful, but might require tweaking data entry and the bulkloader since we currently run into problems with more than 10 parts - or is that better now? I agree with Kyndall's approach.

[dustymc]

parts that share a container

Yes, that's possible now. And yes, I think getting rid of at least some of those combo-parts would make everyone happy.

problems with more than 10 parts

The part bulkloader is accessible from the specimen bulkloader, and that pathway could probably be polished up somehow. Or maybe we could experiment with just adding more parts to the specimen bulkloader - still seems somehow wrong, but I can't identify anything specific that it might break.

[DerekSikes]

I'm all for Kyndall's proposal!

[campmlc]

Yes, me too!!

Especially as it would enable: "getting rid of preservation method grouped with part name (or keep it if it is that big of deal to people"

I also support: "If this table could pull in information elsewhere. For example, I subsample a tissue for a loan and it shows on this history chart that is was subsampled (loan data in Remarks). Also, information from environment report such as testing the ethanol in a jar of pickled specimens."

On Wed, Jul 19, 2017 at 11:16 AM, DerekSikes notifications@github.com wrote:

I'm all for Kyndall's proposal!

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/ArctosDB/arctos/issues/1119#issuecomment-316455702, or mute the thread https://github.com/notifications/unsubscribe-auth/AOH0hMTZTd_COTouzvJvlsZIlaHrYAK_ks5sPjnQgaJpZM4NOvI- .

[dustymc]

I think there's some sort of communication problem.

All of the DATA in Kyndall's original proposal can exist now in containers. As far as I can tell nobody is using that capacity, so I haven't done much with forms. The forms will be complex and take some work, so I think it would greatly benefit us all if we were looking at actual data while the forms are being developed.

Alternative, all of the requested capacity is available through specimen part attributes, and forms to better deal with that could be developed as well.

1183 may be a request to merge those data for display, but I'm not sure I'm understanding that issue either.

Where am I getting lost?

[KyndallH]

So yes - there are two issues with having the data with containers.

1) Accessibility which Mariel and Angie have mentioned, we want fewer clicks to get to that information. Access through the specimen page.

2) Containers are barcodes, and barcodes change. I take a pickled specimen out of a container and move it into a new one. Which container information is retained? Does that specimen have the previous container or the new container environmental conditions? I have tubes given to me that are broken, and I change the tube plus barcode. I need the data tied to the previous container and the new one. I need this history to attached to the part, not to a barcode since a part may have several barcodes in its life. Does that make sense?

[dustymc]

Containers are barcodes,

Not quite, and being precise is important here.

Barcodes are human-readable proxies to container_id, the internal primary key which defines container data-objects. Two types of containers don't (usually) have barcodes; positions (nobody cares right now...) and specimen parts. Specimen parts are simultaneously parts and containers; that's built into the model as the bridge between those two "nodes." So two things:

1. When you "change the tube plus barcode," you are not moving a part from one container to another - the forms won't let you - you're just moving the part-container and the part which it permanently contains (or is) to a new parent, and that creates a history, and

2. "I need this history to attached to the part, not to a barcode" finally makes sense to me. Containers ARE "the specimen" in that they are simultaneously parts and containers. Containers are equally as much a part of the specimen as specimen part attributes (they both join to the same table in the same way).

Which container information is retained

All of it (IF you have decent procedures. No data model or app can deal with "data" in some weird remarks field, or you moving a part from container_type "vat of acid" into a new container with environmental data, or ...).

Does that specimen have the previous container or the new container environmental conditions?

The specimen has everything through container history. But, this is a major/valid concern. We do have full history so it's (theoretically) possible to re-create where things were and get what you REALLY want, but it's non-trivial.

fewer clicks to get to that information

Specifics, please. If I knew what you meant by "make it suck less" I'd have just made it suck less when I built it....

And from #1183...

Is it immediately useful to add the "container location summary" to specimendetail? Something like "907YukonDr:ROOM09:R55:R55C04:UAMEH585:UAMEH2547" for each part with container data?

toggle out of specimen detail into part location and work out of a separate form, with multiple clicks to find info.

Screenshots or something might help me understand what's going on here. Just click the "part location" tab, no?

part condition

Again, specifics. What exactly do you want to see where (and what can we do to make it fit there)?

tissue is subsampled

We have those data (they're in the coll_object of the subsample) but they're not displayed anywhere. What do you want to see, and where?

freezer failure.

That's a good opportunity to back out and look at the big picture and, I hope, make a big decision. Say a freezer fails:

1) With containers, you pull containers (racks) out of one container (dead freezer) and scan them into another (new freezer).

2) With not-containers (eg, specimen part attributes, although there are lots of functionally-equivalent options) you could find all the parts in the freezer by container (if you're still using containers for object tracking - if you're not, I have no idea how you find anything) and upload ~(100 x 13 x 35) new part attributes.

(2) is easy to find from the specimen - each affected part has it's own attribute (or condition determination or whatever). The data will probably be fairly simple because they're difficult to create and require a lot of disk space (eg, same string, determiner, and date on each and every one of 50,000 parts.) You're doing lots of work for easy-to-interpret data.

(1) could come from a data logger - some sort of networked device automagically writing temperature data to Arctos, or just a replacement for paper freezer logs where someone manually enters ~daily data (eg through some freezer logger app which would be simple to write). The data can be very thorough (eg, periodic cabinet temperature), and storing them requires very minimal resources (eg, one determination linked to 50,000 parts through some intermediate containers). Getting to those data from the specimen is something we still need to work out - seems possible/practical, but I don't see it being trivial, and I don't see displaying a decade's worth of daily freezer temperature temperatures (much less hourly!) on the specimen detail page. Doesn't require much (or any, with loggers) work from you, but it does require lots of work from me and perhaps following some fairly convoluted pathways to figure out what exactly was in the freezer when it died.

The data are ultimately pretty much the same and I don't think there's any technical reason to DEMAND one or the other approach. I can try to make recording part attributes easier (but I don't see them ever being as nice to work with as normalized data), or I can develop the container environment model, or maybe I can do bits of both and and add some sort of "for every part in this container, add specimen part attribute {fill in the blanks and click the button}" app, or .... (That last thing would have practical limitations - it's just a LOT of data.)

So, can we make a big-picture decision? In which direction do we want to go with this?

[dustymc]

This has progressed to #1384