Separate Part Name from Part Preservation

campmlc commented 6 years ago

Proposal to separate part name (e.g. "tissue") from part preservation (e.g. "frozen"). Posting this as new issue related to repeated earlier discussion.

From email string: These are clearly part attributes - they apply to parts, there can be many of them, they're determinations, they're time-dependent, etc. You can add them with the "add more" widget on data entry. We'll need to set up code table values before you can use them - that's fairly trivial, you can do it under edit code tables.

They can be bulkloaded with the parts bulkloader, and are in partdetail in specimen results/downloads.

I agree that preservation should not be confounded with part name. Getting it out affects lots of things (labels, the "tissues" search, data entry, etc.) so will need some discussion. Issue please.

Current status: https://docs.google.com/spreadsheets/d/1AY1EXQzKUSAg9EvZFjayTXcCBciXd_syCxt0GSX2tww/edit#gid=485619714 will serve as a migration path on 2021-07-01

dustymc commented 5 years ago

plant term

The code table says "having all moisture removed using either heat or silica." "Silica" makes sense (to me...) as preservation - and "heat"==>"air-dried"??

info I need to determine suitability of material for loans

Sounds like a compelling reason to standardize.

Option One: You get a request for 7000 parts (biggest current loan that seems to actually examine specimens and not just data is 7054 parts). You need to find 7000 parts and read the remark for each and every one of them.

Option Two: You get a request for 7000 parts. You need filter for any that are "fixed."

campmlc commented 5 years ago

Happy to keep "fixed" in the part attribute name. As I've said before: Ideally all of this would be most "correctly" tracked through object tracking/part history, where we'd have all the above - fixtation, preservation, storage, location, refill/topoff/remount/reframe/check dates and by whom, plus history of freezer temp logs or humidity levels etc etc. And then we could have a reciprocal link to this info from the parts table, rather than a full display.

However, this what was was suggested yesterday: "preservation" with the value "XXXX" "preservation" with the value "YYYY" (for fixation).

I don't like that option unless we can specify "fixation" somewhere. Otherwise, let's go with For anything that cuurently has "YYYY, XXXX-fixed", have two attributes: "fixation" with the value "XXXX" "preservation" with the value "YYYY"

On Tue, Jan 15, 2019 at 2:43 PM dustymc notifications@github.com wrote:

plant term

The code table says "having all moisture removed using either heat or silica." "Silica" makes sense (to me...) as preservation - and "heat"==>"air-dried"??

info I need to determine suitability of material for loans

Sounds like a compelling reason to standardize.

Option One: You get a request for 7000 parts (biggest current loan that seems to actually examine specimens and not just data is 7054 parts). You need to find 7000 parts and read the remark for each and every one of them.

Option Two: You get a request for 7000 parts. You need filter for any that are "fixed."

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ArctosDB/arctos/issues/1460#issuecomment-454563106, or mute the thread https://github.com/notifications/unsubscribe-auth/AOH0hEk6feefuqSHsbUE4NYP-KQlv9X6ks5vDkuagaJpZM4SXYNg .

Jegelewicz commented 5 years ago

"fixation" with the value "XXXX" "preservation" with the value "YYYY"

I prefer the two attributes because I believe there is a difference between fixing something in formalin, then storing it in ethanol and storing something in formalin, then transferring it to storage in ethanol.

dustymc commented 5 years ago

two attributes

That's probably "correct" but...

1) I'm not completely convinced there's a clear separation, and if there's not then arbitrary usage will make it impossible to find specimens. If there is a clear separation - given the same thing the kiddos will have no choice but to enter things the same way - then this isn't a problem.

2) That's another weird code table that we'd have to look at to determine "tissue-ness." Not a deal-breaker, but it's some added complexity (and processors) that I'm not certain is necessary.

unless we can specify "fixation"

Is there some useful/functional difference between plunking something in formalin for a while and then moving it to ethanol, and fixing it in formalin and then moving it to ethanol? If not, then formalin + ethanol probably make sense. If so, then we probably need something like formalin (fixative) and formalin and ethanol - a "fixed" part would go through the first and last, something stored in formalin would use the middle (and maybe the last if it's transferred to ethanol).

KyndallH commented 5 years ago

Fixation is the same as preservation but at different times. Fixation is the initial preservation.

The code table says "having all moisture removed using either heat or silica." "Silica" makes sense (to me...) as preservation - and "heat"==>"air-dried"?? No -they actually have ovens that dry them at a low heat though higher than ambient temperature with circulating air.

campmlc commented 5 years ago

From http://clade.ansp.org/malacology/people/rosenberg/archiving/method/fixation_preservation.html : "The words fixation and preservation are used casually and often incorrectly in the collection care literature. There is a distinct difference between a fixative and a preservative. In general, fixation is more desirable for histological or detailed morphological studies, but specimens should never be stored in any fixative for long (or even short) periods of time, and indeed, with many crustaceans, fixation may not be necessary at all (as opposed to preservation in ethanol or other preservative).

A fixative actually "fixes" a specimen by stabilizing (cross-linking) the proteins within its tissues such that long afterwards, the tissues will still retain a semblance of their appearance in life. Additionally, fixing usually raises the refractive index of the tissue making it more susceptible to staining. By definition, then, a fixative is a toxic chemical with a strongly adverse affect on tissue, in a sense chemically "freezing" the tissues permanently in place. "

On Tue, Jan 15, 2019 at 5:27 PM Kyndall Hildebrandt < notifications@github.com> wrote:

Fixation is the same as preservation but at different times. Fixation is the initial preservation.

The code table says "having all moisture removed using either heat or silica." "Silica" makes sense (to me...) as preservation - and "heat"==>"air-dried"?? No -they actually have ovens that dry them at a low heat though higher than ambient temperature with circulating air.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ArctosDB/arctos/issues/1460#issuecomment-454604358, or mute the thread https://github.com/notifications/unsubscribe-auth/AOH0hLs7XJwEMMfUoYeWsXm2fL-2uADWks5vDnHvgaJpZM4SXYNg .

campmlc commented 5 years ago

Here's another: " Fixation involves the preservation and stiffening of tissues so that they do not decompose and so that the specimen remains in an easy-to-study posture. Preservation involves the storage of the specimen in a medium that allows for its safe study for many years. " https://www2.clarku.edu/faculty/pbergmann/Resources/SOP%20Specimen%20Preservation.pdf

95% ethanol is a fixative in the sense that it dehydrates (dessicates!) tissues such that they are hardened and inflexible and protected from decomposition. But it can also be a preservative for DNA.

On Tue, Jan 15, 2019 at 5:47 PM Mariel Campbell campbell@carachupa.org wrote:

From http://clade.ansp.org/malacology/people/rosenberg/archiving/method/fixation_preservation.html : "The words fixation and preservation are used casually and often incorrectly in the collection care literature. There is a distinct difference between a fixative and a preservative. In general, fixation is more desirable for histological or detailed morphological studies, but specimens should never be stored in any fixative for long (or even short) periods of time, and indeed, with many crustaceans, fixation may not be necessary at all (as opposed to preservation in ethanol or other preservative).

A fixative actually "fixes" a specimen by stabilizing (cross-linking) the proteins within its tissues such that long afterwards, the tissues will still retain a semblance of their appearance in life. Additionally, fixing usually raises the refractive index of the tissue making it more susceptible to staining. By definition, then, a fixative is a toxic chemical with a strongly adverse affect on tissue, in a sense chemically "freezing" the tissues permanently in place. "

On Tue, Jan 15, 2019 at 5:27 PM Kyndall Hildebrandt < notifications@github.com> wrote:

Fixation is the same as preservation but at different times. Fixation is the initial preservation.

The code table says "having all moisture removed using either heat or silica." "Silica" makes sense (to me...) as preservation - and "heat"==>"air-dried"?? No -they actually have ovens that dry them at a low heat though higher than ambient temperature with circulating air.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ArctosDB/arctos/issues/1460#issuecomment-454604358, or mute the thread https://github.com/notifications/unsubscribe-auth/AOH0hLs7XJwEMMfUoYeWsXm2fL-2uADWks5vDnHvgaJpZM4SXYNg .

dustymc commented 5 years ago

So that's talking about the substance - formalin is a fixative, ethanol (at some concentrations, for some purposes, except when nobody told the student it needs to be diluted....) is a preservative. Under that, "formalin fixed" is redundant. Since we can (by that definition) filter based on the substance (and not how it's used), I'm not seeing any advantage to the word "fix" in any context (if it is or was in a fixative that's presumable the only thing it could be there for), nor to separating fixatives into another table (we can identify them by what they are, so it doesn't matter where they are).

ovens

Would they use them if they were working in Albuquerque? Is the result different than freeze-drying? I'm guessing the time (at temperature??) it took to dry is what matters?? Eg, I think the goal here is (or should be) to determine suitability for use, and it's not clear how much technique we need to get that. I'm totally fine with having 900 ways to say "dry" if we need them, but they should be defined such that they're not arbitrarily used.

Jegelewicz commented 5 years ago

formalin is a fixative, ethanol (at some concentrations, for some purposes, except when nobody told the student it needs to be diluted....) is a preservative.

Not necessarily. Some specimens are preserved in formalin.....(amphibian eggs and larva)

Jegelewicz commented 5 years ago

I'm not seeing any advantage to the word "fix" in any context (if it is or was in a fixative that's presumable the only thing it could be there for), nor to separating fixatives into another table (we can identify them by what they are, so it doesn't matter where they are).

I think the attributes "fixation" and "preservation" could use the same code table, but I like the idea of having two different terms based upon Mariel's definitions above.

Jegelewicz commented 5 years ago

Would they use them if they were working in Albuquerque?

We didn't in El Paso.....

dustymc commented 5 years ago

preserved in formalin.

If I'm reading http://clade.ansp.org/malacology/people/rosenberg/archiving/method/fixation_preservation.html correctly, that's not possible - you (and most everyone else AFAIK) just left them in a fixative - eg, the difference is in the chemical, not in the technique.

two different terms

Elaborate please - are you suggesting "formalin (somehow used as a preservative)" and "formalin (somehow used as a fixative)"?

campmlc commented 5 years ago

Yes, 95% ethanol can also be both a fixative and a preservative. In this case, if we had a specimen fixed in 95% ethanol and then transferred to 70% ethanol as a preservative, and both were "preservative" according to Arctos, we'd need a way to distinguish which came first. Fixative by definition comes first before preservative. If we don't distinguish these by terms or by remarks, we need a way to distinguish them based on order of use.

We also need to consider data entry. Students now see "whole organism (ethanol-fixed)". What will they see if we break this up into two part attributes, and how will they indicate that it was fixed in 95% and then transferred to 70% for long term storage?

On Wed, Jan 16, 2019 at 10:30 AM dustymc notifications@github.com wrote:

preserved in formalin.

If I'm reading http://clade.ansp.org/malacology/people/rosenberg/archiving/method/fixation_preservation.html correctly, that's not possible - you (and most everyone else AFAIK) just left them in a fixative - eg, the difference is in the chemical, not in the technique.

two different terms

Elaborate please - are you suggesting "formalin (somehow used as a preservative)" and "formalin (somehow used as a fixative)"?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ArctosDB/arctos/issues/1460#issuecomment-454867486, or mute the thread https://github.com/notifications/unsubscribe-auth/AOH0hJS-PD_TV3zJIcUhkmVKTeczLpj_ks5vD2GYgaJpZM4SXYNg .

dustymc commented 5 years ago

campmlc commented 5 years ago

OK, that could work from a data standpoint. But from a data entry standpoint, this is getting very complex. We don't want students to have to use the add more tool for every part that is entered. We would need this to be visually integrated into the standard data entry form. Possible? Also, using attributes in this way should not require that CMs have to approve "extra" data for every load. That doubles our workload and will lead to staff resistance. How can we integrate the data entry and make it a more seamless process that doesn't require multiple complex interrelated steps?

On Wed, Jan 16, 2019 at 11:12 AM dustymc notifications@github.com wrote:

[image: screen shot 2019-01-16 at 10 07 57 am] https://user-images.githubusercontent.com/5720791/51269286-f512dc00-1976-11e9-8acd-4911f79fa92a.png

[image: screen shot 2019-01-16 at 10 10 13 am] https://user-images.githubusercontent.com/5720791/51269291-f9d79000-1976-11e9-8b27-05d091b1a703.png

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ArctosDB/arctos/issues/1460#issuecomment-454882183, or mute the thread https://github.com/notifications/unsubscribe-auth/AOH0hGUDQ87TuAhLS0MOdwBN_yKf7pcRks5vD2ttgaJpZM4SXYNg .

dustymc commented 5 years ago

visually integrated

Certainly possible, but there are some things to work out. (Like screen real estate - there are a couple dozen possible "related" loaders, all of which can contain infinite "rows.")

not require that CMs have to approve "extra" data for every load

There are a bunch of old Issues about this - it's basically a matter of me not liking the idea of loading data that ya'll haven't seen. It's more-integrated than it was, but I don't really know how you approve stuff so I'm not sure if you see "extras" or not. In any case it's 100% social - there's no reason I can't auto-load "extras" instead of sending email (I suppose I'd still need to send email for the failures). Probably a new Issue if you want to revisit that - I think it's just a matter of making sure we're all on the same page as far as what the "load" button does.

If we get to Oracle 12 there might be some other possibilities - it allegedly speaks JSON (and has a datatype to hold it), so the form could write a "partstring " (many parts each with many attributes) into one field, and unroll it for display on the data entry form, when you're approving records to load, etc. I don't have the capability to move beyond theoretical at the moment, but that's a standard way of moving complex data around so from a very superficial viewpoint I don't see why it wouldn't work.

Jegelewicz commented 5 years ago

Elaborate please - are you suggesting "formalin (somehow used as a preservative)" and "formalin (somehow used as a fixative)"?

I am suggesting that in the image you created above, the first "preservation" would instead be "fixation".

dustymc commented 5 years ago

FYI that's just a screen shot of data entry in production.

the first "preservation" would instead be "fixation"

I think that's "arbitrary usage==nobody can find anything" territory.

A specimen is stored in 95% ethanol because some student forgot to dilute.
A specimen is being fixed in 95% ethanol and will be transferred to 70% tomorrow.
A specimen is allegedly being fixed in 95% ethanol, but the transfer scheduled for tomorrow won't actually happen so it's really storage.
A specimen is stored in 95% ethanol because whatever made http://arctos.database.museum/SpecimenResults.cfm?part_name=whole%20organism%20(95%25%20ethanol) (11249 specimens) exist happens.
Etc. I don't see any reason a student would pick one or the other, and therefore I don't see any way a researcher (or collection manager) could find those specimens.

That's also one more weird code table, more expensive queries, etc.

I'd be more accepting of something that can ONLY appear in one table or the other, so it can't be arbitrarily used, but that gets us back to "95% ethanol is a fixative, not a preservative" which (1) seems to conflict with existing data, and (2) negates any benefit of the separation.

campmlc commented 5 years ago

Issues meeting 5-2-19 @campmlc OKs @Dusty will proceed with MSB Weirdness: tissueness comes from parts; but that comes from preservation; this might affect has tissues search Use DGR:Mamm

campmlc commented 5 years ago

Set up separate tissue flag meeting to assign tissueness to preservation types http://arctos.database.museum/Admin/CodeTableEditor.cfm?action=editCTPART_PRESERVATION&tbl=CTPART_PRESERVATION Mariel Carla Andy Others?

dustymc commented 5 years ago

go with DGR Mamm

dustymc commented 5 years ago

@campmlc @Jegelewicz DGR:Mamm data attached.

CID - my identifier
GUID - guid
NEW_PART_NAME - part name will become
NEW_PRES_1 - one preservation attribute will become
NEW_PRES_2 - one preservation attribute will become
PART_NAME - old/existing part name
PID - my identifier

I added blood (EDTA) to https://docs.google.com/spreadsheets/d/1jTgHlcdu_bptiHx7FTK17PYUFfpHsrixfX_favM1kGc/edit#gid=25221597 - I don't think anything else of that nature slipped through the cracks, but a quick review of this before I proceed would be greatly appreciated - do you see anything wrong with the attached data?

I'm not sure what to do with this:

2279707	DGR:Mamm:2094	whole organism, dissected	DELETE, but any part currently with this preservation will need 2 attributes	whole organism, dissected (ethanol, formalin-fixed)	24484329

I suppose lacking better ideas I'll just ignore it for now??

Shall I proceed with normalizing DGR:Mamm parts?

temp_msbmamm_pts(1).csv.zip

Jegelewicz commented 5 years ago

Here is what I would do with that:

NEW_PART_NAME - whole organism NEW_PRES_1 - formalin NEW_PRES_2 - ethanol Condition - dissected

Jegelewicz commented 5 years ago

Otherwise - I think the rest looks OK and will be a good test run, but let @campmlc decide!

campmlc commented 5 years ago

I agree with Teresa on that one. I made a couple of edits to the google spreadsheet. Karyotype (dry, slide) should just be karyotype (slide). K2CR2O7 should be potassium dichromate. All else looks good to go.

On Mon, May 13, 2019 at 1:47 PM Teresa Mayfield-Meyer < notifications@github.com> wrote:

Otherwise - I think the rest looks OK and will be a good test run, but let @campmlc https://github.com/campmlc decide!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ArctosDB/arctos/issues/1460?email_source=notifications&email_token=ADQ7JBG7UJBPL7CVSWN6RHDPVHAWVA5CNFSM4ES5QNQKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODVJLRGY#issuecomment-491960475, or mute the thread https://github.com/notifications/unsubscribe-auth/ADQ7JBGJJ4MXFVSUUW72QTLPVHAWVANCNFSM4ES5QNQA .

dustymc commented 5 years ago

@campmlc the header had been excel-sorted into the data - I recovered it back, I hope I didn't overwrite anything but please check.

Anything in the pres columns should also be in CTPART_PRESERVATION - please just put the values in there. Here are the problems:

UAM@ARCTOS>  select distinct PRES1 from temp_part_map where PRES1 not in (select PART_PRESERVATION from CTPART_PRESERVATION);

PRES1
------------------------------------------------------------------------------------------------------------------------
formalin, 5% buffered
DELETE, but any part currently with this preservation will need 2 attributes
use slide only; no need for dry

I added an append_to_condition column and can deal with that during the migration.

I think "whole organism, dissected (ethanol, formalin-fixed)" is where it needs to be now, but everything else with random stuff in the pres columns will need sorted out.

Here's the updated DGR data - I might have it pushed out before you see this....

temp_msbmamm_pts(2).csv.zip

dustymc commented 5 years ago

MSB:Mamm is changed.

campmlc commented 5 years ago

OK, I fixed the file - please check. DGR looks ok.

On Mon, May 13, 2019 at 3:28 PM dustymc notifications@github.com wrote:

@campmlc https://github.com/campmlc the header had been excel-sorted into the data - I recovered it back, I hope I didn't overwrite anything but please check.

Anything in the pres columns should also be in CTPART_PRESERVATION - please just put the values in there. Here are the problems:

UAM@ARCTOS> select distinct PRES1 from temp_part_map where PRES1 not in (select PART_PRESERVATION from CTPART_PRESERVATION);

PRES1

formalin, 5% buffered DELETE, but any part currently with this preservation will need 2 attributes use slide only; no need for dry

I added an append_to_condition column and can deal with that during the migration.

I think "whole organism, dissected (ethanol, formalin-fixed)" is where it needs to be now, but everything else with random stuff in the pres columns will need sorted out.

Here's the updated DGR data - I might have it pushed out before you see this....

temp_msbmamm_pts(2).csv.zip https://github.com/ArctosDB/arctos/files/3174895/temp_msbmamm_pts.2.csv.zip

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ArctosDB/arctos/issues/1460?email_source=notifications&email_token=ADQ7JBFRRFP7BVNEMWP2J7TPVHMONA5CNFSM4ES5QNQKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODVJTUQI#issuecomment-491993665, or mute the thread https://github.com/notifications/unsubscribe-auth/ADQ7JBAFWJCAU5VZ4GR4VMLPVHMONANCNFSM4ES5QNQA .

Jegelewicz commented 5 years ago

Concatenate preservation methods for UI.

dustymc commented 5 years ago

add to bulkloader

part_preservation_n_1
part_preservation_date_n_1
part_preservation_n_2
part_preservation_date_n_2

EDIT: AWG says do it

Jegelewicz commented 5 years ago

Can we magic in "frozen" + date any time something is scanned into a freezer?

dustymc commented 5 years ago

Can we magic in "frozen" + date any time something is scanned into a freezer?

Yes, but I think that's another discussion.

it's denormalization - who should we believe if there's a conflict?
it's denormalization - do you really want 12000 'yup, still frozen' attributes? (You probably should...)
it could get fairly complex - does scanning a rack into a freezer trigger this for all parts? (Probably yes, but not sure I have the processors to pull it off)
etc.

Jegelewicz commented 5 years ago

Yeah, I thought about all that after I posted this - ideally the container history will suffice, it would just be nice not to also have to enter the preservation method as frozen, which kinda doubles the work. Maybe some brilliant way to do that will come to me in a dream....

Jegelewicz commented 5 years ago

No options when attempting to enter preservation part attribute.

I decided to try out the preservation part attribute, but when I select attribute type preservation, there are no choices in the value field.

dustymc commented 5 years ago

Stupid tissue thing...

Should be happy now - thanks!

Jegelewicz commented 5 years ago

Well it works for creating, but not for editing...

dustymc commented 5 years ago

blargh - reload....

Jegelewicz commented 5 years ago

@campmlc I used this on http://arctos.database.museum/guid/MSB:Mamm:129862

It should be easy to pull the "frozen" out and add an attribute, but the fact that this was moved from -20 to -80 (included in the specimen remarks) might take a little more effort. Now that you have this example - how would you like it to appear on the specimen record? Does it facilitate the kinds of searches you might do?

Jegelewicz commented 5 years ago

We now have the ability to create attributes for parts and thus separate preservation from the part name. Next up is to document, test, create a better UI (after PG), make recommendations for use.

See ArctosDB/arctos#2354

dustymc commented 4 years ago

This will have a major impact on essentially all collections, and therefore seems like a good test of https://github.com/ArctosDB/arctos/issues/2346.

dustymc commented 4 years ago

This seems to have floated to the top of my pile again, and we now have a functional test environment. I'm still concerned about how this might work (or fail to!) in the bulkloader. I think there are three obvious approaches.

Do nothing; use the 'extras' for attribute-having parts. This is the most robust solution (it'll deal with six of any attribute for any number of parts), but also has some separation in that it writes to multiple "files." https://github.com/ArctosDB/arctos/issues/2974 has the capacity to make this more integrated than it currently is, but I'm not sure that'll completely mitigate the disassociated nature of this approach.
Add to the specimen bulkloader
- part_n_attribute_type_m
- part_n_attribute_value_m
- part_n_attribute_unit_m
- part_n_attribute_date_m
- part_n_attribute_determiner_m
- part_n_attribute_remark_m

where n is the number of parts (currently 12) and m is the number of attributes per part (not sure what's realistic, 2 seems to be a minimum). That gets everything in one table, but it's a lot of numbers to keep straight and a lot of columns. part_12_attribute_6 (6 attributes is probably a "should handle most normal situations" number) would be (12 parts 6 columns) + (6 attributes 6 columns * 12 parts)==about a thousand part-columns. I hate it.

https://github.com/ArctosDB/arctos/issues/1460#issuecomment-517406899 would add (4 preservation-columns * 12 parts)==48 new columns, be fairly simple to understand, BUT only work for preservation (which I believe would exclude a bunch of recent paleo-data that's been pushed to attributes) and would come with no ability to record who/when/remarks/etc.

My vote is for (1); I think it's the most usable and it's certainly the most capable.

https://arctos.database.museum/info/ctDocumentation.cfm?table=ctspecpart_attribute_type has grown significantly since this Issue was started; I'm bumping this back to needs discussion.

Jegelewicz commented 4 years ago

My vote is for (1); I think it's the most usable and it's certainly the most capable.

Agree - although I would actually suggest more than six attributes per part...maybe?

dustymc commented 4 years ago

more than six attributes per part

That would just involve rebuilding the parts bulkloader, which doesn't seem too crazy and wouldn't (much) affect any "core" tools. Separate Issue I think - a use case would be helpful, that seems like a lot, analyzing might lead somewhere useful.

campmlc commented 4 years ago

If ArctosDB/arctos#1, this must be done in conjunction with improvements in the single record data entry and catalog record parts UI, otherwise it is going to cause extreme upset and confusion. Any way to add a "preservation" column to single record data entry parts table and catalog record parts table, but the preservation data that get entered or are viewed there are actually populating/populated by the part attribute loader/part attributes?

On Tue, Aug 4, 2020 at 12:14 PM dustymc notifications@github.com wrote:

[EXTERNAL]*

more than six attributes per part

That would just involve rebuilding the parts bulkloader, which doesn't seem too crazy and wouldn't (much) affect any "core" tools. Separate Issue I think - a use case would be helpful, that seems like a lot, analyzing might lead somewhere useful.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ArctosDB/arctos/issues/1460#issuecomment-668748442, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADQ7JBDKDJXFBHDVNTADVFDR7BFWJANCNFSM4ES5QNQA .

dustymc commented 4 years ago

improvements in the single record data entry

New Issue. I'll need details.

single record data entry and catalog record parts UI

I'm not sure what this means - those handle very different kinds of data.

Any way to add a "preservation" column to single record data entry parts table and catalog record parts table, but the preservation data that get entered or are viewed there are actually populating/populated by the part attribute loader/part attributes?

Not that I can see - you'd end up with a big pile of things that need to be linked but have nothing capable of linking them, nor preventing them from being manipulated independently.

We could perhaps get rid of parts in the bulkloader altogether and display from the linked table in that grid, but that's https://github.com/ArctosDB/arctos/issues/2178 and not something that I'd want to approach incrementally.

campmlc commented 4 years ago

We need a better UI for adding parts as "extras" than the tiny drop down box at the bottom of the page. If all parts have to be entered that way in order to capture preservation, there will be a lot of very unhappy users. Am I understanding correctly?

On Tue, Aug 4, 2020, 12:32 PM dustymc notifications@github.com wrote:

[EXTERNAL]*

improvements in the single record data entry

New Issue. I'll need details.

single record data entry and catalog record parts UI

I'm not sure what this means - those handle very different kinds of data.

Any way to add a "preservation" column to single record data entry parts table and catalog record parts table, but the preservation data that get entered or are viewed there are actually populating/populated by the part attribute loader/part attributes?

Not that I can see - you'd end up with a big pile of things that need to be linked but have nothing capable of linking them, nor preventing them from being manipulated independently.

We could perhaps get rid of parts in the bulkloader altogether and display from the linked table in that grid, but that's ArctosDB/arctos#2178 https://github.com/ArctosDB/arctos/issues/2178 and not something that I'd want to approach incrementally.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ArctosDB/arctos/issues/1460#issuecomment-668756866, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADQ7JBFGVTNAVNMBYLXJXPLR7BH3LANCNFSM4ES5QNQA .

Jegelewicz commented 4 years ago

Hopefully that is what we are working toward with the Locality Bulkload Tool. I'm pretty sure Dusty has a global plan....

dustymc commented 4 years ago

global plan

Ish, but there are plenty of usability details to work out - eg stuff that works in a UI may not work for ya'll tabular form and vice-versa. Displaying a component loader in the DE screen is trivial, writing to it is trivial-ish, displaying that in a single table, or avoiding melting that UI (and the software that feeds it) with the 8 terabytes of parts you've bulkloaded separately, maybe not so much.

tiny drop down box at the bottom of the page.

That's just a UI object capable of spawning a javascript function. I can do a full-screen flashing red button if you want....

a lot of very unhappy users.

That's what I'm trying to avoid by throwing this back for more discussion. I need ya'll to tell me what (a) satisfies this Issue, and (b) works for you and your techs in whatever format you intend to use it.

campmlc commented 4 years ago

What would work best is to do one of two things: 1) keep the current user interface for single record data entry as is for now, except for adding a separate column for preservation next to disposition and condition. Make that a code table controlled field that will eventually populate a part attribute. But all the user sees and enters is a value for preservation in it's own column. 2) In the current user interface, replace the part data entry with a modified version of the extras table as a popup. But substantial modifications are necessary. For one, it needs to be at least visible. Right now, if I click extras, I see nothing until I scroll down and finally hit the table. It isn't visible initially in my browser view in Firefox. If I add a part, it takes several clicks to say, yes, I accept it, and then it disappears from view permanently. No idea what I just did, and no ability to make any corrections from the same data entry view.

If we switch to pop up boxes for each table in data entry, that may actually make data entry a lot easier. Allow scrolling back or continue to next. Prompt entry of one type of data at a time.

On Wed, Aug 5, 2020 at 9:58 AM dustymc notifications@github.com wrote:

[EXTERNAL]*

global plan

Ish, but there are plenty of usability details to work out - eg stuff that works in a UI may not work for ya'll tabular form and vice-versa. Displaying a component loader in the DE screen is trivial, writing to it is trivial-ish, displaying that in a single table, or avoiding melting that UI (and the software that feeds it) with the 8 terabytes of parts you've bulkloaded separately, maybe not so much.

tiny drop down box at the bottom of the page.

That's just a UI object capable of spawning a javascript function. I can do a full-screen flashing red button if you want....

a lot of very unhappy users.

That's what I'm trying to avoid by throwing this back for more discussion. I need ya'll to tell me what (a) satisfies this Issue, and (b) works for you and your techs in whatever format you intend to use it.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ArctosDB/arctos/issues/1460#issuecomment-669276741, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADQ7JBDMSXREKR6MO3V4ULLR7F6TFANCNFSM4ES5QNQA .

dustymc commented 4 years ago

Your (1) is my (3) in https://github.com/ArctosDB/arctos/issues/1460#issuecomment-668743745

Your (2) is somewhere between my (1) (if we do some confusing semi-redundant thing with the part-columns in table bulkloader or ???) and https://github.com/ArctosDB/arctos/issues/2178.

switch to pop up boxes for each table in data entry, that may actually make data entry a lot easier.

Been there, done that, twice - once in Versata, once in Access. Users struggle with multiple views.

Might be that we just can't do this without ArctosDB/arctos#2178

ArctosDB / arctos

Separate Part Name from Part Preservation #3762

PRES1