dustymc
commented
1 year ago Thanks for bringing this here.
I think the approach of the old summarize form just isn't sustainable. (The code/libraries definitely isn't sustainable; just keeping what's there is not an option.)
I think a new dedicated summarize form which uses the current search form (a vague idea I carried for a while) isn't going to be sustainable for all the same reasons - I just can't adequately anticipate a near-infinite number of possibilities from many sources (and I don't think we have the CPU to reliably support that if I could - a constant issue on the current form).
I like the current summarize form, and don't think it should be replaced. Summarizing results is useful for lots of reasons. (I'm also not a fan of multiple ways of doing the same thing, but I don't think that's quite any of what I'm about to propose either.)
Potentially there could be some sort of passthrough on the existing form, but that does not avoid the expense of getting everything that you intend to summarize in a cache first, and it would add significant complexity to a form which I'm desperately trying to keep on this side of "sustainable." I think we can rule that out (but I'm putting it out here Just In Case).
A new summarize form which reuses the search form code (as above), but in a limited capacity seems perhaps most plausible. I think I can make that work if we limit it to things that are in FLAT, if that's viable. (A new dedicated summarizing API would be required so even if that's all acceptable and somehow works better than I could possibly hope for this would not be entirely trivial.)
Individual count must be the sum of all the individual counts of that taxon
And that makes me think that a data-driven self-assembling summarize form just isn't possible, and none of the above should be considered. There's nothing in the data which suggests some column must be summed rather than grouped; that requires me writing code which anticipates usage (and potentially dealing with errors, because those data aren't typed as strongly as this usage really needs). The core use case leading to this Issue seems to involve a very few pre-planned summaries, and maybe a dedicated form is just wild overkill for that. Maybe we'd be much better off addressing those as reports, or dedicated forms, or SOMETHING which meets the need (maybe also while explaining what's going on to the user - it's sort of a disjointed arbitrary table at the moment...) without trying to also develop a complex form and complex UI behind it in perhaps-futile attempts to do more than what's really needed.
DerekSikes
Jegelewicz
campmlc
ebraker
Issue Documentation is http://handbook.arctosdb.org/how_to/How-to-Use-Issues-in-Arctos.html
The new summarize by has lost many of the very valuable features of the old summarize by. Some issues below
The prior version was actually much faster than a nonsummary search - in this version we have to do a nonsummary search first, which can be very slow and even time out for big searches (which was not a problem with the prior version because far fewer fields were being returned). I would sometimes use the summary search as a shortcut to search for catalog records because of the massive time savings. It would be better if this advantage of summarize was restored to its prior superiority.
And, the summarize by shows no logical sort or arrangement by taxon as the prior results had. The taxa ranks should be sorted from highest (eg Kingdom) on the left to lowest (species or identified by) on the right as they had been. And ideally, sorted alphabetically by each or at least the highest rank displayed. This is how everyone who knows higher scientific names expects them to appear in list summaries.
At the bottom of my collection webpage I have saved summary searches that use the old form/results to allow visitors to my page to see what's in the collection - and this new one looks like a big problem - (not the least of which is if we require login they won't work if the user is not logged in!!). The old results had the taxa sorted as they should (larger on left, smaller on right) and sorted them alphabetically. For example:
http://arctos.database.museum/saved/uamphylaclassesorders
It's a massively important function of Arctos to provide the public with quick summaries (not just cat records data for download) of what's in a collection. If this fails (and it seems to be headed in the wrong direction) then I might have to make static snapshots summaries that I build into my website like it's the 1990s.
And individual count is not working on summarize. Individual count must be the sum of all the individual counts of that taxon as it was before. It must be equal to or greater than the RecordCount for that row in the summary table.
The other summarize by was working pretty damn well and only needed to have the link to records problem solved. Now that is solved and a dozen other problems have appeared.
Priority Please assign a priority-label. Unprioritized issues get sent into a black hole of despair. high - the public-facing value of summarize by is massively important!