denshoproject / ddr-local

Web UI used for interacting with DDR collections and entities on a local machine.
Other
3 stars 0 forks source link

Entity detail page throws TypeError related to sorting for some Entities with child Segments #291

Closed gjost closed 4 years ago

gjost commented 4 years ago

2020-08-19 15:46 @sarabeckman reports problem displaying ddr-densho-1003-9 using local VM running 5.0.11. TypeError at _ui_entityddr-densho-1003-9.zip

15:53 @gjost gjost it's trying to sort entities against files. what are you trying to do? 15:54 @sarabeckman This happened the first time I'm trying to look at the entity in editor 15:55 So I clicked into the collection, then the page with the object list, then clicked on entity 9 and got this error 15:57 @gjost gjost can you look at other entities? in other collections? 15:57 @sarabeckman let me try real quick 15:59 alright I was able to look at entities in a photo/doc collection ddr-densho-371 15:59 However I tried another VH collection ddr-densho-1001-2 and I got the same sort of error 16:02 @gjost well see if you can find some pattern 16:03 @sarabeckman Could it be caused by html tags being used in the description? 16:04 @gjost nope, not a problem with the inputs. it's a problem with sorting the IDs. the program splits the IDs into parts and it can't compare an int to a string. 16:04 i'm just curious why we never saw this before. i haven't touched this part of the codebase in forever 16:04 @GeoffFroh can you open any of the entities in ddr-densho-1003? or is just 9? 16:06 @sarabeckman I can load entity 10 16:06 can't load 1-9 16:06 @GeoffFroh Ok — one sec… 16:09 hmmm…I’m flumoxxed. 16:10 @sarabeckman It looks like entity 10 is the only one I can load in ddr-densho-1003 16:10 @gjost i started at 8 today so i'm at my limit 16:11 @sarabeckman okay I'm near my quitting time too. 16:28 @pkikawa 10 is the only entity that doesnt have anything in the "file_groups" field 16:28 1003-1 entity.json 16:28 1003-10 entity.json 16:30 all 1-9 have transcript in the file_group while 10 does not (edited) 16:32 testing now, though i did test transcript files pre-deployment. 08:28 @GeoffFroh I agree — don’t think that’s a related issue 08:32 @pkikawa: to clarify, you are not able to replicate the error that @sara.beckman reported earlier with ddr-densho-1003-1 — ddr-densho-1003-10? i.e., trying to open the entity detail page throws a TypeError exception with the message < not supported between instances of 'str' and 'int' 08:35 @gjost The problem with the sorting is that Python can't sort/compare different types - it can't compare a str and an int. In this case we're comparing an entity and a file for some reason, which wasn't really in the use-case. 08:37 (The Identifier sorting algo compares the internal representation of IDs as lists of str or int. Number segments are turned into int so that they sort numerically rather than as strings.) 08:38 but first we should figure out why it's comparing Entity and File objects. My guess is the Entity has Segment objs

GeoffFroh commented 4 years ago

Entities displaying error behavior:

ddr-densho-1003-1 -- ddr-densho-1003-9 ddr-densho-1003-11 -- ddr-densho-1003-16 ddr-densho-1001-2 ddr-densho-1000-1 -- ddr-densho-1000-18 ddr-densho-1000-20 -- ddr-densho-1000-25

Entities tested known not to display error behavior:

ddr-densho-1003-10 (NOTE: this interview is missing transcript data) ddr-densho-1000-19

TODO:

gjost commented 4 years ago

Sorting occurs in ddr-cmdln/ddr/DDR/models/common.py in the __eq__ and __lt__ methods

OBJECT.identifier.id_sort is a list of str and int (ex: ['ddr', 'densho', 1003, 1])

pkikawa commented 4 years ago

objects ddr-densho-1003-11 thru 1003-16 throw the same error

pkikawa commented 4 years ago

took a look at the first page of ddr-densho-1000 objects. objects 1-18 and 20-25 throw the error, 19 does not

pkikawa commented 4 years ago

all of the 1001 objects (that have children) throw the error

pkikawa commented 4 years ago

looked at a couple other collections with segments/children (created in testing) and wasn't able to replicate the error

gjost commented 4 years ago

Looks like this is due to the Python3 upgrade. Python2 would sort mixed lists of str and int but Python3 decided not to.

The ddr-local webui.views.entities.detail view where this code branches off isn't actually trying to list the objects. It's trying to build that list of the types/roles of children the entity might have (i.e. segments, master files, mezzanines, transcripts, etc). The list of roles has item counts. We're not displaying the child objects so it looks like we just need to avoid sorting them.

gjost commented 4 years ago

Fixed in ddr-cmdln commit 39bc6bdf23 for package ddrlocal-master_5.0.13~deb10. Check out develop branch if you want to try it right now.

pkikawa commented 4 years ago

confirmed working on dev branch on local vm

gjost commented 4 years ago

Fixed in ddr-cmdln commit 39bc6bdf23 for package ddrlocal-master_5.0.14~deb10.

pkikawa commented 4 years ago

confirmed working on master 5.0.14 cmdln/local on local vm.