hcts-hra / ecpo-annotator

Annotator for Page Segmentations in ECPO
https://ecpo.existsolutions.com/exist/apps/ecpo/
MIT License
2 stars 2 forks source link

Error of loading annotations #105

Open xiejia1995 opened 2 years ago

xiejia1995 commented 2 years ago

there are 5 issues with errors when loading the annotations

https://ecpo.existsolutions.com/exist/apps/ecpo/annotator/?document=https://kjc-sv002.kjc.uni-heidelberg.de:8080/fcgi-bin/iipsrv.fcgi?IIIF=imageStorage/ecpo_new/jingbao/1939/04/jb_3798_1939-04-24_0002%25252B0003.tif/full/full/0/default.jpg

if i click "download", it shows "error: 500 ....."

check here: https://ecpo.existsolutions.com/exist/apps/ecpo/api/annotations/?document=https://kjc-sv002.kjc.uni-heidelberg.de:8080/fcgi-bin/iipsrv.fcgi?IIIF=imageStorage/ecpo_new/jingbao/1939/04/jb_3798_1939-04-24_0002%25252B0003.tif/full/full/0/default.jpg&items-per-page=1000

the 5 issues are: jb_3796_1939-04-22_0005+0008 jb_3798_1939-04-24_0002+0003 jb_3803_1939-04-29_0002+0003 jb_3803_1939-04-29_0005+0008 jb_3803_1939-04-29_0006+0007

line-o commented 2 years ago

Did these occur today? I am currently making a data backup and this might lead to problems. Are you currently working with the demo instance? When can the system be offline for a backup?

line-o commented 2 years ago

The backup is done now but the problem persists. I was able to download the annotations of a different image. So this must be an issue with one or more annotations of this specific image.

MatthiasArnold commented 2 years ago

we did not do the tests today, but last week. because of a conference this weekend, it took a bit longer to document the issue. so the issue is not because of the data backup. thanks for backing the data up! :)

yes. our experience is that the issue only occurs with the annotations of these 5 images. drawing boxes and grouping works with the other images. my guess is that because of the dysfunctional annotation system last year, some errors might have become too big, so the annotations cannot be loaded correctly anymore.

how to proceed? 1) i'd like to ask you to try to restore as many of the annotations of these 5 folds as possible - if possible. 2) if that does not work, we probably must start again from scratch for these folds. in this case, can you delete all the erroneous remnants of the previous bounding boxes? thanks!

line-o commented 2 years ago

@MatthiasArnold unfortunately I will not have time to do either but I am confident that most of the annotations can be recovered (it is very likely just botched group annotations) from the data backup and one or two queries in eXide.

line-o commented 2 years ago

While investigating this issue it is indeed very likely that only GroupAnnotations created at a certain point in the past now cause the problem.

One sample for an annotation for jb_3803_1939-04-29_0005-0008

<group id="s-15737505168123" color="#e67c73" top="" left="" width="" height=""/>
line-o commented 2 years ago

In the recent backup 32 such annotations could be found. The good news is that there are over 1500 GroupAnnotations that do have the boundaries set correctly.

line-o commented 2 years ago

I was able to remove all 32 broken group annotations on the demo server. Now all remaining 99 annotations (shapes) load again for https://ecpo.existsolutions.com/exist/apps/ecpo/annotator/?document=https://kjc-sv002.kjc.uni-heidelberg.de:8080/fcgi-bin/iipsrv.fcgi?IIIF=imageStorage/ecpo_new/jingbao/1939/04/jb_3803_1939-04-29_0005%25252B0008.tif/full/full/0/default.jpg

line-o commented 2 years ago

@xiejia1995 please review and close if the issue was fixed.

line-o commented 2 years ago

@duncdrum the xquery I used to fix the data was

collection("/db/apps/wap-data")//group[@left=""] ! xmldb:remove("/db/apps/wap-data", util:document-name(.))