SSHOC / sshoc-marketplace-backend

Code for the backend
Apache License 2.0
2 stars 0 forks source link

Mixed up results for concept list #397

Closed KlausIllmayer closed 10 months ago

KlausIllmayer commented 1 year ago

Cesare found out that when processing the full concept list of the rather big vocabulary sshoc-keyword there are some concepts missing and some concepts are duplicated (the same concepts, identified by the code). I'm still trying to find a repeatable example, but chances are high, that it could share the same problems as identified in #373. Especially, as it seems that it is connected to pagination. @tparkola Can you have a look into the GET /api/vocabularies/{code} if there is the same problem with creating pages as we saw for GET /api/actors?

KlausIllmayer commented 1 year ago

After some testing we are quite sure that for the GET /api/vocabularies/{code} we do have the same pagination issue as discovered also in #373. Cesare did look for the code sitemap which should only be there one time, but running it on GET /api/vocabularies/sshoc-keyword with perpage=20 returns this value 10 times on different pages and with perpage=100 it is returned two times on two different pages. Should be solved the same way as in #373: order the return list by label and code should solve the issue.

I add this to milestone Q2 because it causes problems with our curation efforts described in https://github.com/SSHOC/marketplace-curation/issues/1

tparkola commented 11 months ago

In theory it should not cause similar problems. In case of vocabularies the order is by "ord" value from the database, so it should be fine. I went through the whole vocabulary (on production site that you mentioned) with perpage=100 and found only one page with sitemap code. Can you provide me links to two separate pages of results that contain the same code? Or is this happening randomly in your experience?

KlausIllmayer commented 10 months ago

Fix works like a charm, can't now find any duplicates with the script.

KlausIllmayer commented 10 months ago

Follow up regarding multiple concepts having the same "ord" value: #420