search

sfg-taxonpages / orthoptera

0 stars 0 forks source link

lost author #12

Closed typophyllum closed 9 months ago

typophyllum commented 10 months ago

Dorjsuren, Altanchimeg She is author of three sources which migrated with only the other authors (two papers have altogether 3 authors, one has 4) . Fixed.

LocoDelAssembly commented 10 months ago

Just in case can you share which sources have been fixed?

typophyllum commented 10 months ago

id=186378, id=173960, id=173961 imagen

Any idea how this could have happened? Found by chance fixing source without serial (copying title into search source task and then searching reference by author and year in old OSF, and surprised that first author was missing in TW).

klausriede commented 10 months ago

I strongly suggest a check routine to compare old vs migrated database. This is trivial for bibliography: Just export two flat tables from each version, dump them into a new database, and compare the tables using SQL. They should be identical!

LocoDelAssembly commented 10 months ago

I have no clue how this happened but according to import logs surprisingly this is the only instance and looks you found all cases:

hernan@sf12:~/onedb2tw/logs/FINAL$ cat *create_source_roles* | grep -i PersonID | grep -i ERROR
[ERROR]2023-08-26 05:24:07.266: Missing person ERROR: SF.PersonID = 31577 (person_nil_counter = 1)
[ERROR]2023-08-26 05:24:07.583: Missing person ERROR: SF.PersonID = 31577 (person_nil_counter = 2)
[ERROR]2023-08-26 06:41:55.315: Missing person ERROR: SF.PersonID = 31577 (person_nil_counter = 3)
hernan@sf12:~/onedb2tw/working$ cat tblPeople.txt | grep 31577
31577   1   31577   0   Dorjsuren   Altanchimeg A.  ""  513 0   2023-08-10 18:47:00 110 2023-05-08 16:00:00 148

Logs from both production and an isolated machine (sf12) that ran the same import both have exactly the same set of missing authors (i.e. background activity is not the reason for this problem).

The create_people stage, although it reports some errors when creating alternate spellings because them are duplicates, there is not a single case of a person failing to be created.

mjy commented 10 months ago

I strongly suggest a check routine to compare old vs migrated database.

@klausriede you can help with this general class of problems! The filtering and API endpoints let you run powerful queries, download the results in numerous different formats, and do some sleuthing (perhaps using OpenRefine) to track down data issues?