kadowa commented 8 years ago

I am currently exploring the possibility of separating the data into two tables, one for the (physical) papyri, and one for descriptions of their pages (recto and verso – there is no hidden third option, is there? :wink:) which is referenced by the first.

The best chance to do this automatized seems to me to rely on the information in the rv and andere_seite field. If I understand correctly, rv specifies, wether a record describes recto, verso, or both and andere_seite indicates, if there is a separate record for the other page or if this page is empty.

Here is a survey of all distinct combinations of rv and andere_seite combined with their count in the current database:

# COUNT(*), rv, andere_seite
793, , 
5, Fleischseite, Verso frei
3, Fleischseite, s. eigenen Datensatz
1, Fleischseite, Haarseite frei
1, Fleischseite, Schriftspuren
2, Fleischseite vermutlich, Haarseite frei vermutlich
3, Haarseite, s. eigenen Datensatz
2, Haarseite u. Fleischseite, s. eigenen Datensatz
1, Haarseite u. Fleischseite,               
4866, Recto, Verso frei
3931, Recto, s. eigenen Datensatz
376, Recto, Schriftspuren
351, Recto, Tintenspuren
205, Recto, 
31, Recto, Verso frei vermutlich
8, Recto,               
6, Recto, Tintenspuren(?)
3, Recto, Schriftspuren(?)
2, Recto, ?
1, Recto,  Tintenspuren             
1, Recto, Recto frei
1, Recto, s. P. Heid. Inv. G 315
1, Recto, Zeichnung u. Schriftspuren
1, Recto, S. P. Heid. Inv. G(Dem) 742c
1, Recto, Unbekannt
356, Recto u. Verso, 
327, Recto u. Verso,               
27, Recto u. Verso, s. eigenen Datensatz
6, Recto u. Verso, Verso frei
1, Recto u. Verso, Tintenspuren
1, Unbekannt, 
3891, Verso, s. eigenen Datensatz
50, Verso, 
13, Verso, Recto frei
8, Verso, Verso frei
3, Verso, Tintenspuren
2, Verso, Schriftspuren
2, Verso, s. P. Heid. Inv. G 295
2, Verso,               
1, Verso, s. P. Heid. Inv. Dem. 742c
1, Verso, Siehe G 4737 Recto
1, Verso, Siehe G 4745 Recto

My observations so far (@jcowey, feel free to add your thoughts):

793 records don't have any information on pages. Is there a default state we can assume for those?
How can Haarseite and Fleischseite be matched to recto and verso (I took the liberty of converting Rückseite to verso and Vorderseite to recto already)? I suggest storing this information in a separate column and convert the terms to recto and verso for ease of processing.
A few andere_seite fields contain actual references to the record describing the page, but most have to be matched via the p_heid_inv etc., so I suggest we ignore this info.
Some of the combinations don't make sense, in particular
- Recto u. Verso, s. eigenen Datensatz
- Recto, Recto frei
- Verso, Verso frei

I will follow up with a list of ids affected by the last point; they probably require manual correction.

kadowa commented 8 years ago

SELECT id FROM papyri.papyri WHERE rv="Recto u. Verso" AND andere_seite="s. eigenen Datensatz";

kadowa commented 8 years ago

SELECT id FROM papyri.papyri WHERE rv="Recto" AND andere_seite="Recto frei";

# id
10392

kadowa commented 8 years ago

SELECT id FROM papyri.papyri WHERE rv="Verso" AND andere_seite="Verso frei";

kadowa commented 8 years ago

SELECT id FROM papyri.papyri WHERE rv="Unbekannt";

# id
191

jcowey commented 8 years ago

@kadowa Can this issue now be closed. I think that everything here has been solved by issues:

https://github.com/kadowa/papyri/issues/15 https://github.com/kadowa/papyri/issues/16 https://github.com/kadowa/papyri/issues/17 https://github.com/kadowa/papyri/issues/18

kadowa commented 8 years ago

Almost; we also need unique identifiers to do this, so #9 needs to be closed as well.

kadowa commented 8 years ago

14 as well, of course.

jcowey commented 8 years ago

Ah OK!

kadowa commented 8 years ago

TO DO:

[x] new field for standardized front/back info
[x] create separate tables ('papyriandpages`)
[x] fill tables
[x] link between tables (new fields front and back in papyri)
[x] add new fields
[x] adapt tables.py to new db structure
[x] integrate rv, andere_seite?

kadowa commented 8 years ago

Procedure for linking back and front page from papyrus table:

[x] clean (or store and clean later) ids which appear more than two times in the pages table (see #29)
[x] ~~papyri, which have one page marked fb and one marked for b: convert fb to opposite page (and store id for possible future corrections?)~~ only affects papyri appearing more than twice in the pages table (see previous item)
[x] papyri, which have exactly one page marked fb: duplicate page, mark original as f and duplicate as b

jcowey commented 8 years ago

@kadowa Lajos Berkes and I were due to have another session after coffee and before lunch today on #31. Is that going to get in your way?

kadowa commented 8 years ago

Not at all; I am currently working with a throwaway copy of the database since I can't run the scripts finally until we have unique ids. So I should be the one getting out of the way. ;-)

kadowa / papyri

Recto and verso in separate table #11

14 as well, of course.