Deal correctly with modification 332, fix #24

[jdesboeufs]

2016:

35890 codes in Geohisto
35885 codes in COG
Town 14624 found in geohisto but not in COG: L'Oudon
Town 20366 found in geohisto but not in COG: Chisa
Town 28361 found in geohisto but not in COG: Saint-Symphorien-le-Château
Town 44060 found in geohisto but not in COG: Le Fresne-sur-Loire
Town 49101 found in geohisto but not in COG: Clefs
Town 51664 found in geohisto but not in COG: Gernicourt
Town 78692 found in geohisto but not in COG: Butry-sur-Oise
Town 95120 found in COG but not in Geohisto: Butry-sur-Oise
Town 2B366 found in COG but not in Geohisto: Chisa

2017:

35421 codes in Geohisto
35416 codes in COG
Town 14624 found in geohisto but not in COG: L'Oudon
Town 20366 found in geohisto but not in COG: Chisa
Town 28361 found in geohisto but not in COG: Saint-Symphorien-le-Château
Town 44060 found in geohisto but not in COG: Le Fresne-sur-Loire
Town 49101 found in geohisto but not in COG: Clefs
Town 78692 found in geohisto but not in COG: Butry-sur-Oise
Town 02344 found in geohisto but not in COG: Gernicourt
Town 95120 found in COG but not in Geohisto: Butry-sur-Oise
Town 2B366 found in COG but not in Geohisto: Chisa

[davidbgk]

@jdesboeufs I still don't get how you've got these results, see the latest line of https://raw.githubusercontent.com/etalab/geohisto/63de2134f92c33d0044c4f173efe48b23d88c48a/exports/towns/towns_2017-01-01.csv at least Chisa is good?

[davidbgk]

@jdesboeufs alright, now Chisa and Butry should be OK 😉

Investigating the others.

[jdesboeufs]

2016:

35890 codes in Geohisto
35885 codes in COG
Town 14624 found in geohisto but not in COG: L'Oudon
Town 28361 found in geohisto but not in COG: Saint-Symphorien-le-Château
Town 44060 found in geohisto but not in COG: Le Fresne-sur-Loire
Town 49101 found in geohisto but not in COG: Clefs
Town 51664 found in geohisto but not in COG: Gernicourt

2017:

35421 codes in Geohisto
35416 codes in COG
Town 14624 found in geohisto but not in COG: L'Oudon
Town 28361 found in geohisto but not in COG: Saint-Symphorien-le-Château
Town 44060 found in geohisto but not in COG: Le Fresne-sur-Loire
Town 49101 found in geohisto but not in COG: Clefs
Town 02344 found in geohisto but not in COG: Gernicourt

🔥 🔥 🔥

[davidbgk]

So far the only remaining issue should be related to Oudon and friends 😎

Note that a few towns have an entry but no existence (Clefs for instance) to be consistent with lineage when there is a rename + merge…

[jdesboeufs]

Now I drop entries when start_datetime equals end_datetime. But it's not trivial for end users.

2016:

35886 codes in Geohisto
35885 codes in COG
Town 51664 found in geohisto but not in COG: Gernicourt

$ cat data/towns_2016-01-01.csv | grep Gernicourt
COM02344@1942-01-01,02344,1942-01-01 00:00:00,2016-12-30 23:59:59,Gernicourt,COM51664@2016-12-31,,DEP02@1860-07-01,47,411
COM51664@1942-01-01,51664,1942-01-01 00:00:00,2016-12-31 23:59:59,Gernicourt,COM51171@2017-01-01,,DEP51@1860-07-01,NULL,331

2017:

35416 codes in Geohisto
35416 codes in COG

🎊

[davidbgk]

Actually there was still errors with Gernicourt and Le Fresne-sur-Loire even for 2017, the latest commit should fix this.

[jdesboeufs]

2016:

35885 codes in Geohisto
35885 codes in COG

2017:

35416 codes in Geohisto
35416 codes in COG