etalab / geohisto

[UNMAINTAINED] Historic information for French regions, counties, overseas collectivities and towns based on INSEE and Wikipedia data, exported as (re)usable CSV files.
Other
28 stars 1 forks source link

Error with Freigné for 2018/01/01 export attempt #64

Open remifroger opened 5 years ago

remifroger commented 5 years ago

I'm trying to generate a version of the French communes for 2018 with this tool. I added new files "France2018" and "historiq2018" from Insee + the file "2018.csv" for EPCI in the sources folder. Also, I redirected filenames to the 2018 files in the loaders functions.

And I get an error (of course ahah). I don't understand how it happens after reading of the functions. This is the error :

C:\Users\rfr\Documents_py\geohisto-master>python -m geohisto Loading towns Loading history Loading populations Loading counties Computing history from actions Record(depcom='44225', mod=331, eff=datetime.datetime(2018, 1, 1, 0, 0), nccoff='Freigné', nccanc='', comech='44180', dep='44', com='225', depanc='', last=None, effdate=datetime.date(2018, 1, 1)) Traceback (most recent call last): File "C:\Python37\lib\runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "C:\Python37\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "C:\Users\rfr\Documents_py\geohisto-master\geohisto__main.py", line 6 6, in main() File "C:\Python37\lib\site-packages\click\core.py", line 716, in call return self.main(args, kwargs) File "C:\Python37\lib\site-packages\click\core.py", line 696, in main rv = self.invoke(ctx) File "C:\Python37\lib\site-packages\click\core.py", line 889, in invoke return ctx.invoke(self.callback, ctx.params) File "C:\Python37\lib\site-packages\click\core.py", line 534, in invoke return callback(args, *kwargs) File "C:\Python37\lib\site-packages\click_log\core.py", line 94, in wrapper return f(args, **kwargs) File "C:\Users\rfr\Documents_py\geohisto-master\geohisto\main__.py", line 3 4, in main compute(towns, history_list) File "C:\Users\rfr\Documents_py\geohisto-master\geohisto\actions.py", line 39 7, in compute raise e File "C:\Users\rfr\Documents_py\geohisto-master\geohisto\actions.py", line 39 4, in compute ACTIONS.get(record.mod, lambda a, b: a)(towns, record) File "C:\Users\rfr\Documents_py\geohisto-master\geohisto\actions.py", line 19 6, in deletion modification=record.mod File "C:\Users\rfr\Documents_py\geohisto-master\geohisto\models.py", line 216 , in generate raise Exception(msg) Exception: You cannot set 2018-01-01 00:00:00 as a start date and 2017-12-31 23:59:59.999999 as an end date for Town <Town (fr:commune:44225@2018-01-01): Freigné from 2018-01-01 to 2018-01-01 with successors and mod 0>.

I found the modification that poses problem on the historical file (Freigné, mod 331) but I don't understand what is happening (from a technical and territorial point of view), if you can enlighten me?

Thks for help and the tool!

remifroger commented 5 years ago

Autant pour moi je n'avais pas vu la PR #63 De ce qu'on m'a expliqué, Freigné a intégré une commune nouvelle d'un département différent de son département d'origine. Et pour ce faire, entre-temps, elle a connu un état second de commune délégué dans le nouveau département avant d'intégrer la commune nouvelle. Cet "état second" correspond au code Insee qui pose problème (44225), il n'a pas vraiment existé physiquement (plutôt une passerelle réglementaire je pense) D'autres cas existent, Pont-Farcy par exemple

remifroger commented 5 years ago

For information, it works if I remove these 6 rows from the historiq2018 source file

Freigné (mod. 331) Pont-Farcy (mod. 331) Evry (mod. 100) Guigneville-sur-Essonne (mod. 100) Arthieul (mod. 310) Blamécourt (mod. 310)

I'm monitoring if it's coherent