search

briot / geneapro

Genealogical software based on the GenTech data model
http://briot.github.com/geneapro/
GNU General Public License v2.0
33 stars 8 forks source link

Import fails when CHAN doesn't exist in GEDCOM file or is None/Null. #38

Closed changeling closed 5 years ago

changeling commented 5 years ago

Note that the failure occurs on the first INDI without a CHAN tag:

2019-02-08 23:13:01,300 (0.000) INSERT INTO "persona" ("name", "description", "last_change") VALUES ('Individual Middle /Tester1/', NULL, '2019-01-28 22:43:37'); args=['Individual Middle /Tester1/', None, '2019-01-28 22:43:37']
2019-02-08 23:13:01,301 (0.000) INSERT INTO "persona" ("name", "description", "last_change") VALUES ('Father Middle /Tester1/', NULL, '2019-01-28 22:43:37'); args=['Father Middle /Tester1/', None, '2019-01-28 22:43:37']
2019-02-08 23:13:01,303 (0.000) INSERT INTO "persona" ("name", "description", "last_change") VALUES ('Mother Middle /Tester2/', NULL, '2019-01-28 22:43:37'); args=['Mother Middle /Tester2/', None, '2019-01-28 22:43:37']
2019-02-08 23:13:01,303 (0.000) INSERT INTO "persona" ("name", "description", "last_change") VALUES ('Mother Middle /Tester3/', NULL, NULL); args=['Mother Middle /Tester3/', None, None]
2019-02-08 23:13:01,305 parse Unexpected Exception during parsing

Using the following test file: test_short.ged.zip

If a CHAN tag is added to that INDI, the import then fails on source.last_change instead of persona.last_change

2019-02-08 23:17:22,651 (0.000) INSERT INTO "persona" ("name", "description", "last_change") VALUES ('Individual Middle /Tester1/', NULL, '2019-01-28 22:43:37'); args=['Individual Middle /Tester1/', None, '2019-01-28 22:43:37']
2019-02-08 23:17:22,652 (0.000) INSERT INTO "persona" ("name", "description", "last_change") VALUES ('Father Middle /Tester1/', NULL, '2019-01-28 22:43:37'); args=['Father Middle /Tester1/', None, '2019-01-28 22:43:37']
2019-02-08 23:17:22,653 (0.000) INSERT INTO "persona" ("name", "description", "last_change") VALUES ('Mother Middle /Tester2/', NULL, '2019-01-28 22:43:37'); args=['Mother Middle /Tester2/', None, '2019-01-28 22:43:37']
2019-02-08 23:17:22,655 (0.000) INSERT INTO "persona" ("name", "description", "last_change") VALUES ('Mother Middle /Tester3/', NULL, '2019-01-28 22:43:37'); args=['Mother Middle /Tester3/', None, '2019-01-28 22:43:37']
2019-02-08 23:17:22,658 (0.001) INSERT INTO "source" ("higher_source_id", "subject_place_id", "jurisdiction_place_id", "researcher_id", "subject_date", "subject_date_sort", "medium", "title", "abbrev", "biblio", "comments", "last_change") VALUES (NULL, NULL, NULL, 1, NULL, NULL, NULL, '"test_short.ged", , exported from unknown, created on None, imported on 2019-02-08 23:17:22 CET', '"test_short.ged",
, exported from unknown, created on None, imported on 2019-02-08 23:17:22 CET', '"test_short.ged", , exported from unknown, created on None, imported on 2019-02-08 23:17:22 CET', NULL, NULL); args=[None, None, None, 1, None, None, None, '"test_short.ged", , exported from unknown, created on None, imported on 2019-02-08 23:17:22 CET', '"test_short.ged", , exported from unknown, created on None, imported on 2019-02-08 23:17:22 CET', '"test_short.ged", , exported from unknown, created on None, imported on 2019-02-08 23:17:22 CET', None, None]
2019-02-08 23:17:22,661 parse Unexpected Exception during parsing
changeling commented 5 years ago

Without digging in too deeply, the following changes got past these errors:

diff -r ./backend/geneaprove/importers/gedcomimport.py /Users/clarson/Desktop/geneapro-master-edited/backend/geneaprove/importers/gedcomimport.py
179c179
<             last_change=date)
---
>             last_change=self._process_CHAN(date))
235c235
<             name=name, description=None, last_change=chan)
---
>             name=name, description=None, last_change=self._process_CHAN(chan))

I haven't done a full trace of these changes, so beware.

briot commented 5 years ago

Without digging in too deeply, the following changes got past these errors:

diff -r ./backend/geneaprove/importers/gedcomimport.py /Users/clarson/Desktop/geneapro-master-edited/backend/geneaprove/importers/gedcomimport.py 179c179 < last_change=date)

        last_change=self._process_CHAN(date))

235c235 < name=name, description=None, last_change=chan)

        name=name, description=None, last_change=self._process_CHAN(chan))

I haven't done a full trace of these changes, so beware.

I have fixed that issue, though a bit differently. Your patch ended up potentially calling _process_CHAN several times. I also added an automatic testsuite with your test gedcom file. It seems anonymous enough that it should not be an issue, but please let me know if you’d rather not have it there.

Emmanuel

changeling commented 5 years ago

Great! And no, please do, I anonymized that file with that thought in mind.

On Mon, Feb 11, 2019 at 2:17 AM Emmanuel Briot notifications@github.com wrote:

Without digging in too deeply, the following changes got past these errors:

diff -r ./backend/geneaprove/importers/gedcomimport.py /Users/clarson/Desktop/geneapro-master-edited/backend/geneaprove/importers/gedcomimport.py 179c179 < last_change=date)

last_change=self._process_CHAN(date)) 235c235 < name=name, description=None, last_change=chan)

name=name, description=None, last_change=self._process_CHAN(chan))

I haven't done a full trace of these changes, so beware.

I have fixed that issue, though a bit differently. Your patch ended up potentially calling _process_CHAN several times. I also added an automatic testsuite with your test gedcom file. It seems anonymous enough that it should not be an issue, but please let me know if you’d rather not have it there.

Emmanuel

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/briot/geneapro/issues/38#issuecomment-462246581, or mute the thread https://github.com/notifications/unsubscribe-auth/AAYd_jgKdYfxmIFoz-VMcBL1Qz4fH570ks5vMScngaJpZM4ax3tj .