All individuals in gedcom skipped

[damonp59]

I've tried to import a gedcom into the Churchill demo without saving. Just as an experiment. Every gedcom I've tried so far, the import process skips every individual, but it seems that everyone and their events are imported. It's very confusing. Skipped implies that nothing was imported.

[clsturgeon]

I have been able to replicate it. In fact, it wasn't tough. It was an ill-formated if structure that would always increase the skip counter. Fixed in build 09, prerelease 2. I have updated the demo. Thank you.

[damonp59]

I tried again today and received the same message. I confirmed that I had the latest installed. It does appear that everyone has been imported, however. I'm going through now and checking for anything out of the ordinary that is not based on the gedcom itself. I've noticed a couple of things, but will not comment on those yet because I think it's a gedcom issue and not an import issue. I'll t;ry some other gedcoms from different sources to see if I notice the same things.

[myfta]

Only a guess, but when I have been importing Gedcoms into family history software they can be very particular about the encoding used. So worth checking what you are exporting.

On Tue, 23 Aug 2022, 18:49 damonp59, @.***> wrote:

I tried again today and received the same message. I confirmed that I had the latest installed. It does appear that everyone has been imported, however. I'm going through now and checking for anything out of the ordinary that is not based on the gedcom itself. I've noticed a couple of things, but will not comment on those yet because I think it's a gedcom issue and not an import issue. I'll t;ry some other gedcoms from different sources to see if I notice the same things.

— Reply to this email directly, view it on GitHub https://github.com/clsturgeon/MemoryKeeper/issues/10#issuecomment-1224456857, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALIWMNRZTL6QXHM4WFYDN2LV2UFI3ANCNFSM566L6VZA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[clsturgeon]

Have you confirmed the issue with the demo (Churchill) file? I have tried a few GEDCOM files with the demo without an issue. I assume you re-downloaded the json file from the demo solution, to get 0.0.09 prerelease 2, for your own solution?

[dixonge]

I tried importing the JSON of tiddler plugins into my existing copy, and I also tried just downloading the demo. Both resulted in 0.0.09 prelease 2.

In both I imported a 100-person sub-tree of my direct ancestors. I also tried importing an individual. In no case did any people or events or anything import. Zilch, nada.

I have inspected the GEDCOM and it is full of normal-looking valid GEDCOM data.

[myfta]

I tried a Gedcom of mine and imported it into the demo Churchill. There did not seem to be any errors reported (or seen). I tried the @Review and @Search options too. I would suggest that when you do an import the individuals imported are listed in the log output, particularly when it is a partial import.

I think the @Searchwould be better if the Surname and Given name fields were linked at an AND rather than what they appear to be, an OR between the two fields, which is a little confusing.

Did I miss it, can you save an empty copy of MK 0.0.09 prerelease 02?

[dixonge]

I tried a Gedcom of mine and imported it into the demo Churchill. There did not seem to be any errors reported (or seen). I tried the @ Review and @Search options too. I would suggest that when you do an import the individuals imported are listed in the log output, particularly when it is a partial import.

Just tried @review separately - this is the result:

`Log Results

GEDCOM review complete Individuals: 0 Events: 0 Notes: 0 Sources: 0 Duplicate Individuals: 0`

[myfta]

Just tried @review separately

Are you using the Churchill demo? Or do you have a different version?

Where was your Gedcom generated?

[myfta]

What does this report for your Gedcom? http://ged-inline.elasticbeanstalk.com/validate

Mine gives:

Generated by                   FAMILY_HISTORIAN
Submitted by                   Calvin Ingram
Encoding                       UTF-8
GEDCOM version in file         5.5.1
GEDCOM version assumed         5.5.1

Analysis time                  3 seconds to analyse the file (excluding upload time)
Speed                          745 records per second

Lines                   37292  Number of lines in the GEDCOM file
Records                  2896  Number of records
Warnings                   24  Total number of warning messages
User-defined             2527  Number of lines with user-defined tags

Individuals               176  Number of individuals in the GEDCOM file
Males                      87  Number of males
Females                    89  Number of females
Other                       0  

Families                   80  Number of families
Marriages                  58  Number of marriages
Places                    773  Number of places mentioned (not necessarily unique)
Source records           1616  Number of source records

There are a few structural errors, orphans etc. but the file still works for me.

[dixonge]

I get this:

`Validation report for Glenn-Direct-Ancestors.ged Generated by SyniumFamilyTree Submitted by Unknown Encoding UTF-8 GEDCOM version in file 5.5.1 GEDCOM version assumed 5.5.1

Analysis time 0 seconds to analyse the file (excluding upload time) Speed 3518 records per second

Lines 16670 Number of lines in the GEDCOM file Records 584 Number of records Warnings 28 Total number of warning messages User-defined 9183 Number of lines with user-defined tags

Individuals 101 Number of individuals in the GEDCOM file Males 52 Number of males Females 49 Number of females Other 0

Families 52 Number of families Marriages 33 Number of marriages Places 307 Number of places mentioned (not necessarily unique) Source records 11 Number of source records

              File does not end with a TRLR record

Line 13: Mandatory tag NAME not found under SUBM Line 39: Tag SECG is not allowed under NAME Line 43: Invalid content for DATE tag: '06/24/1873' is not a valid Line 46: Tag CHAN is not allowed under BIRT Line 50: Invalid content for DATE tag: '11/09/1966' is not a valid Line 53: Tag CHAN is not allowed under DEAT Line 74: Invalid content for DATE tag: '05/19/1879' is not a valid Line 88: Tag CHAN is not allowed under BIRT Line 131: Invalid content for DATE tag: '10/08/1954' is not a valid Line 145: Tag CHAN is not allowed under DEAT Line 184: Invalid content for DATE tag: '7/2/1825' is not a valid Line 220: Invalid content for DATE tag: '4/2/1865' is not a valid Line 268: Unknown XREF type for tag LABL Line 430: Unknown XREF type for tag LABL Line 453: Unknown XREF type for tag LABL Line 971: Unknown XREF type for tag LABL Line 1291: Unknown XREF type for tag LABL Line 14411: Line is more than 255 characters Line 14454: Line is more than 255 characters Line 14455: Line is more than 255 characters Line 14467: Line is more than 255 characters Line 14480: Line is more than 255 characters Line 15989: Line contains illegal character(s) Line 16532: Line contains illegal character(s) Line 16533: Line contains illegal character(s) Line 16637: Line contains illegal character(s) *** Line 16639: Line contains illegal character(s)

Report generated on 13-09-2022 at 19:04 by GED-inline 3.1.0`

[clsturgeon]

I have just got back from a genealogical vacation (aka a dead ancestors tour). I will review (likely tomorrow--I'm still on vacation) the warnings and see how this may impact the import. It does appear it is skipping all the records. Likely a unexpected tag/token.

In the meantime please try other gedcom files. Some time ago I was able to get these to load... gedcom torture test files.

http://www.geditcom.com/gedcom.html

[clsturgeon]

I have made two minor changes to the GEDCOM process., 1. when you do a REVIEW it will now report how many lines were read. I’m wondering if the import was reading your file as a single line. To that end, the other change I made was to support various end-of-line encodings. This is included 09 pre-release-03. You can download the json file here:

craigsturgeon.com/mk/memorykeeper_0.0.09-prerelease-03.json

[dixonge]

I have just got back from a genealogical vacation (aka a dead ancestors tour). I will review (likely tomorrow--I'm still on vacation) the warnings and see how this may impact the import. It does appear it is skipping all the records. Likely a unexpected tag/token.

In the meantime please try other gedcom files. Some time ago I was able to get these to load... gedcom torture test files.

http://www.geditcom.com/gedcom.html

Just tested one - worked fine! Line count showed, etc.

As I mentioned on the TW forum, my own GEDCOM extract file did not process any better, even w/ the 03 update

[myfta]

I have made two minor changes to the GEDCOM process., 1. when you do a REVIEW it will now report how many lines were read. I’m wondering if the import was reading your file as a single line. To that end, the other change I made was to support various end-of-line encodings. This is included 09 pre-release-03. You can download the json file here:

craigsturgeon.com/mk/memorykeeper_0.0.09-prerelease-03.json

I could not work out how to use the JSON file correctly. When I import it into an empty TW file, it creates an incomplete MK wiki.

So I reverted to testing Gedcom imports into the Churchill demo wiki, which #i realise is version 02. What I am finding is it is not creating the Places and Occupation tiddlers, there may be others. Has anyone else had this problem? Do we know how to fix it?

[clsturgeon]

You import the json file into an existing Memory Keeper file (not an empty TW). The json can be imported into an empty MK or one that is populated. In either case this upgrades the MK project.

Be sure to save and reload file afterwards. Any changes to JavaScript macros does not take affect until it is reloaded.

Occupation and place fields should be populated but you are correct it does not import them.

Eg. A specific event would have its place field populated, but the place Tiddler is missing.

I'll have another look at this. Place values in gedcom are typically inconsistent. Eg New York City is enter various ways across various records. These are examples...

New York, New York, USA New York City, USA Manhattan, USA New York City, New York, USA

If these get imported the user will have multiple tiddlers for the same place.

This would be a huge clean up process for user. However, I guess not importing them is just as bad.

Also place tiddlers will be incomplete. ie the gedcom will not provide the type of place.

Your thoughts?

[myfta]

I created a couple of place and occupation tiddlers. I would rather a cleanup process than having to create them all. I could envisage a "merge two tiddlers" function, but don't know how you would write it.

[clsturgeon]

As I mentioned on the TW forum, my own GEDCOM extract file did not process any better, even w/ the 03 update

I'll need a copy of this Gedcom file. If not, can you generate another I can test with. What software was used to generate this gedcom file?

[myfta]

Occupation and place fields should be populated but you are correct it does not import them.

These Occupation and Place fields have an odd behaviour. You cannot see them in the parent Tiddler but if you get the name precisely correct they magically appear. There needs to be a way of creating them at the time of import.

[dixonge]

As I mentioned on the TW forum, my own GEDCOM extract file did not process any better, even w/ the 03 update

I'll need a copy of this Gedcom file. If not, can you generate another I can test with. What software was used to generate this gedcom file?

Glenn-Dixon-Ancestors

Try that? Generated by MacFamilyTree

[clsturgeon]

Thank you. Your file only has CR (carriage-return character to end each line). I was able to replicate the issue. I had coded it to support CR and LF (linefeed) or just LF. I have corrected it. This will be fixed in 09 pre-release 04. I will generate the json file tonight. This build will also import occupations and places (requested by Graham)--not to mention a few more bug fixes not related to the GEDCOM import. The import for this file resulted in:

Individuals: 101 Events: 250 Notes: 47 Sources: 9 Occupations: 1 Places: 32

[clsturgeon]

Download, import, save and reload this json file: [http://craigsturgeon.com/mk/memorykeeper_0.0.09-prerelease-04.json]()

This build corrects gedcom import issue, imports occupations and places, plus a few bug fixes.

[dixonge]

Yep, that does it alright. Upgraded, imported, bingo.

[myfta]

I noticed all the dates are imported with a hyphen prefix. Why? e.g. -1895-06-02 - Marriage -

[myfta]

The Places and Occupation import is good.

I wonder if Causes of Death, Occupations and Roles should be moved under People?

[clsturgeon]

I noticed all the dates are imported with a hyphen prefix. Why? e.g. -1895-06-02 - Marriage -

Can you provide a GEDCOM example or snippet of the GEDCOM file with this data for this example?

[clsturgeon]

I wonder if Causes of Death, Occupations and Roles should be moved under People?

Tiddlers under People are reserved for "Family Member" tiddlers. I think these belong somewhere else too!

[clsturgeon]

FYI... More bugs are fixed, plus a change in an attempt to address the dash in dates. Sources and places in events were not imported as expected.

http://craigsturgeon.com/mk/memorykeeper_0.0.09-prerelease-05.json

[myfta]

I could not work out how to use the JSON file correctly. When I import it into an empty TW file, it creates an incomplete MK wiki.

So I reverted to testing Gedcom imports into the Churchill demo wiki, which

i realise is version 02. What I am finding is it is not creating the

Places and Occupation tiddlers, there may be others. Has anyone else had this problem? Do we know how to fix it?

On Wed, 14 Sept 2022 at 19:10, clsturgeon @.***> wrote:

I have made two minor changes to the GEDCOM process., 1. when you do a REVIEW it will now report how many lines were read. I’m wondering if the import was reading your file as a single line. To that end, the other change I made was to support various end-of-line encodings. This is included 09 pre-release-03. You can download the json file here:

craigsturgeon.com/mk/memorykeeper_0.0.09-prerelease-03.json

— Reply to this email directly, view it on GitHub https://github.com/clsturgeon/MemoryKeeper/issues/10#issuecomment-1247131872, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALIWMNRXPFFCAHGHNSWQUOTV6IIH5ANCNFSM566L6VZA . You are receiving this because you commented.Message ID: @.***>

[myfta]

So, it looks like Synium family tree is not producing a correctly formatted Gedcom. Without seeing the actual raw data it does look like it is not using the correct end of line, CRLF, but is also using a large number of custom tags. It might be worth seeing if there are any export options you can specify.

On Tue, 13 Sep 2022, 20:06 Glenn Dixon, @.***> wrote:

I get this:

`Validation report for Glenn-Direct-Ancestors.ged Generated by SyniumFamilyTree Submitted by Unknown Encoding UTF-8 GEDCOM version in file 5.5.1 GEDCOM version assumed 5.5.1

Analysis time 0 seconds to analyse the file (excluding upload time) Speed 3518 records per second

Lines 16670 Number of lines in the GEDCOM file Records 584 Number of records Warnings 28 Total number of warning messages User-defined 9183 Number of lines with user-defined tags

Individuals 101 Number of individuals in the GEDCOM file Males 52 Number of males Females 49 Number of females Other 0

Families 52 Number of families Marriages 33 Number of marriages Places 307 Number of places mentioned (not necessarily unique) Source records 11 Number of source records

          File does not end with a TRLR record

Line 13: Mandatory tag NAME not found under SUBM Line 39: Tag SECG is not allowed under NAME *** Line 43: Invalid content for DATE tag: '06/24/1873' is not a valid

*** Line 46: Tag CHAN is not allowed under BIRT *** Line 50: Invalid content for DATE tag: '11/09/1966' is not a valid *** Line 53: Tag CHAN is not allowed under DEAT *** Line 74: Invalid content for DATE tag: '05/19/1879' is not a valid *** Line 88: Tag CHAN is not allowed under BIRT *** Line 131: Invalid content for DATE tag: '10/08/1954' is not a valid *** Line 145: Tag CHAN is not allowed under DEAT *** Line 184: Invalid content for DATE tag: '7/2/1825' is not a valid *** Line 220: Invalid content for DATE tag: '4/2/1865' is not a valid *** Line 268: Unknown XREF type for tag LABL *** Line 430: Unknown XREF type for tag LABL *** Line 453: Unknown XREF type for tag LABL *** Line 971: Unknown XREF type for tag LABL *** Line 1291: Unknown XREF type for tag LABL *** Line 14411: Line is more than 255 characters *** Line 14454: Line is more than 255 characters *** Line 14455: Line is more than 255 characters *** Line 14467: Line is more than 255 characters *** Line 14480: Line is more than 255 characters *** Line 15989: Line contains illegal character(s) *** Line 16532: Line contains illegal character(s) *** Line 16533: Line contains illegal character(s) *** Line 16637: Line contains illegal character(s) *** Line 16639: Line contains illegal character(s) Report generated on 13-09-2022 at 19:04 by GED-inline 3.1.0` — Reply to this email directly, view it on GitHub , or unsubscribe . You are receiving this because you commented.Message ID: ***@***.***>

[clsturgeon]

I'm not sure exactly what you trying to do. However, I'll make a few comments. Build 9 is now out, which replaces all build 9 prereleases. Build 9 can be downloaded as an empty MK HTML file or a json file. The json file is used to upgrade a previous release of MK. There is no need to apply this json file to an empty HTML file that is already at build 9.

There are numerous fixes since build 9 prerelease 2. Also, over the weekend I upgraded the Churchill demo project to build 9.

Another important bit. This may have caused an issue for you. I moved all the files, MK help/download pages and the Churchill project to GitHub.

Be sure to use this url. This a new url that MK now uses when the user clicks the help icon in MK.

https://clsturgeon.github.io/MemoryKeeper/

You will find, in the new help, a new url to the Churchill demo too.

https://clsturgeon.github.io/MemoryKeeper/Memory%20Keeper%20-%20Demo%20-%20Churchill/Churchill-Example.html

Hope this helps.