Inconsistent Census Handling

[ejmyr]

Testing a tiny GEDCOM file with four individuals: father, mother, daughter, granddaughter. A 1900 census fact is shared among all four individuals, per RootsMagic 8. First, between the father and mother as a census (family) fact:

Then shared to the daughter and granddaughter:

FTA loads the file and recognizes only three census facts:

For some reason, the granddaughter is loaded, but the shared fact is ignored. This is the 'Show Not Found on US Federal Census 1900' popup window:

And this is the 'Colour Census Report Result' window:

Am I wrong in thinking that the cell for the granddaughter in the 1900 US Census column should be green? I can't figure out what's going on here, but it only seems to happen in my file when the census entry has relatives other than parents and children. I'm not familiar enough with GEDCOM to troubleshoot, other than to see that the fact is shared among all four individuals. Is this a RootsMagic quirk or something in the way FTA analyzes files?

The Colour Census Report (which also doesn't filter properly for me) is the only blemish on this otherwise outstanding program. Thank you for providing such a useful tool.

E

[ShammyLevva]

We would need to check the actual GEDCOM produced. I suspect that a shared family fact is only tagged against a family. In GEDCOM terms a family is strictly made up of up to two parents plus children. A grandchild would be a different family unit. However without seeing the GEDCOM produced it's difficult to tell if the issue is in how the GEDCOM was created vs how FTAnalyzer is dealing with the data.

[ejmyr]

Thanks for the very quick response! You're probably right about the family unit thing, although it's weird that it'd appear seemingly correct in the file (and RM). Maybe it's something specific to RM.

Once I started noticing this on my main tree, I isolated a single census reference that showcased the potential issue and exported it to a new tree. I then cleared out everything not related to that census fact and loaded it in FTA (the screenshots above). See below for the stripped-down GEDCOM. The individual in question is on line 48 and the share is on line 68. Here you go:

rmExportTest.txt

[ShammyLevva]

I've had a look at the GEDCOM you kindly supplied and can see that it is clear the CENS fact (line 62) is attached to family 1 (0 @F1@ FAM on line 58).

Individual 4 - the granddaughter has no census fact of her own and her family record family 2 (0 @F2@ FAM line 89 has no CENS record either.

So sadly because the CENS record doesn't appear in the GEDCOM for her then FTAnalyzer cannot determine she has been found in the census. This means the issue lies in the GEDCOM generation.

Interestingly there is an additional tag I've not seen before at lines 66-69. These are attached to the shared CENS record that starts at line 62 as is evidenced by the 2 at the start of the line indicating it's a sub part of the item at line 62 that started with a 1.

The lines 66-69 are

2 _SHAR @I3@
3 ROLE Witness
2 _SHAR @I4@
3 ROLE Witness

The _SHAR perhaps is trying to indicate a shared fact? and the @I3@ and @I4@ suggests a shared fact with individuals 3 & 4 although there's a number of problems with this. First Individual 3 is a child of that family so is already included in the shared fact. The _SHAR is followed by a 3 ROLE Witness tag which suggests they witnessed the original census event.

A bit confusing as SHAR is not a standard GEDCOM tag (the indicates this) so it's not at all clear what it's trying to indicate. Are you aware of any documentation on this or a RM user group that might explain the _SHAR & ROLE tags? I've never used Roots Magic myself so a bit in the dark.

[ejmyr]

The custom _SHAR tag must be what's causing the discrepancy between RootsMagic and FTA. When a census fact is created in RM, it's attached to either an individual or a couple and may then be shared with any other individuals that appear in the census. The custom tag must be what the program uses to display the fact on the other individuals.

I'm not familiar with any RootsMagic docs regarding the custom tag and there doesn't seem to be much discussion of it here: https://community.rootsmagic.com/ although there seems to be a consensus that shared facts are poorly implemented. It looks like the 'solution' is to go through the database and replace the shared facts with individual ones, which breaks enough other functionality that I'm not going to go that route.

Thanks for looking into this anyway.

E

On Sat, Feb 26, 2022 at 6:22 PM Alexander Bisset @.***> wrote:

I've had a look at the GEDCOM you kindly supplied and can see that it is clear the CENS fact (line 62) is attached to family 1 (0 @F1 https://github.com/F1@ FAM on line 58).

Individual 4 - the granddaughter has no census fact of her own and her family record family 2 (0 @F2 https://github.com/F2@ FAM line 89 has no CENS record either.

So sadly because the CENS record doesn't appear in the GEDCOM for her then FTAnalyzer cannot determine she has been found in the census. This means the issue lies in the GEDCOM generation.

Interestingly there is an additional tag I've not seen before at lines 66-69. These are attached to the shared CENS record that starts at line 62 as is evidenced by the 2 at the start of the line indicating it's a sub part of the item at line 62 that started with a 1.

The lines 66-69 are 2 _SHAR @I3@ 3 ROLE Witness 2 _SHAR @I4@ 3 ROLE Witness The _SHAR perhaps is trying to indicate a shared fact? and the @i3 https://github.com/i3@ and @i4 https://github.com/i4@ suggests a shared fact with individuals 3 & 4 although there's a number of problems with this. First Individual 3 is a child of that family so is already included in the shared fact. The _SHAR is followed by a 3 ROLE Witness tag which suggests they witnessed the original census event.

A bit confusing as SHAR is not a standard GEDCOM tag (the indicates this) so it's not at all clear what it's trying to indicate. Are you aware of any documentation on this or a RM user group that might explain the _SHAR & ROLE tags? I've never used Roots Magic myself so a bit in the dark.

— Reply to this email directly, view it on GitHub https://github.com/ShammyLevva/FTAnalyzer/issues/262#issuecomment-1052776354, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEBFP5QHRMUWVU6SEIRDITTU5FOBZANCNFSM5PJHX6IA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

-- "Hope will never be silent." --- Harvey Milk

[ejmyr]

Hi again,

I downloaded a copy of Legacy Family Tree 9.0 and created a new tree with the same four individuals. I created the same 1900 census fact on the father, then shared the same fact between the wife, daughter, and granddaughter. When I went to export the tree to GEDCOM, Legacy offered the option to convert shared facts to standard facts. So, I exported both with and without the box checked:

legacyWithShared.txt legacyNoShared.txt

Both the shared and standard Legacy files produced the expected 4 census facts in FTA:

Just to see what would happen, I exported my main DB from RootsMagic to Legacy, then re-exported from Legacy to a standard GEDCOM with the shared box checked and loaded the result in FTA. The individuals with shared census facts no longer appear on the 'Show Not Fount on Census' list. So there's a workaround at least.

Looks like the issue is probably the way RootsMagic exports shared facts to GEDCOM. I'll e-mail them with a feature request to see if they can add the option to export to GEDCOM and convert shared facts to standard facts instead.

Thanks again,

E

[ShammyLevva]

I could add support for _SHAR to add other people to the shared fact. What I would want to know though is what the ROLE Witness is about. Is there some tag that suggests this person was only a witness to the shared fact vs actually needing the fact added.

I am therefore wondering if there are other ROLEs that can be added, and if so what those roles are.

[ShammyLevva]

I came across this document https://jfcardinal.github.io/GEDCOM-5.5.2/extensions.html and it suggests that yes _SHAR is a custom GEDCOM but is used by various software.

Importantly though it mentions the following:

_SHAR is often accompanied by an additional _ROLE subrecord to designate the role of the co-participant. For some roles, the meaning of the event applies to the co-participant. For other roles, the meaning of the event does not apply to the co-participant; the meaning is supplied by the role name only.

So, for example, a co-participant with the role "Principal" might be used to add a co-purchaser to an event that describes a land purchase by several individuals. The meaning of the event—purchased real estate—is applied to the person underwhich the event appears and also to the co-participant.

Conversely, a co-participant with the role "Best Man" might be used to add a person to a marriage event. The meaning of the event—was married—is not applied to the co-participant.

If no _ROLE (or equivalent) is supplied, or if the _ROLE value is unrecognized, the meaning of the event should not be applied to the co-participant. The co-participant should be treated as generically interested in the event without applying a specific meaning.

It gives an example

0 @F1@ FAM 1 HUSB @I1@ 1 WIFE @I2@ 1 MARR 2 DATE 25 AUG 1945 2 _SHAR @I3@ 3 _ROLE Maid of Honor 2 _SHAR @I4@ 3 _ROLE Officiant

So you can see the two ROLE's here it would be inappropriate to add the marriage to the individuals 3 and 4 as they took part in the marriage but they weren't the ones getting married.

For a census though the ROLE Witness might not be right to add to the individual as that suggests they simply observed the census taking place not took part. To be fair Witness for a census is a somewhat bad choice of role. However it may be that given a limited number of options RootsMagic defaults to using Witness to mean apply this fact to the participant.

If that later option is the case it would be trivial for me to add support for a _SHAR fact that had a ROLE Witness. Ideally though I'd like to understand what the possible roles are and only apply a shared fact if the role was appropriate.

Are you able to do some testing to determine what ROLE options exist? If so I can easily add code to recognise _SHAR as a shared fact for the individual. I just need to be able to determine from the role whether it's appropriate to add the shared fact or not. Maybe initially all I add is the shared fact if it's a CENS fact and for no other fact type?

Thoughts?

[ShammyLevva]

Hmm just shows my memory isn't great. I looked at the code I had for shared facts and this is it.

void CheckForSharedFacts(XmlNode node)
        {
            XmlNodeList list = node.SelectNodes("_SHAR");
            foreach (XmlNode n in list)
            {
                string indref = n.Attributes["REF"]?.Value;
                string role = FamilyTree.GetText(n, "ROLE", false);
                if (indref != null && role == "Household Member")
                    FamilyTree.Instance.AddSharedFact(indref, this);
            }
        }

What that shows is that it is indeed checking for a _SHARed fact and looking at the ROLE tag However it's looking for a ROLE of "Household Member" is this a selection option in Roots Magic? If so that would solve the issue as Witness just feels wrong.

[ejmyr]

Hi again,

This is the bit about roles right from the RM wiki, if it helps: http://wiki.rootsmagic.com/wiki/RootsMagic_8:Roles

And this is from https://wiki-en.genealogy.net/GEDCOM-Tags :

Kind of weird that there's a standard ROLE tag, but not a standard SHAR tag. If there was a single set of rules for sharing facts, it'd probably make things a lot easier.

The 'Witness' ROLE is the RootsMagic (and Family Tree Maker 2017 - I just checked) default for any fact shared with another individual. In Legacy 9.0 the default is 'Household Member'. FTM is limited to only sharing predefined shared events such as Marriages or custom events. The ROLE type is set to Witness and cannot be changed, as far as I can tell. Legacy allows for almost any fact (probably every fact, I only tested a handful) to be shared, but again the ROLE cannot be changed from 'Household Member'. RM offers the most flexibility. Any fact may be shared and custom ROLEs may be defined.

Here's a RootsMagic community post regarding roles: https://community.rootsmagic.com/t/narrative-reports-shared-census-fact/3657 . There's a screenshot of a Census fact with multiple custom roles set up four posts from the top that's similar to what I'm using on my main tree. There are differences, though. I have roles of 'partner' and 'step-grandson' set up because I have people in my tree that have been enumerated as such. The number of potential roles for a Census fact is rather large and entirely user-dependent. Some, like me, are going to customize the roles to suit their tree. Others are going to use whatever default is provided by the program. It may be a good idea to discard the ROLE check for the Census fact. After all, there can be any number of people in any number of relationships enumerated on a census and it's a valid fact for all of them since they were all counted.

E

[ShammyLevva]

Hmm given the wide variety of effectively free text that a user COULD enter as a role I think it’s next to impossible to impose a standard on that. Basically any text could go in there and mean that they were intended to be included in the census.

I think therefore the question becomes are there any conceivable circumstances someone would add a shared census fact and record a role that DID NOT intend the person to get the shared fact. If we think that circumstance would not occur then it may be simpler for CENSus facts to always include the shared fact regardless of the role text.

A tricky one as obviously that invites the cases that we can’t think of just now, where it shouldn’t added to a shared census.

Gut feeling is I remove the role check and say if it’s a shared fact on a census it adds it as a fact to the individual.

I suspect all this ambiguity is why it never made in into the standard and is not present even in the brand new GEDCOM 7 standard that was adopted last year.

[ejmyr]

I'm thinking there's only two ways to reference a census sheet because there's only two ways a name will appear on it. Either as an individual (alone or as part of a family) or as the enumerator. I've had this happen a couple times. I've always used the census sheet as a source for an OCCU fact instead since they're not enumerating themselves; it's evidence they were doing the census as a job. I know others use the EVEN tag when the come across a relative as an enumerator.

I can't really come up with a situation in which a person would share a CENS fact with an enumerator or any situation in which a CENS fact would be shared among people not actually on the census sheet as part of that household. I think you're right in thinking that removing any role check for this fact type is probably the best way to handle this. It's probably also the only fact type where this would make sense. Everything else is too open to interpretation, but there's just not a whole lot of wiggle room when it comes to a census.

E

[ShammyLevva]

This should be fixed in v10.0.0.0-beta2 can you test and let me know please. It seemed to work with the test file you sent but there may be other issues on a larger tree.

[ShammyLevva]

Going to close this one as I believe it's fixed and user hasn't disputed that.