Importing from TMG?

[MichaelErichsen]

There will be stuff (e.g. User-defined) that cannot be imprted into HRE. Do we clean up before or after importing? A reporting function to run to tell what would not be imported? Switches on the import to decide what to ignore?

[richard-damon]

On Aug 3, 2018, at 12:33 PM, Michael Erichsen notifications@github.com wrote:

There will be stuff (e.g. User-defined) that cannot be imprted into HRE. Do we clean up before or after importing? A reporting function to run to tell what would not be imported? Switches on the import to decide what to ignore?

I’m curious what you think HRE will not be able to import? I thought one of the goals was to be able to import everything.

I suppose some preferences don’t apply (like which version of the pick list to use), but I would think any data should be able to reasonably make it in.

[HREferg]

This is something we’re not sure about, as it depends on how ‘badly’ people have warped TMG, particularly in the area of pseudo-persons.

The target has been set to import everything – whether it goes into the best places is another point entirely.

We are also aware that some things should not be moved e.g. many TMG Timelines were proprietary and copyright, so we cannot legally use them.

However, it seems logical that cleanup can only be done post-import (in terms of moving things to the ‘right’ place)– but whether re-assigning pseudo-people correctly (as, say, bits of land, or ships, or whatever) can be done by some internal HRE process is at this stage a premature question.

Don

[richard-damon]

On 8/3/18 9:33 PM, HREferg wrote:

This is something we’re not sure about, as it depends on how ‘badly’ people have warped TMG, particularly in the area of pseudo-persons.

The target has been set to import everything – whether it goes into the best places is another point entirely.

We are also aware that some things should not be moved e.g. many TMG Timelines were proprietary and copyright, so we cannot legally use them.

However, it seems logical that cleanup can only be done post-import (in terms of moving things to the ‘right’ place)– but whether re-assigning pseudo-people correctly (as, say, bits of land, or ships, or whatever) can be done by some internal HRE process is at this stage a premature question.

Don That sounds like a more reasonable situation. My guess is that HRE will want some form of bulk update scripting actions (sort of like what people us TMGU for) to do things, and while there may be advantages to defining rules for how to classify different types of 'People' on import, having the post import scripting can cover the need and is wanted for other purposes.

For my own database, as long as the TMG Flag values get preserved in a way that I can run a filter on them, and based on that filter result change the focus type of those 'people' (from like a Person to a Ship) and then (or prior) change certain relationship types to some better type of relationship.

-- Richard Damon

[RobinLamacraft]

Hi all,

My apologies for other reasons I have been off-line all yesterday. I had 36 HRE messages to examine this morning.

I will try to address as many as possible of them today.

As far as import to from TMG is concerned, the best method of importing that I can imagine is one where (1) all TMG data is copied into some known predetermined (possibly temporary) location within the HRE schema (2) then some user selected series of scripts are run (like TMGU) to move or manipulate that data to be stored in a HRE definitive way.

This 2 stage approach will enable us to continue to refine the importing process by adjusting the scripts and not requiring new versions of HRE to be installed. The aim should be that all data that was in TMG that was consistently entered can be adjusted without the the user needing to examine each record in turn. (A similar approach can be used for GEDCOM importing.)

Dealing with different uses of pseudo-people and some user's deliberate use of some data fields for a collection of non-equivalent values will always be a problem. This later problem main applies to Place and Person Name Styles. It also occurs to some degree in the definition and use of Source Elements. In the case of pseudo-persons the process to convert then to a separate class of object will need to be a process that is adjusted by the user. Some users have embellished to the name with identifying name elements that may need removing, etc.

I know of no recognized issue related to sentence constructs. HRE incorporates equivalents for all TMG sentence and output templates (and in many cases providing more power). Dates will be transcribed to have the same meaning with more available precision. A minor adjustment from the TMG definitions of BEFORE and AFTER which are actually are AT or BEFORE and AT or AFTER a date  to be come BEFORE or AFTER a date.

There are some TMG sentence adjustment rules and automatic updating of flags, etc what we will need to address once we understand what their implications are in the HRE environment. Somethings we will have mark as values that have been imported and we don't know what that each user's conventions were - sureties are one case - so imported TMG sureties will be marked as such. The HRE surety is modeled on the US Genealogical Proof Standard.

I hope that clarifies some of the previous discussions.

Robin

On 04-Aug-18 12:13 PM, Richard Damon wrote:

On 8/3/18 9:33 PM, HREferg wrote:

This is something we’re not sure about, as it depends on how ‘badly’ people have warped TMG, particularly in the area of pseudo-persons.

The target has been set to import everything – whether it goes into the best places is another point entirely.

We are also aware that some things should not be moved e.g. many TMG Timelines were proprietary and copyright, so we cannot legally use them.

However, it seems logical that cleanup can only be done post-import (in terms of moving things to the ‘right’ place)– but whether re-assigning pseudo-people correctly (as, say, bits of land, or ships, or whatever) can be done by some internal HRE process is at this stage a premature question.

Don That sounds like a more reasonable situation. My guess is that HRE will want some form of bulk update scripting actions (sort of like what people us TMGU for) to do things, and while there may be advantages to defining rules for how to classify different types of 'People' on import, having the post import scripting can cover the need and is wanted for other purposes.

For my own database, as long as the TMG Flag values get preserved in a way that I can run a filter on them, and based on that filter result change the focus type of those 'people' (from like a Person to a Ship) and then (or prior) change certain relationship types to some better type of relationship.

-- Richard Damon

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/History-Research-Environment/HRE--History-Research-Environment/issues/67#issuecomment-410417077, or mute the thread https://github.com/notifications/unsubscribe-auth/AVeLtKkUGKhp00CKqVpMvinvc029iPHxks5uNQo2gaJpZM4VuSOJ.

-- Robin Lamacraft, Adelaide, Australia

[MichaelErichsen]

I am adding issues to document what needs to be done in the implementation of HRE. I appreciate all comments and input. I have set this to milestone V0.7 now, and don't expect to have the technical details solved before then.