Preservation of Global IDs

[MikeMillerGIS]

In testing, we found that you can preserve global ids using the (arcpy.Copy)[http://pro.arcgis.com/en/pro-app/tool-reference/data-management/copy.htm]. The downside of using this, I do not think it can filter data prior to copy, it would have to be deleted after.

We are still pushing the dev team to support the preserve global id on FGDB-> FGDB

[MikeMillerGIS]

Talking with dev again, they are not going to expose this, so the only valid workflow is EGDB -> EGDB Needs verification

• [ ] EGDB -> EGDB

[previnWong]

1) I copied a FGDB FC into an EGDB 2) I added Global ID to the FC in the EGDB 3) I mapped EGDB FC to another EGDB FC 4) In the enivornment setting, I checked "preserve global ids" 5) ran the append tool, saw this message

6) after loading completed, selected features on the map, they don't have matching global ids

[MikeMillerGIS]

I get the same thing. It worked on a simple polygon layer outside a feature dataset. But when I used Lyrx file files, both from the toc and disk point to layers in a feature dataset, I get the above issue.

[MikeMillerGIS]

Trying the feature classes themselves, I get the following error: Spatial References do not match, unable to preserve GlobalIDs

Here is the target:

here is the source:

[MikeMillerGIS]

full log: Running script AppendData... Appending Data Configuration file: C:_MyFiles\GitHub\utility-data-models\SourceData\Water\SourceTarget2.xml SourceData\Owner@SourceData.sde\SourceWater.UNADMIN.WaterDistribution\SourceWater.UNADMIN.wCasing TargetData\Owner@TargetData.sde\WaterFromAP.UNADMIN.WaterFromAP\WaterFromAP.UNADMIN.StructureLine Exporting Data from SourceData\Owner@SourceData.sde\SourceWater.UNADMIN.WaterDistribution\SourceWater.UNADMIN.wCasing WCASING_View View Created 18 source rows Source and Target datasets both have GlobalID fields Spatial References do not match, unable to preserve GlobalIDs Removing Default Value property from intermediate database fields GlobalID removed from fieldMappings SHAPE_Length removed from fieldMappings 18 source rows exported to STRUCTURELINE Field Calculator: C:_MyFiles\GitHub\utility-data-models\SourceData\Water\SourceTarget2.xml Field added: ASSETGROUP Field added: ASSETTYPE Field added: CONTAINMENTSTATUS Field added: SUBNETWORKNAME Field added: CREATIONDATE Field added: CREATOR Field added: UPDATEDBY Field altered: MATERIAL to material Field added: measuredlength Field added: assetid C:\Users\mich3661\AppData\Local\ESRI\ArcGISPro\AssemblyCache{28142961-b645-420f-ba2a-72bcf8212558}\GPTools\arcpy\dla.gdb\STRUCTURELINE, 18 features Appending STRUCTURELINE TO STRUCTURELINE Rows appended 18 features in source dataset 18 features in target dataset Start Time: Tuesday, March 21, 2017 9:42:49 PM Row Count = 18 Succeeded at Tuesday, March 21, 2017 9:42:49 PM (Elapsed Time: 0.01 seconds) Data Assistant - Append Data Process completed with 0 errors Completed script AppendData...

[MikeMillerGIS]

Ran is again on a new source-target file, same SDE source, Same SDE target, different layers - same issue:

Running script AppendData... Appending Data Configuration file: C:_MyFiles\GitHub\utility-data-models\SourceData\Water\MainsToLines.xml SourceData\Mains_defQuery.lyrx TargetData\Lines.lyrx Exporting Data from SourceData\Mains_defQuery.lyrx WMAIN_View View Created 1 source rows Source and Target datasets both have GlobalID fields Source Workspace type prevents preserving GlobalIDs Removing Default Value property from intermediate database fields GlobalID removed from fieldMappings Shape_Length removed from fieldMappings 1 source rows exported to WATERFROMAP

[MikeMillerGIS]

Edit: not it

[MikeMillerGIS]

I manually fix the spatial reference in the xml files to be the same and I did not get the error saying the layer types do not support preserving Global ID. So maybe this issue is related to the spat ref.

[MikeMillerGIS]

well, I was wrong.

Added some error messages and instead of CIM workspace, I actually get a path: source.dataSource = C:\Users\mich3661\Documents\ArcGIS\Projects\UtilityNetworkDataModels\UNAdmin@SQL_SERVER@WaterFromAP.sde\WaterFromAP.UNADMIN.Water\WaterFromAP.UNADMIN.WaterDevice

[MikeMillerGIS]

@SteveGrise here is the fix for my issue where a path is being returned.

def get_geodatabase_path(input_table):
    '''Return the Geodatabase path from the input table or feature class.
    :param input_table: path to the input table or feature class
    '''
    workspace = os.path.dirname(input_table)
    if [any(ext) for ext in ('.gdb', '.mdb', '.sde') if ext in os.path.splitext(workspace)]:
        return workspace
    else:
        return os.path.dirname(workspace)
def checkDatabaseType(path):
    # check database types - SQL source db and SQL gdb as target
    supported = False
    try:
        if path.lower().startswith(_http):
            supported = True
        elif path.lower().count(_sde) == 1:
            supported = True
        elif path.lower().endswith(_lyrx):
            source = getLayerFromString(path)
            if source.dataSource.startswith(_CIMWKSP):
                supported = True
            elif arcpy.Exists(source.dataSource):
                path = get_geodatabase_path(input_table=source.dataSource)
                d = arcpy.Describe(path)
                if d.workspaceType == "RemoteDatabase":
                    supported = True

[MikeMillerGIS]

@SteveGrise do you recall if a unique index on Global ID is required? For some reason, my index of global id is not set to unique

causing this error Source Dataset does not have a unique index on GlobalID field, unable to preserve GlobalIDs

[MikeMillerGIS]

according to this, it does, why the heck would the default not be unique http://pro.arcgis.com/en/pro-app/tool-reference/environment-settings/preserve-globalids.htm

[SteveGrise]

I am not sure, I think it happens when data is copied from File GDB to SQL databases.

[previnWong]

Add note to recreate index post process to the wiki. https://github.com/Esri/data-assistant/wiki/Recreate-GlobalID-index-after-running-tools

closing since we don't want to manipulate the user's data or schema.

[SimmonsY]

Apparently the nonUnique Indexing is to support versioning, I can't explain why, refer to link.

https://geonet.esri.com/ideas/13150-preserve-globalid-on-append