Export and Import Lookups/Record Values

[AzureWeller]

I'm currently trying to setup a new environment which uses data from the original enviroment

I have now managed to get all the solutions imported from the old instance to the new instance however I am struggling to export the results and import them in the new environment. They are not on the same domain/network.

I currently have the below:

$Conn = CONNECTIONSTRINGHERE
$i = 0

Write-Host "Getting CRM Entity Metadata"
$records = Get-CrmEntityAllMetadata -conn $conn | select LogicalName | where-object LogicalName -
like xcor_lk* | Sort-Object LogicalName
$entities = $records.logicalname

Write-Host "Beginning Export"
foreach ($Entitie in $Entities){
$results = @()
$i++

$EntitieRecords = Get-CrmRecords -conn $conn -EntityLogicalName $Entitie -AllRows -fields *

Write-Progress -Activity "Exporting Lookups..." -Status "Exporting $entitie : $i of $($entities.Count)" -PercentComplete (($i / $entities.Count) * 100) 
$EntitieRecords.CrmRecords | export-csv -path "C:\support\downloads\powershell\LKCSV\$entitie.csv"}`

This returns all the CRMRecords Data relating to that entities as expected.

I then take the files to the other machine/network and do:

$conn = OTHERCONNECTIONSTRING
$csvs = get-childitem E:\lkcsv
foreach ($CSV In $CSVs){

$ImportCSV = Import-csv $csv.Fullname

      foreach ($import in $importcsv){
      $Entityname = $import | select -expandproperty logicalname
      New-CrmRecord -conn $conn -entitylogicalname $entityname -fields@{}
      }

}

The new-crmrecord is where I am struggling as I can't quite figure how to set the fields dynamically

Any help would be appreciated

Regards

[seanmcne]

For instance create two connections - $conn1 = connect-crmonline ...; $conn2 = connect-crmonline ...

Then get your records like you are already doing, don't export them to csv though, instead:

#$EntityRecords come from $conn1, just write those records into $conn2 
#you can even preserve the record id to keep them the same 
#bear in mind, this is not the most efficient way to migrate data but for small datasets it works   
foreach($rec in $EntityRecords.CrmRecords){
    new-crmrecord -conn $conn2 -CrmRecord $rec -PreserveCrmRecordId
}

[AzureWeller]

I cannot create a 2nd connection as the new instance is not accessible from the same domain / network

[seanmcne]

If it's the same PC across two networks you can just leave powershell open, how much data are you looking to transfer?

You could potentially look at serializing the entire crmrecords array and objects within it to xml using a technique such as: https://blogs.technet.microsoft.com/heyscriptingguy/2011/09/06/learn-how-to-save-powershell-objects-for-offline-analysis/

Export-CliXml will serialize the object out to xml, and the import will deserialize it into the same object

$EntityRecords.CrmRecords | Export-Clixml foo.xml

#... 

$records = import-clixml foo.xml 

#$EntityRecords come from $conn1, just write those records into $conn2 
#you can even preserve the record id to keep them the same 
#bear in mind, this is not the most efficient way to migrate data but for small datasets it works   
foreach($rec in $records){
    new-crmrecord -conn $conn2 -CrmRecord $rec -PreserveCrmRecordId
}

[AzureWeller]

Different machines unfortunately. I should of let you know.

It's quite a large amount of data (One Entity has approx 56k records inside).

I'll have a look at the CLIXML

[seanmcne]

Might be OK for a one time import, but for data imports I tend to use an ssis adapter or product designed to move data and expose controls that matter to throughput (connection and request concurrency, data translation, validation, field mappings, etc). Another option is to use the CRM ConfigurationMigration utility but even that has some limitations. For a one off migration of some records powershell will probably work OK but it's not a great ETL / migration tool. I'll close this for now but feel free to reopen or open a new issue if you have any questions - thx!

[AzureWeller]

So I tried the Import-CLI method suggested however this error keeps throwing everytime:

The formatter threw an exception while trying to deserialize the message: There was an error while trying to deserialize parameter http://schemas.microsoft.com/xrm/2011/Contracts/Services:request. The InnerException message was 'Error in line 1 position 922. Element 'http://schemas.datacontract.org/2004/07/System.Collections.Generic:value' contains data from a type that maps to the name 'System.Management.Automation:PSObject'. The deserializer has no knowledge of any type that maps to this name. Consider changing the implementation of the ResolveName method on your DataContractResolver to return a non-null value for name 'PSObject' and namespace 'System.Management.Automation'.'. Please see InnerException for more details.