Export Config Migration Data - Export Failed

[CF1tz]

We seem to be getting a generic Export Failed error when running the MSCRM Export Config Migration Data step. I can't work out what is failing and have tried to change various settings but keep getting the same error. Are you able to shed any light on it?

[debug]+ Export-CrmDataFile @exportParams -Verbose

[debug]+ ~~~~~~~~~

[debug] + CategoryInfo : InvalidResult: (:) [Export-CrmDataFile], Exception

[debug] + FullyQualifiedErrorId : 1,Microsoft.Xrm.Tooling.ConfigurationMigration.CmdLets.ExportCrmDataFile

[debug]

[debug]Script stack trace:

[debug]at , D:\a\tools\MSCRMBuildTools\9.0.23\xRMCIFramework\9.0.0\ExportCMData.ps1: line 67

[debug]at , D:\a_tasks\MSCRMExportCMData_0be5756e-887c-41dd-aa63-e52c293b2ad4\9.0.7\MSCRMExportCMData.ps1: line 57

[debug]at , : line 1

[debug]at , : line 22

[debug]at , : line 18

[debug]at , : line 1

[debug]Exception:

[debug]System.Exception: Export failed.

[debug] at System.Management.Automation.MshCommandRuntime.ThrowTerminatingError(ErrorRecord errorRecord)

[error]Export failed.

[debug]Processed: ##vso[task.logissue type=error]Export failed.

[debug]Processed: ##vso[task.complete result=Failed]

[section]Finishing: Export Config Files

[WaelHamze]

@CF1tz You could try the MS powershell libraries directly, if this doesn't work you could open a case with MS. I have updated the task to use the latest MS powershell libraries so you can give this another go.

[hero0118]

Hi i am also having this problem, i have used the Configuration Migration tool from the SDK and that seems to work fine. Just running this via dev ops seems to throw this error.

[HIREN-AITC]

I am also getting the error, I have tried to use Direct power shell which actually finishes exporting all entities and then at the end it fails. Finally, I have tried below command but the same error there also.

Export-CrmDataFile -CrmConnection $CRMConn -SchemaFile $schemaFile -DataFile $dataFile -Verbose

[WaelHamze]

@HIREN-AITC if you are getting error with task then the issue is not with the task.

I have updated the tools to the latest version of the PS module today.

Please re-open if you have issues with the task.

[ghost]

Hi, as a note, I have found that issue is with the tooling itself, when I run the command manually on my local, i get more detailed errors in the logs:

    DataMigrationUtility.Export Error   2   27/02/20 15:30:34  Message: Exception Raised during Zip
    Source  : mscorlib
    Method  : _GetAccountingInfo
    Date    : 27/02/20
    Time    : 15:30:34
    Error   : Unable to determine the identity of domain.
    Stack Trace : at System.IO.IsolatedStorage.IsolatedStorage._GetAccountingInfo(Evidence evidence, Type evidenceType, IsolatedStorageScope fAssmDomApp, Object& oNormalized)
       at System.IO.IsolatedStorage.IsolatedStorage.GetAccountingInfo(Evidence evidence, Type evidenceType, IsolatedStorageScope fAssmDomApp, String& typeName, String& instanceName)
       at System.IO.IsolatedStorage.IsolatedStorage._InitStore(IsolatedStorageScope scope, Evidence domainEv, Type domainEvidenceType, Evidence assemEv, Type assemblyEvidenceType, Evidence appEv, Type appEvidenceType)
       at System.IO.IsolatedStorage.IsolatedStorage.InitStore(IsolatedStorageScope scope, Type domainEvidenceType, Type assemblyEvidenceType)
       at System.IO.IsolatedStorage.IsolatedStorageFile.GetStore(IsolatedStorageScope scope, Type domainEvidenceType, Type assemblyEvidenceType)
       at MS.Internal.IO.Packaging.PackagingUtilities.ReliableIsolatedStorageFileFolder.GetCurrentStore()
       at MS.Internal.IO.Packaging.PackagingUtilities.ReliableIsolatedStorageFileFolder..ctor()
       at MS.Internal.IO.Packaging.PackagingUtilities.GetDefaultIsolatedStorageFile()
       at MS.Internal.IO.Packaging.PackagingUtilities.CreateUserScopedIsolatedStorageFileStreamWithRandomName(Int32 retryCount, String& fileName)
       at MS.Internal.IO.Packaging.SparseMemoryStream.EnsureIsolatedStoreStream()
       at MS.Internal.IO.Packaging.SparseMemoryStream.SwitchModeIfNecessary()
       at MS.Internal.IO.Packaging.SparseMemoryStream.Write(Byte[] buffer, Int32 offset, Int32 count)
       at MS.Internal.IO.Zip.ZipIOFileItemStream.Write(Byte[] buffer, Int32 offset, Int32 count)
       at System.IO.Compression.DeflateStream.PurgeBuffers(Boolean disposing)
       at System.IO.Compression.DeflateStream.Dispose(Boolean disposing)
       at System.IO.Stream.Close()
       at MS.Internal.IO.Packaging.CompressStream.ChangeMode(Mode newMode)
       at MS.Internal.IO.Zip.ZipIOLocalFileBlock.FlushExposedStreams()
       at MS.Internal.IO.Zip.ZipIOLocalFileBlock.UpdateReferences(Boolean closingFlag)
       at MS.Internal.IO.Zip.ZipIOBlockManager.SaveContainer(Boolean closingFlag)
       at MS.Internal.IO.Zip.ZipIOBlockManager.SaveStream(ZipIOLocalFileBlock blockRequestingFlush, Boolean closingFlag)
       at MS.Internal.IO.Zip.ZipIOModeEnforcingStream.Dispose(Boolean disposing)
       at System.IO.Stream.Close()
       at System.IO.Packaging.PackagePart.Close()
       at System.IO.Packaging.Package.DoClose(PackagePart p)
       at System.IO.Packaging.Package.DoOperationOnEachPart(PartOperation operation)
       at System.IO.Packaging.Package.System.IDisposable.Dispose()
       at Microsoft.Xrm.Tooling.Dmt.DataMigCommon.Utility.ZipUtils.ZipFiles(List`1 fileNames, String directoryToAdd, String outputZipFile, Boolean flattenStructure)
    ======================================================================================================================

    DataMigrationUtility.Export Error   2   27/02/20 15:30:34  Failed to create compressed output file
    DataMigrationUtility.Export Information 8   27/02/20 15:30:34  *********************************************************
    DataMigrationUtility.Export Information 8   27/02/20 15:30:34  Error : No Valid Data Found to export
    DataMigrationUtility.Export Information 8   27/02/20 15:30:34  *********************************************************
    DataMigrationUtility.Export Information 8   27/02/20 15:30:34  ** EXIT EXPORT PROCESS FAIL **
    DataMigrationUtility.Export Information 8   27/02/20 15:30:34  *********************************************************

Have messaged the owners in a hope to get a solution.

Hope this helps :)

[assmatt]

I am also having @pierrejoubert 's issue. That's the exact log file I get back when I run the Dev Ops task. I can run the same export from my desktop machine in the config migration utility with no issue. I believe it's related to the size of the output... as soon as I added in some attachments to my config data, an export that had run fine previously now errors out.

It looks like the tooling package is using System.IO.Packaging under the hood. Searching around, it looks like past a fairly small threshold (~1.5 MB) that library starts saving the packaged file to disk instead of memory. Wondering if there's anything that can be done in the build tools to make sure that it has the correct permissions it needs to do so, or if this would be an issue that would have to be taken up with the Microsoft team.

[WaelHamze]

@assmatt @pierrejoubert thanks for sharing your thoughts. I task doesn't really control the permissions as these depends on the permissions of the account running the agent. In most cases though you can write to these directories with no issues. Did you get a chance to raise this with MS?

[ghost]

I found the cause of my issue... two things, first, I needed to create the 'ReferenceData' directory in the location I am referring to in my yaml. Second, there was an issue with my schema file. I am still figuring out the issue part, but simply exporting just my adx_website entity, it completed successfully. :)

[ghost]

I am having an issue with Annotations, having that in my Schema, it fails outright... not sure why though, it does seems to export.

[tomsalu]

I was experiencing the same issue and after I removed a few un-used entities from the schema XML file it went through fine. I feel it was a size issue as @assmat mentioned. I tested it on the Data Migration Tool and removing the un-used entities from the schema reduced my ZIP export size from ~35,000kb to ~150kb.

[ghost]

I think you are right @tomsalu, I have sorted mine out completely now, I can provide the full config if required, but basically, I added a filter to target data related to the particular website(s) where applicable:

<filter>&lt;filter type="and"&gt;&lt;condition attribute="adx_websiteid" operator="eq" value="{D78574F9-20C3-4DCC-8D8D-85CF5B7AC141}" /&gt;&lt;/filter&gt;</filter>

For entities without the direct link to the Website, I configured the filter to link to it's parent that had a link to the Website(s) that I wanted to target.

Such as for the Annotations, I added the filter limited to notes attached to ONLY webfiles related to my target website(s):

<filter>&lt;fetch version="1.0" mapping="logical" distinct="false"&gt;&lt;entity name="annotation"&gt;&lt;link-entity name="adx_webfile" from="adx_webfileid" to="objectid" link-type="inner" alias="ab"&gt;&lt;filter type="and"&gt;&lt;condition attribute="adx_websiteid" operator="eq" value="{D78574F9-20C3-4DCC-8D8D-85CF5B7AC141}" /&gt;&lt;/filter&gt;&lt;/link-entity&gt;&lt;/entity&gt;&lt;/fetch&gt;</filter>

And problem solved, the DevOps tasks executes and completes successfully. Happy Days :)

[WaelHamze]

@pierrejoubert are you still having issues with the logs? Did you try to download the logs from the build/release and see whether you have everything you need in there?

[ghost]

Your timing is pretty good, yes, I am encountering the issue again.

But, but it's nothing to do with your libraries but rather what seems to be a shortcoming of the PowerShell version of the DayaMigration libraries and the documentbody property of the annotation record.

Removing the documentbody field from the schema, results in the error going away... With one caveat... The attachment is there but without a body... Basically a blank file which defeats the purpose.

This doesn't occur when using the DataMigration utility, but the PowerShell library fails.

I have come up with an PowerShell script which creates annotations for the adx_webfile records with the attached file from files in the source as the only alternative is a manual process outside of DevOps.

[WaelHamze]

@pierrejoubert thanks for sharing your experience

Do you get the same error if you import the data using the Packager Deployer?

[MichaelHolmesWP]

Have hit this issue myself as soon as my exported archive topped 10MB, as soon as I explicitly filtered out a couple of annotation records to bring it under it was fine. I've requested that the CrmSdk powershell team add a -DataFolder argument to optionally just export the 3 config migration files to a folder without archiving them as a way of bypassing the issue, my ALM immediately unzips the archive anyway.

[ghost]

@pierrejoubert thanks for sharing your experience

Do you get the same error if you import the data using the Packager Deployer?

No, the issue doesn't occur with the Package Deployer at all.

[JonTisski]

Has anyone come up with a workaround for this? Is it the data.xml that needs to be under 10MB or the final zip file?

[KimBon]

Limiting the notes worked for us too. @pierrejoubert Thanks for sharing!

[erez-yaron]

limiting the notes helped, didn't solve the problem all the time. marking the task to continue on error keeps the generated files so I could add a different task to zip the files.

[cwarnholtz]

Has this issue been resolved? Having a limit of 10mb for a data export is unreasonable. I am trying to migrate a portal app that uses the notes (annotations) heavily. I am not able to reduce the size any more without spliting the web related notes.

[MichaelHolmesWP]

@cwarnholtz The Microsoft team in charge of the package don't answer their contact email so I wouldn't hold my breath, some have had luck with wrapping the entire powershell invocation in a Try/Catch and then just manually grabbing the pre-compressed data.xml file from the temp directory it creates, otherwise the newer PowerApps CLI portal mover is much more stable than the ConfigMigration method for moving portals

[sarul84]

@pierrejoubert , I am also facing this error, if the export data pipeline task is working for you, can you please share screen shot of your task configuration? My data size is less than 10MB, below is task config screen shot.

[ghost]

@pierrejoubert , I am also facing this error, if the export data pipeline task is working for you, can you please share screen shot of your task configuration? My data size is less than 10MB, below is task config screen shot.

@sarul84 unfortunately at the time of this issue, i switched from using the tooling as tasks but rather entirely as part of a powershell script calling the various components of the tooling. I only use Build.yaml and PowerShell in my build/release pipelines now. Sadly it is all business IP so legally I wouldn't be able to share it with you.

[sarul84]

@pierrejoubert , I am also facing this error, if the export data pipeline task is working for you, can you please share screen shot of your task configuration? My data size is less than 10MB, below is task config screen shot.

@sarul84 unfortunately at the time of this issue, i switched from using the tooling as tasks but rather entirely as part of a powershell script calling the various components of the tooling. I only use Build.yaml and PowerShell in my build/release pipelines now. Sadly it is all business IP so legally I wouldn't be able to share it with you.

@pierrejoubert, no worries. I also turned to power shell script and below LinkedIn article helped.

https://www.linkedin.com/pulse/alm-power-apps-portals-ashish-sharma/?trackingId=rbY84uxNSL6MBxK4niLAJQ%3D%3D