WaelHamze / dyn365-ce-vsts-tasks

VSTS Extension for Dynamics 365 Customer Engagement
https://marketplace.visualstudio.com/items?itemName=WaelHamze.xrm-ci-framework-build-tasks
GNU General Public License v3.0
115 stars 54 forks source link

Schema Validation Failed for the missing fields on the entities - See log for details #217

Open frick-brian opened 3 years ago

frick-brian commented 3 years ago

So, in DevOps, I am not able to see the detailed logs for this failure. The fail step is Export Config Migration data task.

When I run the exact same schema file, on the exact same environment, in the Configuration Migration toolset from the SDK, I get no errors at all.

Are the detail logs written somewhere in DevOps that I can take a look? When I download the log from the task, it just shows the same error with no additional information. My alternate way to get the logs is to do it manually in the ConfigMigration tool, but again, no failures happen in that tool.

Error in DevOps image

Error In Downloaded Log image

Successful Export using Config Migration SDK Tool image

WaelHamze commented 3 years ago

@frick-brian are you still facing issues with this?

frick-brian commented 3 years ago

Yes, still facing the issue. Using Config Migration Manager as a workaround - but would love to be able to use CICD for this again.

It had worked in the past - so I am not sure what changed. But I can't even get into the detailed logs to see which fields it's upset about.

Thanks man! Know you're a busy fella

frick-brian commented 3 years ago

Hi @WaelHamze - any updates on this? I have been manually working around it for a while, but the time has come where I need to fully automate this process.

If it's of any help to you - I end up getting errors when I use the latest Config Migration Tool from the SDK. To resolve, I had to downgrade my Config Migration tool. However, the errors are on login, so I do not have the ability to actually trigger the error message and see if I can get any better information.

This one works (can login - can import data): image

This one doesn't work (cannot login): imageZ

So, I can't currently tell if using the latest Config Migration tool will give me the same error or not - I kind of remember getting a similar error in Config Migration as I am seeing in DevOps a month back or so - and that may have been the reason I downgraded. But now I cannot even login using the latest tool, so I cannot confirm.

Just to re-illustrate - here is the error from DevOps (which of course, I cannot drill into the logs any further than what is exposed through UI) image

And here is the success message from the same schema file, same environment, same user, using the old version of Config Migration Tool: image

frick-brian commented 3 years ago

Just wanted to follow up - I was able to hack my way in to the new version (.34) of config migration manager. So, I can login.

I then feed my schema file to export the records. It fails saying I am missing fields.

I run the EXACT same schema file through the old version (.9) of config migration manager. It works flawlessly.

I am wondering if there's some difference/issue with the newer version of Config Migration manager? I assume your api calls on the DevOps tooling are using whatever is latest from Microsoft, right?

This may end up being a MS ticket.

madsdevries-dxc commented 3 years ago

I'm experiencing this problem too. To better diagnose issues with the configuration export, I set the pipeline to continue if the task errors out and I added a command line task next. In this task I put the following code:

CD $(ConfMigrationDataDirectory)\
FOR %%X IN (Microsoft.Xrm.Tooling.ConfigurationMigration-*.log) DO (ECHO %%X & TYPE %%X)

$(ConfMigrationDataDirectory) is the folder that I have set as Log Folder on the Export task. This gave me a bit more info, but still puzzling: image

Additionally, the issue appeared over this weekend with no apparent reason - the schema file was not altered nor was the exported entity or the records being exported. And DataMigrationUtility.exe has no problem exporting the data locally.

frick-brian commented 3 years ago

I have opened a Microsoft ticket for this, but to no avail. Very unfortunate too - as I have 3 data sets that need to be selectively and periodically pushed to 25+ environments. And the number of data sets, and environments, keeps growing every so often.

Still using the old version of Config Data Migration tool to get it firing. I have been unable to use CICD for this since at least October 2020, likely before then.

PS, I do get the same errors in CICD as I do in Config Data Migration v .34, where if I use v .9, the files go in without issue.

EHulburd commented 2 years ago

Pretty sure this is a problem with the ConfigurationMigration tool, and not this Task/repo. I was having the same error, and the schema file worked fine when run through the ConfigurationMigration tool UI.

I went through and removed entities and fields one at a time from the schema, and eventually discovered a bad field it didn't like. Once this field was removed it worked fine. In my case the field was "membershiptype" on the team entity.