search

Capgemini / xrm-datamigration-xrmtoolbox

Export and import data for Microsoft Dataverse as an XrmToolBox plugin. Supports JSON and CSV.
MIT License
8 stars 4 forks source link

Multithreaded Processing #6

Closed tdashworth closed 1 year ago

tdashworth commented 3 years ago

Describe the bug When running the Data Import, I get the below message in the console. 

Info:Starting Single Threaded processing, you must configure connection string for multithreaded processing adn set up max threads to more than 1

To Reproduce I am running the migration via the XRM Toolbox plugin. I used the SDK authentication method.

tdashworth commented 3 years ago

It seems that only Office 365 connection strings are supported by the Data Migrator. Consuming applications of the Data Migrator must be updated.

LinnZawWin commented 3 years ago

@tdashworth I used the O365 connection string for my XrmToolBox connection but the Data Import process does not progress further than the message in your first comment. Any idea?

image

tdashworth commented 3 years ago

Hi @LinnZawWin, when was the environment created that you are connecting to?

The AuthType Office365 is becoming deprecated (please see https://docs.microsoft.com/en-us/powerapps/developer/data-platform/authenticate-office365-deprecation). I would try the OAuth type as described in the article although I didn't have much luck with that either.

We are aware of this issue and looking to release a fix soon.

LinnZawWin commented 3 years ago

@tdashworth That is the trial environment which was created less than a month ago.

I tried both Connection Wizard and OAuth connection methods but it's not working either. 😔

Thanks for planning to fix this issue soon. Please keep me posted.

ksulikow commented 3 years ago

@tdashworth This issue is not related to the engine itself but to the XrmToolbox plugin implementation - can we move this issue to XrmToolbox plugin repository, please.

ksulikow commented 3 years ago

@tdashworth That is the trial environment which was created less than a month ago.

I tried both Connection Wizard and OAuth connection methods but it's not working either. 😔

Thanks for planning to fix this issue soon. Please keep me posted.

Multi threading processing is using currently CrmServiceClient Clone method which is only supported for OAuth connections: https://docs.microsoft.com/en-us/dotnet/api/microsoft.xrm.tooling.connector.crmserviceclient.clone?view=dynamics-xrmtooling-ce-9

If you use connection string eg. with clientid and secret then it is working as I just tested it.

ksulikow commented 3 years ago

@tdashworth @LinnZawWin Can you try multithreading with the OAuth connection, please? The latest version 1.0.5 is using the Clone operation as I mentioned above, and it is working for me, be careful as you will see two sections for Thread Count - I opened a new issue for it, it will be fixed, for now set up both to eg 2 threads. Let me know if it works, please

LinnZawWin commented 3 years ago

@ksulikow Thanks for the fix. The initial issue was not just Multithreaded Processing, Data Import process was not working at all for any connection methods.

Now, I just tested version 1.0.5.0 and it is working for all types of Connection Methods for Single-Threaded processing. Multithreaded Processing is working for OAuth, SDK Login Control and Client Id / Secret connection types.

tdashworth commented 3 years ago

@ksulikow can we close this now?