BULK/Batch Create Records, The quickest way to create multiple new records in Dynamics CRM.

[smartinsight-at]

What is the best way to approach creating multiple new records in a batch on a a Dynamics CRM. I am successfully creating individual records but this seems a slow process.

I have to create 300,000 records a day in the CRM as a log of all emails sent out by the client. The table has been created and I can do it one record at a time, each post takes about 3-4 seconds. As the time seems to be in the call and response, can I bulk upload e.g 100 records+ at a time and get back an array of Records Id's created in the same way as I get the record Id when I create a single item.

If possible an Example would be appreciated.

Our current method that works for single records is as follows.

    require_once '../vendor/autoload.php';

    $settings = new \AlexaCRM\WebAPI\OData\OnlineSettings();
    $settings->instanceURI = 'https://xxxxxxxxxx.dynamics.com';
    $settings->applicationID = 'xxxxxxx-xxxxxx-xxxx-xxxxx-exxxxxxxxx';
    $settings->applicationSecret = 'Btl8QXXXXXXXXXXXXXXXXXXX';
    $settings->apiVersion = '9.2';

$middleware = new \AlexaCRM\WebAPI\OData\OnlineAuthMiddleware( $settings ); $odataClient = new \AlexaCRM\WebAPI\OData\Client( $settings, $middleware ); $client = new \AlexaCRM\WebAPI\Client( $odataClient );

$record = new \AlexaCRM\Xrm\Entity('xpg_trackedactivity'); // this is to create new existing

$record['xpg_source'] = 930590000; $record['xpg_description'] = $row['description']; $record['xpg_campaignname'] = $row['campaignname'];

try { // run your code here $externalresult=$client->Create( $record ); $status="Sent"; } catch (exception $e) { $status="Error"; //code to handle the exception echo "error Trapped: ".$e; }

Many Thanks Adam Taylor

[georged]

Hey Adam,

Consider batch requests. The toolkit does not have any built-in support for this, you'd have to go down to raw Web API level. You can also spin multiple threads under separate app users (to avoid being throttled) each doing part of the insert.

Considering that your insert is a simple one, the other options to consider is to try using existing tools like data import or Power Automate.

HTH George

[smartinsight-at]

Hi George,

Thank you for that, I will have a look.

Adam

From: George Doubinski <> Sent: 04 July 2022 02:20 To: AlexaCRM/dynamics-webapi-toolkit @.> Cc: smartinsight-at @.>; Author @.***> Subject: Re: [AlexaCRM/dynamics-webapi-toolkit] BULK/Batch Create Records, The quickest way to create multiple new records in Dynamics CRM. (Issue #79)

Hey Adam,

Consider batch requests https://docs.microsoft.com/power-apps/developer/data-platform/webapi/execute-batch-operations-using-web-api . The toolkit does not have any built-in support for this, you'd have to go down to raw Web API level. You can also spin multiple threads under separate app users (to avoid being throttled) each doing part of the insert.

Considering that your insert is a simple one, the other options to consider is to try using existing tools like data import or Power Automate.

HTH George

— Reply to this email directly, view it on GitHub https://github.com/AlexaCRM/dynamics-webapi-toolkit/issues/79#issuecomment-1173231544 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ATPT3ASD35IVAC4ZEVSIX2TVSI333ANCNFSM52N7QBZA . You are receiving this because you authored the thread. https://github.com/notifications/beacon/ATPT3AXB77CCUSDEYAG57ILVSI333A5CNFSM52N7QBZKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOIXXBPOA.gif Message ID: @. @.> >

[SpartakusMd]

Hello, as another recommendation I would suggest to use a cache pool if you don't do it already. We also had slow responses from the CRM as we forgot to plug it. Afterwards, the responses were super fast. The slowest part before was to get the odata and the authentication.