search

microsoft / bc2adls

Exporting data from Dynamics 365 Business Central to Azure data lake storage
MIT License
92 stars 53 forks source link

Truncation and Comparing Records Counts from BC to ADL #91

Closed tglink72 closed 1 year ago

tglink72 commented 1 year ago

Hello,

    We have been having issues with truncation of some data during the export process.  The export reports success, but the delta and data tables do not contain records consistent with what we are seeing in BC. We are trying to develop a process to better identify when this occurs. We noticed the TableInformationCache-8700 is an entity that can be added to the export. We were hoping to use that table to then compare the cached record count with the record count we have in Synapse.

You do not have sufficient permissions to read from the table."ADLSE Execute"(CodeUnit 66056).ExportTableUpdates line 17 - BC2ADLS "ADLSE ExecuteSSI"(CodeUnit 66056).TryExportTableData line 12 - "ADLSE ExecuteSSI"(CodeUnit 66056).OnRun(Trigger) line 46 -

Is this table able to be exported? We have tried in a newer version of the BC2ADL tool and while it doesn't error, the table also does not seem to initiate staying in the Last Exported State of "Never Run", although it is enabled in the export. Thanks for any help!

Tom

DuttaSoumya commented 1 year ago

Please note that the table Table Information Cache is not exposed outside the module Table Information. So while you are able to select it for export, the bc2adls extension cannot access its data. Note: image

This is By Design as per this documentation. This also applies to fields in extensions that have been marked as Internal.

If you own the source code for an extension that you would like the internal table data exported to the lake for, consider using the internalsVisibleTo property on the app.json for that extension giving internal access to your bc2adls extension.