Bertverbeek4PS / bc2adls

Exporting data from Dynamics 365 Business Central to Azure data lake storage or MS Fabric lakehouse
MIT License
49 stars 20 forks source link

Set position on AddBlockToDataBlob #10

Closed Arthurvdv closed 1 year ago

Arthurvdv commented 1 year ago

When exporting a delta larger then the "Max playload size (MiBs)” setting, an error occurs.

image

I've try'ed to keep the size of the payload in runtime after each call, but then on larger files (80MB+) a mismatch eventually occurs. To be 100% sure, retrieve the current size of the file before append and flush. With a HEAD (in stead of GET) it should be not much of a performance penalty?

Tested it up to almost a 2 GB file image