The current paginated object array data export endpoints will still require hundreds of API calls to extract a days worth of data for medium to large motor freight carriers.
There are some options that would not overburden the server with data export requests; in all cases making the files available for download from a static content service (possibly a different server). e.g. rather than returning the data requested, a redirect to another location serving static content would be returned.
make the exported data available on a predefined schedule; e.g. nightly files covering the past 24h. To recognize when to download the data, clients could either poll a status endpoint or submit a callback URL.
allow clients to submit export requests and either a. poll a status endpoint or submit a callback URL
A predefined data export schedule is obviously simplest to implement. We would need to get feedback from MF carriers to known if it would be acceptable.
The current paginated object array data export endpoints will still require hundreds of API calls to extract a days worth of data for medium to large motor freight carriers.
There are some options that would not overburden the server with data export requests; in all cases making the files available for download from a static content service (possibly a different server). e.g. rather than returning the data requested, a redirect to another location serving static content would be returned.
A predefined data export schedule is obviously simplest to implement. We would need to get feedback from MF carriers to known if it would be acceptable.