mindsphere / mindconnect-nodejs

NodeJS Library for Industrial IoT Connectivity - TypeScript SDK for Industrial IoT - Command Line Interface - Development Proxy - typescript-sdk is waiting for your contributions!
https://developer.siemens.com/industrial-iot-open-source/index.html
MIT License
68 stars 23 forks source link

Handling API limit #344

Closed viktorsebok closed 5 months ago

viktorsebok commented 1 year ago

All Mindsphere API call (e.g. Mindconnect API) can answer with an error "API rate limit exceeded"... This special error also gives an info about a relative time (Retry-After) which indicates that how long in 'seconds' have to wait before making a new request.

My opinion is that, the mindconnect-nodejs should handle this error, and automatically resend the requested message when the defined time is elapsed. Probably the best place is all implementation of TokenRotation.HttpAction interface function: https://github.com/mindsphere/mindconnect-nodejs/blob/d4470fbadbca8c92fd3a7ed7bc1c2068411f5919/src/api/mindconnect-base.ts#L51

sn0wcat commented 1 year ago

The library provides the

export const retry = async (n: number, func: Function, timoutinMilliseconds: number = 300, logFunction?: Function) 

which can be used to repeat the calls. Now that mindsphere provided retry-after (which is a relatively new feature) , maybe we can add a new function which can wrap such calls and use the retry-after suggestion, instead of preconfigured timeout.

I don't want to have implicit error handling for whole framework, because the API behavior has sometimes subtle differences from API to API - but a better retry method would be worth looking at.

sn0wcat commented 1 year ago

Example for using the retry function:

 const assetMgmt = sdk.GetAssetManagementClient();
 const asset = (await retry(options.retry, () =>
   assetMgmt.GetAsset(options.assetid)
)) as AssetManagementModels.AssetResourceWithHierarchyPath;
viktorsebok commented 1 year ago

Sounds promising, but you should consider some special SDK functions, e.g.: https://github.com/mindsphere/mindconnect-nodejs/blob/ece7151e81e9550102f659d8e877f4536321b9fe/src/api/mindconnect-agent.ts#L218

In case of MindConnectAgent.PutDataMapping, the number of API calls is depending on the input argument. For instance the input argument has 100 elements, which causes 100 API calls, but the API exceed the limit at 50th element. We can't handle this situation outside of the SDK function with a "retry" mechanism.

sn0wcat commented 1 year ago

I never understood why there is no bulk method for creation of data mappings.

stale[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.