EnergieID / entsoe-py

Python client for the ENTSO-E API (european network of transmission system operators for electricity)
MIT License
421 stars 186 forks source link

Feature Request: Enable Asynchronous Operations in entsoe-py #315

Open JrtPec opened 6 months ago

JrtPec commented 6 months ago

Preface: I have brainstormed this idea with GPT-4 and asked it to write a Github issue, here it is:

Hi everybody,

I'm currently integrating entsoe-py into a "Data Collector Engine" deployed on Azure, which consists of various functions aimed at collecting and processing data. These functions are triggered both by a schedule and by HTTP calls. As part of this integration, I've encountered a challenge related to the synchronous nature of entsoe-py, which can block and potentially interfere with other asynchronous operations within the Azure environment.

Given the importance of non-blocking operations in modern cloud applications, I'd like to propose an enhancement to entsoe-py to better support asynchronous workflows. This could greatly enhance its compatibility with environments like Azure Durable Functions, which provide built-in mechanisms for asynchronous HTTP requests.

To achieve this, I suggest modifying entsoe-py to allow for a more flexible use of its functionality, specifically by:

  1. Allowing entsoe-py to prepare and validate queries, and return the necessary URL and parameters for the request.
  2. Enabling users to defer the actual HTTP call to their own asynchronous handling mechanisms (e.g., Azure Functions in my case).
  3. Using entsoe-py to parse the HTTP response once it's received.
  4. This approach would not only solve the blocking issue but also leverage the strengths of the user's environment (such as Azure's asynchronous HTTP capabilities).

For my immediate needs, I'm particularly interested in applying this to the day-ahead prices functionality. However, I believe this could be beneficial for a broader range of entsoe-py's features and its users.

I would love to hear the community's thoughts on this proposal:

Thank you for considering this proposal. I am looking forward to your feedback and am open to contributing to the implementation of this feature.

JrtPec commented 6 months ago

That was actually easier than expected. See PR #316.

Works only for Day Ahead Prices for now.

fleimgruber commented 5 months ago

Before seeing this feature and it's design, I often wished for an API such as client.query_day_ahead_prices_async(country_code, start, end) that I could await from async user calling code.

The proposed design turns the problem inside out so would be preferable to the mentioned alternative design with a lot of new API code. For this to work smoothly the parsing code should be tightly coupled to the preparation code.

fboerman commented 4 months ago

hi @JrtPec and @fleimgruber I like this design idea in the PR to do the prepare, but not fully switch to async. that last one would be a lot of work and require loads of changes, while adding preparation is a lot simpler. lets discuss in the PR