Chavithra / degiro-connector

This is yet another library to access Degiro's API.
BSD 3-Clause "New" or "Revised" License
215 stars 47 forks source link

how to check that ChartRequest will create an URL which is too long for Degiro? #143

Open lucaga opened 4 months ago

lucaga commented 4 months ago

A bit of context, maybe it helps: My goal is to optimize portfolios that I can create with the stocks on the exchange(s) I have access without fee because of where my degiro account has been created. Yes, i am that cheap ;-)

My code collects the vwdId of several stocks (~100) and creates a chart_request object with the ChartRequest class. When this chart_request is fed to an object of the ChartFetcher class, through the method get_chart(), this returns CRITICAL:degiro_connector.quotecast.tools.chart_fetcher:404 Client Error: Not Found for url: https://charting.vwdservices.com/hchart/v1/deGiro/data.js? ...

when the code asks for less stocks (<60), the get_chart() does its job and returns all the data required.

Of course, a good, common-sense practice would be to request smaller chunks and it is all solved. Though, I was wondering if there would be a need and a way to find out how small these chunks should be.

Looking at the code of chart_fetcher.py, I think that this CRITICAL error comes from the http_request here:

        http_request = requests.Request(method="GET", url=url, params=params)
        prepped = session.prepare_request(http_request)

        try:
            response = session.send(prepped)
            response.raise_for_status()
            response_map = json.loads(
                response.text[len(chart_request.callback) + 1 : -1]
            )

            if raw is True:
                chart = response_map
            else:
                chart = Chart.model_validate(obj=response_map)

            return chart
        except requests.HTTPError as e:
            logger.fatal(e)
            if isinstance(e.response, requests.Response):
                logger.fatal(e.response.text)
            return None
        except Exception as e:
            logger.fatal(e)
            return None

it might help to insert a check on http_request length. The maximum length for a URL might be around 2000, according to the first search engine result to the question.

Since the get_chart introduces in the URL a (constant?) number of characters, in theory, the check could be done with a method on the ChartRequest object in chart.py

with some try and error and this code (after the creation of a ChartRequest object):

    chart_request_dict = chart_request.model_dump()
    json_data = json.dumps(chart_request_dict)
    estimated_legth_chart_request_dict = len(json_data)
    print("The size of the chart_request:  {}".format(estimated_legth_chart_request_dict))

I see that anything longer than ~1600 would create problems.

would it make sense to have such code in ChartRequest as a test?

Chavithra commented 4 months ago

Hi @lucaga,

I wasn't aware of this endpoint limitating, thanks for the enlightenment.

That would be a nice feature to have.

I am sure that will help future users.

Thanks