Timeout on large requests: httpx default timeout bottleneck

[callumrollo]

When making a large request of an ERDDAP server, my responses sometimes timeout

e.g.

from erddapy import ERDDAP
e_ifremer = ERDDAP(
    server="https://erddap.ifremer.fr/erddap",
    protocol="tabledap",
)
e_ifremer.dataset_id = "ArgoFloats"

e_ifremer.constraints = {
 'latitude>=': 30,
 'latitude<=':  35,
 'longitude>=':-41,
 'longitude<=': -40}

ds = e_ifremer.to_xarray()

Returns the error ReadTimeout: The read operation timed out

However, downloading the .nc direct from the server causes no issues https://erddap.ifremer.fr/erddap/tabledap/ArgoFloats.nc?&latitude%3E=30&latitude%3C=35&longitude%3E=-41&longitude%3C=-40

so its not the ERDDAP itself cancelling the request for being too large.

This appears to be caused by a default timeout of 5 seconds from httpx. I have been able to get around it by adding timeout=30 to the call

httpx.get(url, follow_redirects=True, auth=auth, **kwargs)

in line 20 of erddapy/core/url.py

While it is possible to pass request_kwargs including an increased timeout in the to_pandas method (erddapy.py line 343). This is not available in to_xarray. Could it be included?

[ocefpaf]

Could it be included?

It is probably a mistake that this is not there. I would also argue our min timeout should be higher than httpx's defaults. Do you want to send a PR ;-p ?