python-poetry / poetry

Python packaging and dependency management made easy
https://python-poetry.org
MIT License
31.81k stars 2.28k forks source link

How can I modify REQUESTS_TIMEOUT? #6009

Closed LGDHuaOPER closed 1 year ago

LGDHuaOPER commented 2 years ago

Issue

The default value of REQUESTS_TIMEOUT is 15. How can I modify REQUESTS_TIMEOUT? Thanks.

dimbleby commented 2 years ago

Not configurable.

Do you need it to be configurable? I think an MR would be likely to be accepted.

Secrus commented 2 years ago

We are trying to avoid adding too many configuration options. Could you please provide an example of when this could be useful?

LGDHuaOPER commented 2 years ago

I'm certainly happy to provide PR. But then again, there are indeed cases where this configuration is useful.

For example, when I use docker to build a python program image based on the poetry v1.2.0b3 package manager and have configured the image source.

[[tool.poetry.source]]
name = "tsinghua"
url = "https://pypi.tuna.tsinghua.edu.cn/simple"
default = true

[[tool.poetry.source]]
name = "aliyun"
url = "https://mirrors.aliyun.com/pypi/simple"
secondary = true

Then there is the following timeout error.

Installing collected packages: webencodings, pylev, ptyprocess, msgpack, lockfile, distlib, urllib3, tomlkit, six, shellingham, pyparsing, pycparser, poetry-core, platformdirs, pkginfo, pexpect, jeepney, idna, filelock, crashtest, charset-normalizer, certifi, cachy, virtualenv, requests, packaging, html5lib, dulwich, cleo, cffi, requests-toolbelt, cryptography, cachecontrol, SecretStorage, keyring, poetry-plugin-export, poetry
Successfully installed SecretStorage-3.3.2 cachecontrol-0.12.11 cachy-0.3.0 certifi-2022.6.15 cffi-1.15.1 charset-normalizer-2.1.0 cleo-1.0.0a5 crashtest-0.3.1 cryptography-37.0.4 distlib-0.3.5 dulwich-0.20.45 filelock-3.7.1 html5lib-1.1 idna-3.3 jeepney-0.8.0 keyring-23.7.0 lockfile-0.12.2 msgpack-1.0.4 packaging-21.3 pexpect-4.8.0 pkginfo-1.8.3 platformdirs-2.5.2 poetry-1.2.0b3 poetry-core-1.1.0b3 poetry-plugin-export-1.0.5 ptyprocess-0.7.0 pycparser-2.21 pylev-1.4.0 pyparsing-3.0.9 requests-2.28.1 requests-toolbelt-0.9.1 shellingham-1.4.0 six-1.16.0 tomlkit-0.11.1 urllib3-1.26.10 virtualenv-20.15.1 webencodings-0.5.1
Removing intermediate container c4061be924d2
 ---> f80e64cb90a4
Step 27/28 : RUN poetry install --only main
 ---> Running in 06eb0dbf3f66
Creating virtualenv alogger-aggregation-task-sl-StNVKczr-py3.10 in /home/alogger/.cache/pypoetry/virtualenvs
Installing dependencies from lock file

Package operations: 19 installs, 1 update, 0 removals

  • Installing tzdata (2022.1)
  • Installing pytz-deprecation-shim (0.1.0.post0)
  • Installing certifi (2022.6.15)
  • Installing charset-normalizer (2.1.0)
  • Installing idna (3.3)
  • Installing pytz (2022.1)
  • Installing tzlocal (4.2)
  • Installing urllib3 (1.26.10)
  • Installing clickhouse-driver (0.2.4)
  • Installing markupsafe (2.1.1)
  • Installing requests (2.28.1)
  • Updating setuptools (62.6.0 -> 63.2.0)
  • Installing six (1.16.0)
  • Installing aioch (0.0.2)
  • Installing apscheduler (3.9.1)
  • Installing dynaconf (3.1.9)
  • Installing jinja2 (3.1.2)
  • Installing requests-aws (0.1.8)
  • Installing requests-toolbelt (0.9.1)
  • Installing urlsearchparams (1.2)

  CalledProcessError

  Command '['/home/alogger/.cache/pypoetry/virtualenvs/alogger-aggregation-task-sl-StNVKczr-py3.10/bin/python', '/home/alogger/.local/lib/python3.10/site-packages/virtualenv/seed/wheels/embed/pip-22.1.2-py3-none-any.whl/pip', 'install', '--use-pep517', '--disable-pip-version-check', '--prefix', '/home/alogger/.cache/pypoetry/virtualenvs/alogger-aggregation-task-sl-StNVKczr-py3.10', '--no-deps', '/home/alogger/.cache/pypoetry/artifacts/37/01/66/fa2136f3c3a158a1fe836582d3da83d9bee7ea3032c7767eaffc95d8b6/aioch-0.0.2.tar.gz']' returned non-zero exit status 1.

  at /usr/local/lib/python3.10/subprocess.py:524 in run
       520│             # We don't call process.wait() as .__exit__ does that for us.
       521│             raise
       522│         retcode = process.poll()
       523│         if check and retcode:
    →  524│             raise CalledProcessError(retcode, process.args,
       525│                                      output=stdout, stderr=stderr)
       526│     return CompletedProcess(process.args, retcode, stdout, stderr)
       527│
       528│

The following error occurred when trying to handle this error:

  EnvCommandError

  Command ['/home/alogger/.cache/pypoetry/virtualenvs/alogger-aggregation-task-sl-StNVKczr-py3.10/bin/python', '/home/alogger/.local/lib/python3.10/site-packages/virtualenv/seed/wheels/embed/pip-22.1.2-py3-none-any.whl/pip', 'install', '--use-pep517', '--disable-pip-version-check', '--prefix', '/home/alogger/.cache/pypoetry/virtualenvs/alogger-aggregation-task-sl-StNVKczr-py3.10', '--no-deps', '/home/alogger/.cache/pypoetry/artifacts/37/01/66/fa2136f3c3a158a1fe836582d3da83d9bee7ea3032c7767eaffc95d8b6/aioch-0.0.2.tar.gz'] errored with the following return code 1, and output:
  Processing /home/alogger/.cache/pypoetry/artifacts/37/01/66/fa2136f3c3a158a1fe836582d3da83d9bee7ea3032c7767eaffc95d8b6/aioch-0.0.2.tar.gz
    Installing build dependencies: started
    Installing build dependencies: still running...
    Installing build dependencies: finished with status 'error'
    error: subprocess-exited-with-error

    × pip subprocess to install build dependencies did not run successfully.
    │ exit code: 1
    ╰─> [7 lines of output]
        WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='pypi.org', port=443): Read timed out. (read timeout=15)")': /simple/setuptools/
        WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='pypi.org', port=443): Read timed out. (read timeout=15)")': /simple/setuptools/
        WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='pypi.org', port=443): Read timed out. (read timeout=15)")': /simple/setuptools/
        WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='pypi.org', port=443): Read timed out. (read timeout=15)")': /simple/setuptools/
        WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='pypi.org', port=443): Read timed out. (read timeout=15)")': /simple/setuptools/
        ERROR: Could not find a version that satisfies the requirement setuptools>=40.8.0 (from versions: none)
        ERROR: No matching distribution found for setuptools>=40.8.0
        [end of output]

    note: This error originates from a subprocess, and is likely not a problem with pip.
  error: subprocess-exited-with-error

  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.

  at ~/.local/lib/python3.10/site-packages/poetry/utils/env.py:1458 in _run
      1454│                 output = subprocess.check_output(
      1455│                     command, stderr=subprocess.STDOUT, env=env, **kwargs
      1456│                 )
      1457│         except CalledProcessError as e:
    → 1458│             raise EnvCommandError(e, input=input_)
      1459│
      1460│         return decode(output)
      1461│
      1462│     def execute(self, bin: str, *args: str, **kwargs: Any) -> int:

The following error occurred when trying to handle this error:

  PoetryException

  Failed to install /home/alogger/.cache/pypoetry/artifacts/37/01/66/fa2136f3c3a158a1fe836582d3da83d9bee7ea3032c7767eaffc95d8b6/aioch-0.0.2.tar.gz

  at ~/.local/lib/python3.10/site-packages/poetry/utils/pip.py:51 in pip_install
       47│
       48│     try:
       49│         return environment.run_pip(*args)
       50│     except EnvCommandError as e:
    →  51│         raise PoetryException(f"Failed to install {path.as_posix()}") from e
       52│

  CalledProcessError

  Command '['/home/alogger/.cache/pypoetry/virtualenvs/alogger-aggregation-task-sl-StNVKczr-py3.10/bin/python', '/home/alogger/.local/lib/python3.10/site-packages/virtualenv/seed/wheels/embed/pip-22.1.2-py3-none-any.whl/pip', 'install', '--use-pep517', '--disable-pip-version-check', '--prefix', '/home/alogger/.cache/pypoetry/virtualenvs/alogger-aggregation-task-sl-StNVKczr-py3.10', '--no-deps', '/home/alogger/.cache/pypoetry/artifacts/90/6f/42/f53803f960c564f7c41720c95f8e67bc625a8f1401aabcdbf92c7cae81/requests-aws-0.1.8.tar.gz']' returned non-zero exit status 1.

  at /usr/local/lib/python3.10/subprocess.py:524 in run
       520│             # We don't call process.wait() as .__exit__ does that for us.
       521│             raise
       522│         retcode = process.poll()
       523│         if check and retcode:
    →  524│             raise CalledProcessError(retcode, process.args,
       525│                                      output=stdout, stderr=stderr)
       526│     return CompletedProcess(process.args, retcode, stdout, stderr)
       527│
       528│

The following error occurred when trying to handle this error:

  EnvCommandError

  Command ['/home/alogger/.cache/pypoetry/virtualenvs/alogger-aggregation-task-sl-StNVKczr-py3.10/bin/python', '/home/alogger/.local/lib/python3.10/site-packages/virtualenv/seed/wheels/embed/pip-22.1.2-py3-none-any.whl/pip', 'install', '--use-pep517', '--disable-pip-version-check', '--prefix', '/home/alogger/.cache/pypoetry/virtualenvs/alogger-aggregation-task-sl-StNVKczr-py3.10', '--no-deps', '/home/alogger/.cache/pypoetry/artifacts/90/6f/42/f53803f960c564f7c41720c95f8e67bc625a8f1401aabcdbf92c7cae81/requests-aws-0.1.8.tar.gz'] errored with the following return code 1, and output:
  Processing /home/alogger/.cache/pypoetry/artifacts/90/6f/42/f53803f960c564f7c41720c95f8e67bc625a8f1401aabcdbf92c7cae81/requests-aws-0.1.8.tar.gz
    Installing build dependencies: started
    Installing build dependencies: still running...
    Installing build dependencies: finished with status 'error'
    error: subprocess-exited-with-error

    × pip subprocess to install build dependencies did not run successfully.
    │ exit code: 1
    ╰─> [9 lines of output]
        WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='pypi.org', port=443): Read timed out. (read timeout=15)")': /simple/setuptools/
        Collecting setuptools>=40.8.0
          WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out. (read timeout=15)")': /packages/a4/53/bfc6409447ca024558b8b19d055de94c813c3e32c0296c48a0873a161cf5/setuptools-63.2.0-py3-none-any.whl
          WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out. (read timeout=15)")': /packages/a4/53/bfc6409447ca024558b8b19d055de94c813c3e32c0296c48a0873a161cf5/setuptools-63.2.0-py3-none-any.whl
          WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out. (read timeout=15)")': /packages/a4/53/bfc6409447ca024558b8b19d055de94c813c3e32c0296c48a0873a161cf5/setuptools-63.2.0-py3-none-any.whl
          WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out. (read timeout=15)")': /packages/a4/53/bfc6409447ca024558b8b19d055de94c813c3e32c0296c48a0873a161cf5/setuptools-63.2.0-py3-none-any.whl
          WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out. (read timeout=15)")': /packages/a4/53/bfc6409447ca024558b8b19d055de94c813c3e32c0296c48a0873a161cf5/setuptools-63.2.0-py3-none-any.whl
        ERROR: Could not install packages due to an OSError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Max retries exceeded with url: /packages/a4/53/bfc6409447ca024558b8b19d055de94c813c3e32c0296c48a0873a161cf5/setuptools-63.2.0-py3-none-any.whl (Caused by ReadTimeoutError("HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out. (read timeout=15)"))

        [end of output]

    note: This error originates from a subprocess, and is likely not a problem with pip.
  error: subprocess-exited-with-error

  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.

  at ~/.local/lib/python3.10/site-packages/poetry/utils/env.py:1458 in _run
      1454│                 output = subprocess.check_output(
      1455│                     command, stderr=subprocess.STDOUT, env=env, **kwargs
      1456│                 )
      1457│         except CalledProcessError as e:
    → 1458│             raise EnvCommandError(e, input=input_)
      1459│
      1460│         return decode(output)
      1461│
      1462│     def execute(self, bin: str, *args: str, **kwargs: Any) -> int:

The following error occurred when trying to handle this error:

  PoetryException

  Failed to install /home/alogger/.cache/pypoetry/artifacts/90/6f/42/f53803f960c564f7c41720c95f8e67bc625a8f1401aabcdbf92c7cae81/requests-aws-0.1.8.tar.gz

  at ~/.local/lib/python3.10/site-packages/poetry/utils/pip.py:51 in pip_install
       47│
       48│     try:
       49│         return environment.run_pip(*args)
       50│     except EnvCommandError as e:
    →  51│         raise PoetryException(f"Failed to install {path.as_posix()}") from e
       52│

The command '/bin/sh -c poetry install --only main' returned a non-zero code: 1

But when I copy the following pip.conf to dir '/home/user/.config/pip', and add the following command to Dockerfile before 'RUN poetry install --only main', and run 'docker build' again, it works.

pip.conf

[install]
trusted-host = pypi.python.org
               pypi.org
               raw.githubusercontent.com
               files.pythonhosted.org
               pypi.tuna.tsinghua.edu.cn
               mirrors.aliyun.com
               pypi.mirrors.ustc.edu.cn
               pypi.douban.com

Dockerfile

RUN python3 -m pip config --user set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple
RUN python3 -m pip config --user set global.timeout 150

The default value of REQUESTS_TIMEOUT is 15. So I guess if I turn it up, it will take effect.

Thanks.

dimbleby commented 2 years ago

REQUESTS_TIMEOUT is used for poetry's own requests calls, it is not passed on to the pip subprocess. So I don't think that this is what you want.

LGDHuaOPER commented 2 years ago

But why does the following errors appear when installing the .tar.gz packages when I use 'RUN poetry install --only main' ?

× pip subprocess to install build dependencies did not run successfully.
    │ exit code: 1
    ╰─> [7 lines of output]
        WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='pypi.org', port=443): Read timed out. (read timeout=15)")': /simple/setuptools/
        WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='pypi.org', port=443): Read timed out. (read timeout=15)")': /simple/setuptools/
        WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='pypi.org', port=443): Read timed out. (read timeout=15)")': /simple/setuptools/
        WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='pypi.org', port=443): Read timed out. (read timeout=15)")': /simple/setuptools/
        WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='pypi.org', port=443): Read timed out. (read timeout=15)")': /simple/setuptools/
        ERROR: Could not find a version that satisfies the requirement setuptools>=40.8.0 (from versions: none)
        ERROR: No matching distribution found for setuptools>=40.8.0
        [end of output]

    note: This error originates from a subprocess, and is likely not a problem with pip.
  error: subprocess-exited-with-error

  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.

  at ~/.local/lib/python3.10/site-packages/poetry/utils/env.py:1458 in _run
      1454│                 output = subprocess.check_output(
      1455│                     command, stderr=subprocess.STDOUT, env=env, **kwargs
      1456│                 )
      1457│         except CalledProcessError as e:
    → 1458│             raise EnvCommandError(e, input=input_)
      1459│
      1460│         return decode(output)
      1461│
      1462│     def execute(self, bin: str, *args: str, **kwargs: Any) -> int:

The following error occurred when trying to handle this error:

  PoetryException

  Failed to install /home/alogger/.cache/pypoetry/artifacts/37/01/66/fa2136f3c3a158a1fe836582d3da83d9bee7ea3032c7767eaffc95d8b6/aioch-0.0.2.tar.gz

  at ~/.local/lib/python3.10/site-packages/poetry/utils/pip.py:51 in pip_install
       47│
       48│     try:
       49│         return environment.run_pip(*args)
       50│     except EnvCommandError as e:
    →  51│         raise PoetryException(f"Failed to install {path.as_posix()}") from e
       52│

  CalledProcessError

Only package aioch and requests-aws have this exception, and I configured the mirror source.

dimbleby commented 2 years ago

sure, it looks as though you have a networking problem. What I am telling you is that it is happening in a pip subprocess and therefore making changes to poetry's REQUESTS_TIMEOUT will not help.

LGDHuaOPER commented 2 years ago

Well, thank you very much. Poetry is a good tool!

Kaiser1989 commented 2 years ago

Joining here: I have problems publishing my artifacts and there poetry uses exactly this 15s REQUESTS_TIMEOUT timeout. Cannot be configured. Publish randomly fails, as my artifact repository sometimes is pretty slow.

This needs to be adjusted

https://github.com/python-poetry/poetry/issues/6787

Solution: Either increase this value (e.g. 60s), or make it configureable, maybe by an env:var or by some config value.

mikeengland commented 2 years ago

Hey @Secrus, I am having a lot of trouble installing pyspark via poetry due to this hard coded 15s timeout. The pyspark tar.gz is around 280mb, and downloading this from an internal pypi mirror can take longer than the timeout, causing it to retry a couple of times before finally failing. In the past I have got round this by using pip install pyspark --default-timeout 300. I want to use poetry going forward for my projects, but this is unfortunately a blocker!

If an approach is agreed I'd be happy to look at implementing a fix PR.

Thanks, Mike

dimbleby commented 2 years ago

the timeout has nothing to do with how large the file is or how long the download takes; it's about how long it takes to establish a connection / the gap between reads.

neersighted commented 2 years ago

We discussed this in a recent call; consensus right now is an unstable environment variable would be accepted to stop any bleeding (though, in most cases improving an unreliable network is a better move), but would not be documented outside a FAQ/stabilized. Longer-term, it should be part of the same mechanism as #1623 and available as a flag as well.

mikeengland commented 2 years ago

FWIW, downgrading to a previous version of Poetry (v1.1.13) seemed to solve the issue for me. I couldn't install pyspark on v1.2.2 due to the 15s timeout errors I was encountering.

stmlange commented 2 years ago

Bumped into the same problem. A config option for such timeout would be great! As suggested using Poetry (v1.1.13) is a viable workaround.

mvthomster commented 2 years ago

Same problem today -- very long downloads on large (e.g., numpy, scipy) packages driving 443 timeout in underlying pip execution. It's not my local or area network that's the problem.

Downgraded to 1.1.13 and it downloads (very slowly) but doesn't time out.

neersighted commented 2 years ago

There are a lot of misconceptions here -- Poetry does not use pip for downloading in 1.2 or 1.1; unless you set experimental.new-installer false. In any case, if you mix 1.1 and 1.2, be aware of the sharp edges re: setuptools with 1.2-generated lock files (if setuptools is in your tree you should add it as a top level dependency, as Poetry 1.1 may uninstall it if it is classified as optional e.g. from an extra).

I can tell that everyone here is not locking with 1.1 as 1.1.13 is/was broken by a PyPI change; you should use 1.1.15, the latest 1.1 release, as it contains a fix for the PyPI API's breaking change.


In any case, we can add a knob for this temporarily until we have a better fix, and PRs are welcome. I just must stress that your local network configuration is the issue/is partially misbehaving; timeouts are related to how long it takes to establish a connection or to get a reply during a transfer, and not to how long the transfer takes. The transfer is either having trouble being established, or is dropping many packets and timing out.

Adding a knob can help stop the bleeding, but fundamentally there are issues with reliable network transfer somewhere on these systems. You can experiment with limiting concurrency with installer.max-workers to see if it's our parallel transfers overloading your connection (and if that's the case, guidance on limiting concurrency is going to be useful/result in better performance and reliability than increasing the transfer timeout and hoping).

stmlange commented 2 years ago

Maybe a silly question, but what exactly do you understand as "troubleshoot your local network"? What exactly should I do here? I'm not too familiar what poetry exactly does so I would like to understand why poetry thinks this request has timeout. A -vvv doesn't reveal anything particular interesting and running a bare requests with 15s timeout works perfectly fine in 10/10 cases:

python3.8 -c 'import requests; open("blahblah-py3-none-any.whl", "wb").write(requests.get("https://..../pypi/blahblah/2.1.3/blahblah-2.1.3-py3-none-any.whl", timeout=15).content)'

So the question truly becomes: What exactly happens within poetry thinks the request has timed-out?

From my perspective I don't really understand how I can blame the "network" when everything works fine under "poetry<=1.1.13" while a "poetry>2" just reports a timeout. Since nothing in the network has changed in-between the problem is something in poetry (or any of the dependencies used) and not the network in question.

I can also give "poetry==1.1.15" a shot, but then what does it mean when this doesn't work? Is the lock-file perhaps not up-to-date? But then a "poetry lock --no-update" doesn't change a thing.

PS: So far I didn't want to go down the route of using strace or wireshark.

mvthomster commented 2 years ago

@neersighted thanks the for the feedback. I understand the issue, though I can't fathom what my local issue might have been. I was in a near panic trying to build on a deadline. Glad there was a funky workaround (which still doesn't make sense why it worked and not ^1.2). I depend heavily on poetry. Amazing project -- super helpful, super easy to compose across repos, I'd be dead without it.

pevers commented 1 year ago

This is really nice and would help me out! I have quite a large Python package that is uploaded to our internal repository from CI/CD, adjusting the Poetry timeout helps but it is hard to do in CI/CD without this being an option.

stefanondisponibile commented 1 year ago

This is where the issue is in my case. I can (very slowly) get to the end of the stream if I increase REQUESTS_TIMEOUT to e.g. 100 (downloading torch=1.11.0).

I can see the point of "not-having-too-many-knobs". I don't see the point of keeping that timeout at 15 though.

What are the drawbacks of increasing that value? I'm questioning the default value, not on whether to configure it or not.

dimbleby commented 1 year ago

15 seconds is really a very long time to see no activity whatever from a server. For almost everyone, that's plenty long enough to indicate that something is broken: and therefore a good time to stop.

For comparison, pip's equivalent default timeout is... 15 seconds.

stefanondisponibile commented 1 year ago

Fair enough, then #7081 is a good idea. Hope it's coming soon and going to stay.

vcaraulean commented 1 year ago

We are working with poetry behind a corporate proxy. One of the effects is that before sending bytes to whom requested the file, it will be cached and scanned on the proxy. It can be observed as such: after starting the download it will sit for some time at 0% and only after that the actual download will start.

When using pip with an increased default timeout (via PIP_REQUESTS_TIMEOUT) we never have seen issues with package download. With poetry I'm on my fifth retry now of poetry install on a project with about 10 packages. Keeping fingers crossed it will succeed...

Please consider merging the PR with an explicit timeout configuration.

lmmentel commented 1 year ago

I'm using poetry with docker for local development and with the current timeout it is not very usable if I have packages with larger wheels like scipy or pyarrow.

I couldn't find a workaround so I reverted back to pip. It would be great if it would be possible to control the timeout or have some other fallback mode to get around this issue.

zenoran commented 1 year ago

Has there been any progress with this request? I understand how some feel like 15 seconds should be a sufficient amount of time in the real world, however; some of us use poetry in large corporations where internal repositories are getting pounded by clueless AI business users who deploy crap that just hammers the servers down to submission. 15 seconds is NOT enough time to even get a container build in these situations. Unfortunately this has been our reality for the past couple of weeks and poetry has become unusable at this point.

Poetry is a good tool but flexibility is key for something becoming a great tool. Configuration options are never a bad option for use cases where thousands of applications are impacted by a problem which can be mitigated by giving the developers more control over the behavior of the tool.

Wenzel commented 1 year ago

Having this timeout issue here, my CI failed to upload my package to PyPi because of this... image

How can I modify this value ? I can't publish my package right now

Edit: I basically had to specify username / password on my local machine (which is warned to be removed by PyPi in favor of API tokens), and hammer poetry publish until it passed image

github-actions[bot] commented 8 months ago

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.