Please use the Simple or JSON API instead

[MartinThoma]

Environment

• pip version: pip 20.3.1 from /home/math/.pyenv/versions/3.7.8/lib/python3.7/site-packages/pip (python 3.7) and pip 20.3.2
• Python version: Python 3.7.8
• OS: Windows with WSL2 (Ubuntu 20.04)

Description

I just got this:

$ pip search mitre
ERROR: Exception:
Traceback (most recent call last):
  File "/home/math/.pyenv/versions/3.7.8/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 224, in _main
    status = self.run(options, args)
  File "/home/math/.pyenv/versions/3.7.8/lib/python3.7/site-packages/pip/_internal/commands/search.py", line 62, in run
    pypi_hits = self.search(query, options)
  File "/home/math/.pyenv/versions/3.7.8/lib/python3.7/site-packages/pip/_internal/commands/search.py", line 82, in search
    hits = pypi.search({'name': query, 'summary': query}, 'or')
  File "/home/math/.pyenv/versions/3.7.8/lib/python3.7/xmlrpc/client.py", line 1112, in __call__
    return self.__send(self.__name, args)
  File "/home/math/.pyenv/versions/3.7.8/lib/python3.7/xmlrpc/client.py", line 1452, in __request
    verbose=self.__verbose
  File "/home/math/.pyenv/versions/3.7.8/lib/python3.7/site-packages/pip/_internal/network/xmlrpc.py", line 46, in request
    return self.parse_response(response.raw)
  File "/home/math/.pyenv/versions/3.7.8/lib/python3.7/xmlrpc/client.py", line 1342, in parse_response
    return u.close()
  File "/home/math/.pyenv/versions/3.7.8/lib/python3.7/xmlrpc/client.py", line 656, in close
    raise Fault(**self._stack[0])
xmlrpc.client.Fault: <Fault -32500: 'RuntimeError: This API has been temporarily disabled due to unmanageable load and will be deprecated in the near future. Please use the Simple or JSON API instead.'>

Expected behavior

No error message (or at least no traceback)

How to Reproduce

pip search something

Output

See above

[pradyunsg]

See https://status.python.org/incidents/grk0k7sz6zkp for context.

PyPI has had significant uptick in automated hits to the XMLRPC API, which has resulted in needing to take that offline, as it hampers the rest of the application's ability to function.

[wkt]

This happen to me too how to fix ?

[Syize]

Happen to me too Have you found any way to fix it?

[MartinThoma]

@wkt @Syize "Me too" is not helpful. You can leave a "thumbs up" on the question. That already indicates you're interested in it. The developers are aware of the problem and are blocked by https://status.python.org/incidents/grk0k7sz6zkp . Those comments stop them from working on issues.

[h3j4k5]

If only there was a way to blacklist ips....

[jrk]

For everyone landing here wondering what to do, note that this only affects pip search. That command is currently broken by this, but you can still search the same database in your browser at https://pypi.org. Once you find a package there, pip install <package-name> etc. still work.

[amirouche]

Here is a quickfix to workaround the fact that pip search is broken:

Usage:

  curl "https://search.hyper.dev/?q=jdango+saerch"

55  django-oscar-docdata    https://pypi.org/project/django-oscar-docdata/
55  django-redirects    https://pypi.org/project/django-redirects/
55  django-oscar-dsd    https://pypi.org/project/django-oscar-dsd/
56  django-search-statistic https://pypi.org/project/django-search-statistic/
56  django-trace    https://pypi.org/project/django-trace/
56  django-rated    https://pypi.org/project/django-rated/
56  django-react    https://pypi.org/project/django-react/
57  django-screener https://pypi.org/project/django-screener/
57  django-sections https://pypi.org/project/django-sections/
57  django-sitecats https://pypi.org/project/django-sitecats/
57  django-retracer https://pypi.org/project/django-retracer/
57  django-sendgrid https://pypi.org/project/django-sendgrid/
57  django-redirect https://pypi.org/project/django-redirect/
58  onionsearch https://pypi.org/project/onionsearch/
58  django-staticshard  https://pypi.org/project/django-staticshard/
58  django-sebs https://pypi.org/project/django-sebs/
58  django-scss https://pypi.org/project/django-scss/
58  jina-search https://pypi.org/project/jina-search/
59  django-sharded  https://pypi.org/project/django-sharded/
59  django-static-echarts   https://pypi.org/project/django-static-echarts/
59  django-oscar-datacash   https://pypi.org/project/django-oscar-datacash/
59  django-ratchet  https://pypi.org/project/django-ratchet/
60  django_rediscache   https://pypi.org/project/django_rediscache/
61  job-search  https://pypi.org/project/job-search/
62  django-shared   https://pypi.org/project/django-shared/
62  django-tracer   https://pypi.org/project/django-tracer/
62  django-trench   https://pypi.org/project/django-trench/
62  django-scanredirect https://pypi.org/project/django-scanredirect/
64  django-oscar    https://pypi.org/project/django-oscar/
69  djangorestsearch    https://pypi.org/project/djangorestsearch/
69  django-searchbar    https://pypi.org/project/django-searchbar/
69  django-oscar-cch    https://pypi.org/project/django-oscar-cch/
77  django-search   https://pypi.org/project/django-search/

[amirouche]

Related issue: https://github.com/pypa/warehouse/issues/8002

[uranusjr]

The issue linked above is about replacing the functionality of the search box you see on pypi.org, not pip search (the issue of which is a standard API that not only applies to PyPI but also custom indexes).

[amirouche]

My understanding is if MeiliSearch is good enough it can sustain the load for pip search too.

[uranusjr]

This is mainly speculation since I am not familiar with how Warehouse is implementing search, but my understanding is that the current load issue is caused by the XML-RPC API, not the actual search itself. So the solution must be a replacement of XML-RPC, and what is providing the actual search results internally does not really matter.

[di]

This is mainly speculation since I am not familiar with how Warehouse is implementing search, but my understanding is that the current load issue is caused by the XML-RPC API, not the actual search itself. So the solution must be a replacement of XML-RPC, and what is providing the actual search results internally does not really matter.

This is accurate.

[pradyunsg]

Closing since #5216 is a superset of this issue and whatever the resolution is there, that'll be the answer to this issue as well. :)