search

argosopentech / argos-translate

Open-source offline translation library written in Python
https://www.argosopentech.com
MIT License
3.47k stars 253 forks source link

Argos_translate no longer works offline? #385

Open stabud opened 6 months ago

stabud commented 6 months ago

System linux mint lmde6 (current release). Installed everything according to the instructions:

git clone https://github.com/argosopentech/argos-translate.git
cd argos-translate
virtualenv env
source env/bin/activate
pip install -e .

When the internet is available, argos-translate works. But if you turn off the Internet, this is displayed:

 Traceback (most recent call last):
  File "/home/user/argos-translate/env/lib/python3.11/site-packages/urllib3/connection.py", line 203, in _new_conn
    sock = connection.create_connection(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/argos-translate/env/lib/python3.11/site-packages/urllib3/util/connection.py", line 60, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/socket.py", line 962, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
socket.gaierror: [Errno -2] Name or service not known

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/user/argos-translate/env/lib/python3.11/site-packages/urllib3/connectionpool.py", line 790, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/home/user/argos-translate/env/lib/python3.11/site-packages/urllib3/connectionpool.py", line 491, in _make_request
    raise new_e
  File "/home/user/argos-translate/env/lib/python3.11/site-packages/urllib3/connectionpool.py", line 467, in _make_request
    self._validate_conn(conn)
  File "/home/user/argos-translate/env/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1096, in _validate_conn
    conn.connect()
  File "/home/user/argos-translate/env/lib/python3.11/site-packages/urllib3/connection.py", line 611, in connect
    self.sock = sock = self._new_conn()
                       ^^^^^^^^^^^^^^^^
  File "/home/user/argos-translate/env/lib/python3.11/site-packages/urllib3/connection.py", line 210, in _new_conn
    raise NameResolutionError(self.host, self, e) from e
urllib3.exceptions.NameResolutionError: <urllib3.connection.HTTPSConnection object at 0x7fcf7ed4e9d0>: Failed to resolve 'raw.githubusercontent.com' ([Errno -2] Name or service not known)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/user/argos-translate/env/lib/python3.11/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/home/user/argos-translate/env/lib/python3.11/site-packages/urllib3/connectionpool.py", line 844, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/home/user/argos-translate/env/lib/python3.11/site-packages/urllib3/util/retry.py", line 515, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /stanfordnlp/stanza-resources/main/resources_1.7.0.json (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7fcf7ed4e9d0>: Failed to resolve 'raw.githubusercontent.com' ([Errno -2] Name or service not known)"))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/user/argos-translate/env/bin/argos-translate", line 7, in <module>
    exec(compile(f.read(), __file__, 'exec'))
  File "/home/user/argos-translate/bin/argos-translate", line 5, in <module>
    cli.main()
  File "/home/user/argos-translate/argostranslate/cli.py", line 63, in main
    print(translation.translate(text_to_translate))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/argos-translate/argostranslate/translate.py", line 63, in translate
    return self.hypotheses(input_text, num_hypotheses=1)[0].value
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/argos-translate/argostranslate/translate.py", line 296, in hypotheses
    translated_paragraph = self.underlying.hypotheses(
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/argos-translate/argostranslate/translate.py", line 173, in hypotheses
    apply_packaged_translation(
  File "/home/user/argos-translate/argostranslate/translate.py", line 418, in apply_packaged_translation
    stanza_pipeline = stanza.Pipeline(
                      ^^^^^^^^^^^^^^^^
  File "/home/user/argos-translate/env/lib/python3.11/site-packages/stanza/pipeline/core.py", line 214, in __init__
    download_resources_json(self.dir,
  File "/home/user/argos-translate/env/lib/python3.11/site-packages/stanza/resources/common.py", line 450, in download_resources_json
    request_file(
  File "/home/user/argos-translate/env/lib/python3.11/site-packages/stanza/resources/common.py", line 152, in request_file
    download_file(url, temppath, proxies, raise_for_status)
  File "/home/user/argos-translate/env/lib/python3.11/site-packages/stanza/resources/common.py", line 114, in download_file
    r = requests.get(url, stream=True, proxies=proxies)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/argos-translate/env/lib/python3.11/site-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/argos-translate/env/lib/python3.11/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/argos-translate/env/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/argos-translate/env/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/argos-translate/env/lib/python3.11/site-packages/requests/adapters.py", line 519, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /stanfordnlp/stanza-resources/main/resources_1.7.0.json (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7fcf7ed4e9d0>: Failed to resolve 'raw.githubusercontent.com' ([Errno -2] Name or service not known)"))
stabud commented 6 months ago

Through trial and error I found the right solution. To correctly install Argos-Translate, which will work offline, you need to follow these steps:

git clone https://github.com/argosopentech/argos-translate.git
cd argos-translate
virtualenv env
source env/bin/activate
pip install argostranslate
pip install argostranslategui

Note! If you do this:

git clone https://github.com/argosopentech/argos-translate.git
cd argos-translate
virtualenv env
source env/bin/activate
pip install -e .

then offline will not work!

bruceleerabbit commented 2 months ago

This is quite alarming because no one expects argos-translate to use the cloud. 

$ argos-translate --help
usage: argos-translate [-h] [--from-lang FROM_LANG] [--to-lang TO_LANG] [TEXT]

Open-source offline translation.
…

I run argos this way to enforce the offline expectation:

firejail --net=none --env=ARGOS_DEVICE_TYPE=auto argos-translate $args

Nonetheless, there is a substantial security issue here for anyone not sandboxing argos (probaby most people), so I suggest prepending “(security)” to the title. To worsen the security problem, note that the argos-translate project intermingles with libretranslate, which is inside the privacy-abusing walled garden of Cloudflare. So it’s concerning that the app uses the cloud when there is a good chance it could be sharing sensitive info with Cloudflare.

The other issue is indeed the packaging. The various pip tools are designed for python developers and not proper tools for end users, which I mention in bug 254. But since there is no proper package we are trapped. That said, the best of the worst is pipx.

I got burnt in a debian upgrade because python3 updates broke all pip-installed apps, thus forcing me to reinstall argos-translate. It was a disaster but one thing I discovered is pipx at least tries to do better than pip by automatically deploying a venv on every install, and by supporting upgrades. It’s still not a proper pkg manager but it’s fewer steps than pip or pip3. You can run something like this:

$ PIPX_HOME=${prefix:-/opt/}/pipx PIPX_BIN_DIR=${prefix:-/usr/local}/bin pipx install --verbose argostranslate

Then to verify:

$ PIPX_HOME=${prefix:-/opt/}/pipx PIPX_BIN_DIR=${prefix:-/usr/local}/bin pipx list
  venvs are in /opt/pipx/venvs
  apps are exposed on your $PATH at /usr/local/bin
     package argostranslate 1.9.6, installed using Python 3.11.2
      - argos-translate
      - argospm

Note that the PIPX_HOME and PIPX_BIN_DIR vars are optional. I use them to do a multi-user systemwide install. But it is a hack that makes some python devs cringe. Python’s toolchains assume everything is installed by and for a single user.