search

cpllab / lm-zoo

Easy black-box access to state-of-the-art language models
https://cpllab.github.io/lm-zoo/
MIT License
14 stars 6 forks source link

couldn't manage to get it to work in the command line or python #64

Open bnicenboim opened 3 years ago

bnicenboim commented 3 years ago

This looks great, unfortunately, I couldn't succeed to run it: From the command line:

lm-zoo tokenize ngram metamorphosis.txt
Traceback (most recent call last):
  File "/home/bruno/anaconda3/lib/python3.7/site-packages/urllib3/connectionpool.py", line 706, in urlopen
    chunked=chunked,
  File "/home/bruno/anaconda3/lib/python3.7/site-packages/urllib3/connectionpool.py", line 394, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/home/bruno/anaconda3/lib/python3.7/http/client.py", line 1244, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/bruno/anaconda3/lib/python3.7/http/client.py", line 1290, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/bruno/anaconda3/lib/python3.7/http/client.py", line 1239, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/bruno/anaconda3/lib/python3.7/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/home/bruno/anaconda3/lib/python3.7/http/client.py", line 966, in send
    self.connect()
  File "/home/bruno/anaconda3/lib/python3.7/site-packages/docker/transport/unixconn.py", line 43, in connect
    sock.connect(self.unix_socket)
FileNotFoundError: [Errno 2] No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bruno/anaconda3/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/home/bruno/anaconda3/lib/python3.7/site-packages/urllib3/connectionpool.py", line 756, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/home/bruno/anaconda3/lib/python3.7/site-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/bruno/anaconda3/lib/python3.7/site-packages/urllib3/packages/six.py", line 734, in reraise
    raise value.with_traceback(tb)
  File "/home/bruno/anaconda3/lib/python3.7/site-packages/urllib3/connectionpool.py", line 706, in urlopen
    chunked=chunked,
  File "/home/bruno/anaconda3/lib/python3.7/site-packages/urllib3/connectionpool.py", line 394, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/home/bruno/anaconda3/lib/python3.7/http/client.py", line 1244, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/bruno/anaconda3/lib/python3.7/http/client.py", line 1290, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/bruno/anaconda3/lib/python3.7/http/client.py", line 1239, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/bruno/anaconda3/lib/python3.7/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/home/bruno/anaconda3/lib/python3.7/http/client.py", line 966, in send
    self.connect()
  File "/home/bruno/anaconda3/lib/python3.7/site-packages/docker/transport/unixconn.py", line 43, in connect
    sock.connect(self.unix_socket)
urllib3.exceptions.ProtocolError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bruno/anaconda3/lib/python3.7/site-packages/docker/api/client.py", line 214, in _retrieve_server_version
    return self.version(api_version=False)["ApiVersion"]
  File "/home/bruno/anaconda3/lib/python3.7/site-packages/docker/api/daemon.py", line 181, in version
    return self._result(self._get(url), json=True)
  File "/home/bruno/anaconda3/lib/python3.7/site-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/home/bruno/anaconda3/lib/python3.7/site-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/home/bruno/anaconda3/lib/python3.7/site-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/home/bruno/anaconda3/lib/python3.7/site-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/bruno/anaconda3/lib/python3.7/site-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/home/bruno/anaconda3/lib/python3.7/site-packages/requests/adapters.py", line 498, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bruno/anaconda3/bin/lm-zoo", line 6, in <module>
    lm_zoo()
  File "/home/bruno/anaconda3/lib/python3.7/site-packages/lm_zoo/commands.py", line 47, in __call__
    raise e
  File "/home/bruno/anaconda3/lib/python3.7/site-packages/lm_zoo/commands.py", line 40, in __call__
    return self.main(*args, **kwargs)
  File "/home/bruno/anaconda3/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/bruno/anaconda3/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/bruno/anaconda3/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/bruno/anaconda3/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/bruno/anaconda3/lib/python3.7/site-packages/click/decorators.py", line 73, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/home/bruno/anaconda3/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/bruno/anaconda3/lib/python3.7/site-packages/lm_zoo/commands.py", line 160, in tokenize
    backend=state.requested_backend)
  File "/home/bruno/anaconda3/lib/python3.7/site-packages/lm_zoo/__init__.py", line 60, in tokenize
    stdin=in_file, backend=backend)
  File "/home/bruno/anaconda3/lib/python3.7/site-packages/lm_zoo/__init__.py", line 197, in run_model_command_get_stdout
    run_model_command(*args, **kwargs)
  File "/home/bruno/anaconda3/lib/python3.7/site-packages/lm_zoo/__init__.py", line 180, in run_model_command
    backend = get_compatible_backend(model, preferred_backends=preferred_backends)
  File "/home/bruno/anaconda3/lib/python3.7/site-packages/lm_zoo/backends/__init__.py", line 69, in get_compatible_backend
    return backend()
  File "/home/bruno/anaconda3/lib/python3.7/site-packages/lm_zoo/backends/docker.py", line 26, in __init__
    self._client = docker.from_env().api
  File "/home/bruno/anaconda3/lib/python3.7/site-packages/docker/client.py", line 101, in from_env
    **kwargs_from_env(**kwargs)
  File "/home/bruno/anaconda3/lib/python3.7/site-packages/docker/client.py", line 45, in __init__
    self.api = APIClient(*args, **kwargs)
  File "/home/bruno/anaconda3/lib/python3.7/site-packages/docker/api/client.py", line 197, in __init__
    self._version = self._retrieve_server_version()
  File "/home/bruno/anaconda3/lib/python3.7/site-packages/docker/api/client.py", line 222, in _retrieve_server_version
    'Error while fetching server API version: {0}'.format(e)
docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

In python, I could manage to load lm_zoo, but I can figure out how to specify a model:

lm_zoo.get_predictions("GRNN", "This is a sentence") #doesn't work

I guess it must be with lm_zoo.models, but I couldn't figure out how.

bnicenboim commented 3 years ago

any ideas?

bnicenboim commented 3 years ago

I managed to run it in the command line, it was an issue with docker. I still can't figure out how to use lm_zoo from python...

TScheffler commented 3 years ago

How did you manage it from the command line? I have the same errors there with downloading the model files (I assume).

bnicenboim commented 3 years ago

Have you installed docker? and tested that it was properly installed? After that it worked. Unfortunately, I'm not sure if this project is alive. I ended up using mostly https://pypi.org/project/minicons/

TScheffler commented 3 years ago

thx - will test!