ImageNotFound error while trying to use augmentation.

rcalsaverini commented 5 years ago

System Information

OS platform and distribution (e.g. Linux Ubuntu 16.04): Ubuntu 18.04.3 LTS
gobbli version: 0.0.2
Python version: 3.7.4
Other information relevant to the problem (GPU model, Docker version, etc): Docker version 19.03.2, build 6a30dfc

Description

I tried to run the BERTMaskedLM augmentation and got the following error:

ImageNotFound: 404 Client Error: Not Found ("pull access denied for gobbli-bert-maskedlm, repository does not exist or may require 'docker login': denied: requested access to the resource is denied")

Code for Minimal Reproducible Example

Install gobbli in a fresh virtualenvironment via pip install gobbli.
Run the following code:

from gobbli.augment import Word2Vec, BERTMaskedLM
import pathlib
wv = BERTMaskedLM(data_dir=pathlib.Path("./data/"))
X = ["Soccer is a fun game."]
wv.augment(X, times=10, p=0.5)

Output

This is the full error output:

---------------------------------------------------------------------------
HTTPError                                 Traceback (most recent call last)
~/.venvs/ctrl/lib/python3.7/site-packages/docker/api/client.py in _raise_for_status(self, response)
    260         try:
--> 261             response.raise_for_status()
    262         except requests.exceptions.HTTPError as e:

~/.venvs/ctrl/lib/python3.7/site-packages/requests/models.py in raise_for_status(self)
    939         if http_error_msg:
--> 940             raise HTTPError(http_error_msg, response=self)
    941 

HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.35/containers/create

During handling of the above exception, another exception occurred:

ImageNotFound                             Traceback (most recent call last)
~/.venvs/ctrl/lib/python3.7/site-packages/docker/models/containers.py in run(self, image, command, stdout, stderr, remove, **kwargs)
    802             container = self.create(image=image, command=command,
--> 803                                     detach=detach, **kwargs)
    804         except ImageNotFound:

~/.venvs/ctrl/lib/python3.7/site-packages/docker/models/containers.py in create(self, image, command, **kwargs)
    860         create_kwargs = _create_container_args(kwargs)
--> 861         resp = self.client.api.create_container(**create_kwargs)
    862         return self.get(resp['Id'])

~/.venvs/ctrl/lib/python3.7/site-packages/docker/api/container.py in create_container(self, image, command, hostname, user, detach, stdin_open, tty, ports, environment, volumes, network_disabled, name, entrypoint, working_dir, domainname, host_config, mac_address, labels, stop_signal, networking_config, healthcheck, stop_timeout, runtime, use_config_proxy)
    428         )
--> 429         return self.create_container_from_config(config, name)
    430 

~/.venvs/ctrl/lib/python3.7/site-packages/docker/api/container.py in create_container_from_config(self, config, name)
    439         res = self._post_json(u, data=config, params=params)
--> 440         return self._result(res, True)
    441 

~/.venvs/ctrl/lib/python3.7/site-packages/docker/api/client.py in _result(self, response, json, binary)
    266         assert not (json and binary)
--> 267         self._raise_for_status(response)
    268 

~/.venvs/ctrl/lib/python3.7/site-packages/docker/api/client.py in _raise_for_status(self, response)
    262         except requests.exceptions.HTTPError as e:
--> 263             raise create_api_error_from_http_exception(e)
    264 

~/.venvs/ctrl/lib/python3.7/site-packages/docker/errors.py in create_api_error_from_http_exception(e)
     30             cls = NotFound
---> 31     raise cls(e, response=response, explanation=explanation)
     32 

ImageNotFound: 404 Client Error: Not Found ("No such image: gobbli-bert-maskedlm:latest")

During handling of the above exception, another exception occurred:

HTTPError                                 Traceback (most recent call last)
~/.venvs/ctrl/lib/python3.7/site-packages/docker/api/client.py in _raise_for_status(self, response)
    260         try:
--> 261             response.raise_for_status()
    262         except requests.exceptions.HTTPError as e:

~/.venvs/ctrl/lib/python3.7/site-packages/requests/models.py in raise_for_status(self)
    939         if http_error_msg:
--> 940             raise HTTPError(http_error_msg, response=self)
    941 

HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.35/images/create?fromImage=gobbli-bert-maskedlm

During handling of the above exception, another exception occurred:

ImageNotFound                             Traceback (most recent call last)
<ipython-input-11-ed0c43bcf157> in <module>
----> 1 wv.augment(X, times=10, p=0.5)

~/.venvs/ctrl/lib/python3.7/site-packages/gobbli/augment/bert/model.py in augment(self, X, times, p)
    142 
    143         run_container(
--> 144             self.docker_client, self.image_tag, cmd, self.logger, **run_kwargs
    145         )
    146 

~/.venvs/ctrl/lib/python3.7/site-packages/gobbli/docker.py in run_container(client, image_tag, cmd, logger, **kwargs)
     77     # Run as current UID to avoid files created by root
     78     container = client.containers.run(
---> 79         image_tag, cmd, user=os.geteuid(), group_add=[os.getegid()], **run_kwargs
     80     )
     81 

~/.venvs/ctrl/lib/python3.7/site-packages/docker/models/containers.py in run(self, image, command, stdout, stderr, remove, **kwargs)
    803                                     detach=detach, **kwargs)
    804         except ImageNotFound:
--> 805             self.client.images.pull(image, platform=platform)
    806             container = self.create(image=image, command=command,
    807                                     detach=detach, **kwargs)

~/.venvs/ctrl/lib/python3.7/site-packages/docker/models/images.py in pull(self, repository, tag, **kwargs)
    442 
    443         pull_log = self.client.api.pull(
--> 444             repository, tag=tag, stream=True, **kwargs
    445         )
    446         for _ in pull_log:

~/.venvs/ctrl/lib/python3.7/site-packages/docker/api/image.py in pull(self, repository, tag, stream, auth_config, decode, platform)
    412         )
    413 
--> 414         self._raise_for_status(response)
    415 
    416         if stream:

~/.venvs/ctrl/lib/python3.7/site-packages/docker/api/client.py in _raise_for_status(self, response)
    261             response.raise_for_status()
    262         except requests.exceptions.HTTPError as e:
--> 263             raise create_api_error_from_http_exception(e)
    264 
    265     def _result(self, response, json=False, binary=False):

~/.venvs/ctrl/lib/python3.7/site-packages/docker/errors.py in create_api_error_from_http_exception(e)
     29         else:
     30             cls = NotFound
---> 31     raise cls(e, response=response, explanation=explanation)
     32 
     33 

ImageNotFound: 404 Client Error: Not Found ("pull access denied for gobbli-bert-maskedlm, repository does not exist or may require 'docker login': denied: requested access to the resource is denied")

jasonnance commented 5 years ago

Since BERTMaskedLM is a model, you need to call .build() on it before trying to use it so the Docker image is built. i.e.:

from gobbli.augment import Word2Vec, BERTMaskedLM
import pathlib
wv = BERTMaskedLM(data_dir=pathlib.Path("./data/"))
wv.build()
X = ["Soccer is a fun game."]
wv.augment(X, times=10, p=0.5)

We really should be catching the ImageNotFound error and give you a more helpful error message, though. I'll see about changing that.

jasonnance commented 5 years ago

Fixed in new release (0.0.3). You should now see this error if you try this:

RuntimeError: gobbli couldn't find the Docker image for the container it was asked to run. This probably means you didn't call .build() on a model before using it.

rcalsaverini commented 5 years ago

Thanks @jasonnance. Doing this solved the problem.

RTIInternational / gobbli