bentoml / bentoctl

Fast model deployment on any cloud 🚀
https://bentoml.com
Other
175 stars 30 forks source link

No such file or directory: when bentoctl build -b #213

Open BaptisteLoquette opened 7 months ago

BaptisteLoquette commented 7 months ago

Describe the bug Hi, I want to deploy a bento to Azure container registry, but when I bentoctl build -b I get the following error

To Reproduce

Expected behavior Builds and push to container registry

Screenshots/Logs


/private/var/folders/xk/sdr1wnb91mz0csc0t9tw90nr0000gn/T/bentoctl-temp-56pjt46i
Traceback (most recent call last):
  File "/Users/bapt/anaconda3/bin/bentoctl", line 8, in <module>
    sys.exit(bentoctl())
             ^^^^^^^^^^
  File "/Users/bapt/anaconda3/lib/python3.11/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/bapt/anaconda3/lib/python3.11/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/Users/bapt/anaconda3/lib/python3.11/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/bapt/anaconda3/lib/python3.11/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/bapt/anaconda3/lib/python3.11/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/bapt/anaconda3/lib/python3.11/site-packages/bentoctl/cli/utils.py", line 92, in wrapper
    return_value = func(*args, **kwargs)
                   ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/bapt/anaconda3/lib/python3.11/site-packages/bentoctl/cli/utils.py", line 57, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/bapt/anaconda3/lib/python3.11/site-packages/bentoctl/cli/utils.py", line 26, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/bapt/anaconda3/lib/python3.11/site-packages/bentoctl/cli/__init__.py", line 316, in build
    generate_deployable_container(
  File "/Users/bapt/anaconda3/lib/python3.11/site-packages/bentoctl/docker_utils.py", line 120, in generate_deployable_container
    backend.build(**buildx_args)
  File "/Users/bapt/anaconda3/lib/python3.11/site-packages/bentoml/_internal/container/base.py", line 186, in build
    return subprocess.check_output(commands, cwd=context_path, env=env)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/bapt/anaconda3/lib/python3.11/subprocess.py", line 466, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/bapt/anaconda3/lib/python3.11/subprocess.py", line 548, in run
    with Popen(*popenargs, **kwargs) as process:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/bapt/anaconda3/lib/python3.11/subprocess.py", line 1026, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/Users/bapt/anaconda3/lib/python3.11/subprocess.py", line 1950, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/var/folders/xk/sdr1wnb91mz0csc0t9tw90nr0000gn/T/tmpxph518_3fsTempFS/'```

-->

**Environment:**
 - OS: MacOS 13.2.1
 - Python Version 3.9.6
 - BentoML Version : 1.1.10
 - Bentoctl Version : 0.4.0
yassinethr commented 7 months ago

Hello Baptiste,

I am having the same issue as you. Were you able to solve it ?

BaptisteLoquette commented 7 months ago

Hi, If you downgrade to bentoctl 0.3.4 it fixes it temporarily.

BaptisteLoquette commented 7 months ago

But I decided to deploy on k8s

yassinethr commented 7 months ago

Thanks for the quick reply. I just checked this solution and it seems to solve this issue. I am working for a deployment on Cloud Run (GCP)

Somehow when I run bentoctl build (to build and push the image to the container registry), and I pulled it for testing purpose, I get this error : Error: [bentoml-cli]servefailed: no Models with name 'blabla' exist in BentoML store <osfs '/home/bentoml/models'>

Did you get this error when deploying ?

michaelromagne commented 7 months ago

Hello @yassinethr, you can reinstall 0.4.0 and make the changes I proposed in the MR here : https://github.com/bentoml/bentoctl/pull/214

What happens with version 0.3.4 is that the models are not copied in the image built, that's why they are not found in your setup.

In version 0.4.0, the context manager supposed to create a temporary file system containing the bento with his models is not passed to the aws-ec2 operator, thus the operator does not find the "/tmp/xxxx" folder containing the bento to build the image.

With the changes above, the models will be copied in the bento and the image will be correctly built.

pmayd commented 6 months ago

I just installed bentoctl 0.4.0 the first time and followed the instructions here and it is still broken, I get the following error:

bentoctl build -b skill-extraction-model:pxp2hxe2rcsyq6q4 -f deployment_config.yaml
Traceback (most recent call last):
  File "/opt/homebrew/Caskroom/miniforge/base/envs/sem/bin/bentoctl", line 8, in <module>
    sys.exit(bentoctl())
  File "/opt/homebrew/Caskroom/miniforge/base/envs/sem/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/sem/lib/python3.10/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/sem/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/homebrew/Caskroom/miniforge/base/envs/sem/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/sem/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/sem/lib/python3.10/site-packages/bentoctl/cli/utils.py", line 92, in wrapper
    return_value = func(*args, **kwargs)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/sem/lib/python3.10/site-packages/bentoctl/cli/utils.py", line 57, in wrapper
    return func(*args, **kwargs)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/sem/lib/python3.10/site-packages/bentoctl/cli/utils.py", line 26, in wrapper
    return func(*args, **kwargs)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/sem/lib/python3.10/site-packages/bentoctl/cli/__init__.py", line 316, in build
    generate_deployable_container(
  File "/opt/homebrew/Caskroom/miniforge/base/envs/sem/lib/python3.10/site-packages/bentoctl/docker_utils.py", line 120, in generate_deployable_container
    backend.build(**buildx_args)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/sem/lib/python3.10/site-packages/bentoml/_internal/container/base.py", line 186, in build
    return subprocess.check_output(commands, cwd=context_path, env=env)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/sem/lib/python3.10/subprocess.py", line 421, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/opt/homebrew/Caskroom/miniforge/base/envs/sem/lib/python3.10/subprocess.py", line 503, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/opt/homebrew/Caskroom/miniforge/base/envs/sem/lib/python3.10/subprocess.py", line 971, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/opt/homebrew/Caskroom/miniforge/base/envs/sem/lib/python3.10/subprocess.py", line 1863, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/var/folders/5_/dqkkbk7n591djv58p8zn29w40000gp/T/tmp_pgvgwnefsTempFS/'
michaelromagne commented 6 months ago

@pmayd you can have a look at my message just above, it should work