bug: Exception raised when creating bentoml directory with home directory containing @ character

niits commented 1 year ago

Describe the bug

Exception raised when creating bentoml directory with home directory containing @ character

~/AI/pet_projects/BentoML/examples/quickstart on main took 21s
% python train.py                  

Traceback (most recent call last):
  File "train.py", line 21, in <module>
    saved_model = bentoml.sklearn.save_model(
  File "/home/tran.duc.trungb@sun-asterisk.com/anaconda3/envs/bentoml-3.8/lib/python3.8/site-packages/bentoml/_internal/frameworks/sklearn.py", line 163, in save_model
    with bentoml.models.create(
  File "/home/tran.duc.trungb@sun-asterisk.com/anaconda3/envs/bentoml-3.8/lib/python3.8/site-packages/simple_di/__init__.py", line 139, in _
    return func(*_inject_args(bind.args), **_inject_kwargs(bind.kwargs))
  File "/home/tran.duc.trungb@sun-asterisk.com/anaconda3/envs/bentoml-3.8/lib/python3.8/site-packages/simple_di/__init__.py", line 110, in _inject_kwargs
    return {k: v.get() if isinstance(v, Provider) else v for k, v in kwargs.items()}
  File "/home/tran.duc.trungb@sun-asterisk.com/anaconda3/envs/bentoml-3.8/lib/python3.8/site-packages/simple_di/__init__.py", line 110, in <dictcomp>
    return {k: v.get() if isinstance(v, Provider) else v for k, v in kwargs.items()}
  File "/home/tran.duc.trungb@sun-asterisk.com/anaconda3/envs/bentoml-3.8/lib/python3.8/site-packages/simple_di/__init__.py", line 72, in get
    return self._provide()
  File "/home/tran.duc.trungb@sun-asterisk.com/anaconda3/envs/bentoml-3.8/lib/python3.8/site-packages/simple_di/providers.py", line 124, in _provide
    value = super()._provide()
  File "/home/tran.duc.trungb@sun-asterisk.com/anaconda3/envs/bentoml-3.8/lib/python3.8/site-packages/simple_di/providers.py", line 103, in _provide
    return inject(self._func)(
  File "/home/tran.duc.trungb@sun-asterisk.com/anaconda3/envs/bentoml-3.8/lib/python3.8/site-packages/simple_di/__init__.py", line 139, in _
    return func(*_inject_args(bind.args), **_inject_kwargs(bind.kwargs))
  File "/home/tran.duc.trungb@sun-asterisk.com/anaconda3/envs/bentoml-3.8/lib/python3.8/site-packages/bentoml/_internal/configuration/containers.py", line 203, in model_store
    return ModelStore(base_dir)
  File "/home/tran.duc.trungb@sun-asterisk.com/anaconda3/envs/bentoml-3.8/lib/python3.8/site-packages/bentoml/_internal/models/model.py", line 371, in __init__
    super().__init__(base_path, Model)
  File "/home/tran.duc.trungb@sun-asterisk.com/anaconda3/envs/bentoml-3.8/lib/python3.8/site-packages/bentoml/_internal/store.py", line 69, in __init__
    self._fs = fs.open_fs(base_path)
  File "/home/tran.duc.trungb@sun-asterisk.com/anaconda3/envs/bentoml-3.8/lib/python3.8/site-packages/fs/opener/registry.py", line 220, in open_fs
    _fs, _path = self.open(
  File "/home/tran.duc.trungb@sun-asterisk.com/anaconda3/envs/bentoml-3.8/lib/python3.8/site-packages/fs/opener/registry.py", line 177, in open
    open_fs = opener.open_fs(fs_url, parse_result, writeable, create, cwd)
  File "/home/tran.duc.trungb@sun-asterisk.com/anaconda3/envs/bentoml-3.8/lib/python3.8/site-packages/fs/opener/osfs.py", line 40, in open_fs
    osfs = OSFS(path, create=create)
  File "/home/tran.duc.trungb@sun-asterisk.com/anaconda3/envs/bentoml-3.8/lib/python3.8/site-packages/fs/osfs.py", line 141, in __init__
    raise errors.CreateFailed(message)
fs.errors.CreateFailed: root path '/home/tran.duc.trungb@sun-asterisk.com/AI/pet_projects/BentoML/examples/quickstart/sun-asterisk.com/bentoml/models' does not exist

To reproduce

Steps to reproduce bugs:

Create new linux user containing @ character such as someone@somewhere
Clone repo https://github.com/bentoml/BentoML.git with commit hash: a30d95e372daa646a2e644a90e0fe337512c6f82
Run quickstart demo placed in BentoML/examples/quickstart with command python3 train.py

Expected behavior

bentoml dir should be created
model was saved successfully with following message:

root@0510a83917a1:~/AI/pet_projects/BentoML/examples/quickstart# python train.py 
Model saved: Model(tag="iris_clf:a7yvf5emxsrjqasc")

Environment

BentoML: 1.0.12 Python: 3.8.13

Platform:

Distributor ID: Ubuntu
Description:    Ubuntu 20.04.5 LTS
Release:        20.04
Codename:       focal

Root dir: /home/tran.duc.trungb@sun-asterisk.com/AI/pet_projects/BentoML/examples/quickstart

Conda env:

name: bentoml-3.8
channels:
  - defaults
dependencies:
  - _libgcc_mutex=0.1=main
  - _openmp_mutex=5.1=1_gnu
  - ca-certificates=2022.10.11=h06a4308_0
  - certifi=2022.9.24=py38h06a4308_0
  - ld_impl_linux-64=2.38=h1181459_1
  - libffi=3.3=he6710b0_2
  - libgcc-ng=11.2.0=h1234567_1
  - libgomp=11.2.0=h1234567_1
  - libstdcxx-ng=11.2.0=h1234567_1
  - ncurses=6.3=h5eee18b_3
  - openssl=1.1.1s=h7f8727e_0
  - pip=22.2.2=py38h06a4308_0
  - python=3.8.13=haa1d7c7_1
  - readline=8.2=h5eee18b_0
  - setuptools=65.5.0=py38h06a4308_0
  - sqlite=3.39.3=h5082296_0
  - tk=8.6.12=h1ccaba5_0
  - wheel=0.37.1=pyhd3eb1b0_0
  - xz=5.2.6=h5eee18b_0
  - zlib=1.2.13=h5eee18b_0
  - pip:
    - aiohttp==3.8.3
    - aiohttp-cors==0.7.0
    - aiosignal==1.2.0
    - alembic==1.8.1
    - anyio==3.6.2
    - appdirs==1.4.4
    - asgiref==3.6.0
    - async-timeout==4.0.2
    - attrs==22.1.0
    - audioread==3.0.0
    - backoff==2.2.1
    - beautifulsoup4==4.11.1
    - bentoml==1.0.12
    - boto3==1.26.3
    - botocore==1.29.3
    - build==0.9.0
    - cattrs==22.2.0
    - cerberus==1.3.4
    - cffi==1.15.1
    - chardet==5.0.0
    - charset-normalizer==2.1.1
    - circus==0.18.0
    - click==8.1.3
    - click-option-group==0.5.5
    - cloudpickle==2.2.0
    - commonmark==0.9.1
    - configparser==5.3.0
    - contextlib2==21.6.0
    - cycler==0.11.0
    - cython==0.29.32
    - decorator==5.1.1
    - deepmerge==1.1.0
    - deprecated==1.2.13
    - distance==0.1.3
    - docker==5.0.2
    - einops==0.3.0
    - exceptiongroup==1.1.0
    - filelock==3.8.0
    - flask==2.2.2
    - frozenlist==1.3.1
    - fs==2.4.16
    - g2p-en==2.1.0
    - gdown==4.4.0
    - googleapis-common-protos==1.57.1
    - grpcio==1.50.0
    - gunicorn==20.1.0
    - h11==0.14.0
    - humanfriendly==10.0
    - idna==3.4
    - importlib-metadata==5.0.0
    - importlib-resources==5.10.0
    - inflect==5.0.2
    - itsdangerous==2.1.2
    - jinja2==3.1.2
    - jmespath==1.0.1
    - joblib==1.2.0
    - kiwisolver==1.4.4
    - librosa==0.8.0
    - llvmlite==0.31.0
    - mako==1.2.3
    - markupsafe==2.1.1
    - matplotlib==3.3.3
    - multidict==6.0.2
    - nltk==3.7
    - numba==0.48.0
    - numpy==1.22.3
    - opentelemetry-api==1.14.0
    - opentelemetry-exporter-otlp-proto-http==1.14.0
    - opentelemetry-instrumentation==0.35b0
    - opentelemetry-instrumentation-aiohttp-client==0.35b0
    - opentelemetry-instrumentation-asgi==0.35b0
    - opentelemetry-proto==1.14.0
    - opentelemetry-sdk==1.14.0
    - opentelemetry-semantic-conventions==0.35b0
    - opentelemetry-util-http==0.35b0
    - packaging==21.3
    - pandas==1.5.2
    - pathspec==0.10.3
    - pep517==0.13.0
    - pillow==9.3.0
    - pip-requirements-parser==32.0.1
    - pip-tools==6.12.1
    - pooch==1.6.0
    - prometheus-client==0.15.0
    - protobuf==3.20.3
    - psutil==5.9.3
    - pycparser==2.21
    - pygments==2.14.0
    - pynvml==11.4.1
    - pyparsing==3.0.9
    - pypinyin==0.39.0
    - pysocks==1.7.1
    - python-dateutil==2.8.2
    - python-json-logger==2.0.4
    - python-multipart==0.0.5
    - pytz==2022.7
    - pyword==1.0
    - pyworld==0.2.10
    - pyyaml==5.4.1
    - pyzmq==24.0.1
    - regex==2022.10.31
    - requests==2.28.1
    - resampy==0.3.1
    - rich==13.0.0
    - ruamel-yaml==0.17.21
    - ruamel-yaml-clib==0.2.7
    - s3transfer==0.6.0
    - schema==0.7.5
    - scikit-learn==0.23.2
    - scipy==1.5.4
    - simple-di==0.1.5
    - six==1.16.0
    - sniffio==1.3.0
    - soundfile==0.10.3.post1
    - soupsieve==2.3.2.post1
    - sqlalchemy==1.3.24
    - sqlalchemy-utils==0.36.5
    - starlette==0.23.1
    - tabulate==0.9.0
    - tgt==1.4.4
    - threadpoolctl==3.1.0
    - tomli==2.0.1
    - torch==1.9.0
    - tornado==6.2
    - tqdm==4.64.1
    - typing-extensions==4.4.0
    - unidecode==1.1.2
    - urllib3==1.25.11
    - uvicorn==0.20.0
    - watchfiles==0.18.1
    - websocket-client==1.4.2
    - werkzeug==2.2.2
    - wrapt==1.14.1
    - yarl==1.8.1
    - zipp==3.10.0
prefix: /home/tran.duc.trungb@sun-asterisk.com/anaconda3/envs/bentoml-3.8

aarnphm commented 1 year ago

cc @sauyon

sudoandros commented 1 year ago

Yeah, I have the same problem with 1.0.21. Was there any progress?

sauyon commented 1 year ago

Ah, whoops, sorry. Missed that ping. This is a known bug due to how we handle paths; for now the workaround is setting your BENTOML_HOME=/some/path/without/at; we haven't resolved this. If anybody would like to contribute this should be a good first issue!

Yuanlim0919 commented 1 year ago

Hi, I have a similar problem but it was occurred when I using command bentoml build, is it a same cause? I'm using the fraud detection example repo.

Error message:

(MLOps) D:\Internship@Webcomm\ieee-fraud-detection\bentoml BentoML main examples-fraud_detection>bentoml build Building BentoML service "fraud_detection:23byh2qpc2u3kper" from build context "D:\Internship@Webcomm\ieee-fraud-detection\bentoml BentoML main examples-fraud_detection". Traceback (most recent call last): File "C:\Users\acer\Anaconda3\envs\MLOps\lib\runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "C:\Users\acer\Anaconda3\envs\MLOps\lib\runpy.py", line 86, in _run_code exec(code, run_globals) File "C:\Users\acer\Anaconda3\envs\MLOps\Scripts\bentoml.exe__main.py", line 7, in sys.exit(cli()) File "C:\Users\acer\Anaconda3\envs\MLOps\lib\site-packages\click\core.py", line 1130, in call return self.main(*args, kwargs) File "C:\Users\acer\Anaconda3\envs\MLOps\lib\site-packages\click\core.py", line 1055, in main rv = self.invoke(ctx) File "C:\Users\acer\Anaconda3\envs\MLOps\lib\site-packages\click\core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "C:\Users\acer\Anaconda3\envs\MLOps\lib\site-packages\click\core.py", line 1404, in invoke return ctx.invoke(self.callback, ctx.params) File "C:\Users\acer\Anaconda3\envs\MLOps\lib\site-packages\click\core.py", line 760, in invoke return callback(*args, kwargs) File "C:\Users\acer\Anaconda3\envs\MLOps\lib\site-packages\bentoml_cli\utils.py", line 334, in wrapper return func(*args, *kwargs) File "C:\Users\acer\Anaconda3\envs\MLOps\lib\site-packages\bentoml_cli\utils.py", line 305, in wrapper return_value = func(args, kwargs) File "C:\Users\acer\Anaconda3\envs\MLOps\lib\site-packages\bentoml_cli\utils.py", line 262, in wrapper return func(*args, kwargs) File "C:\Users\acer\Anaconda3\envs\MLOps\lib\site-packages\simpledi__init__.py", line 139, in return func(*_inject_args(bind.args), *_inject_kwargs(bind.kwargs)) File "C:\Users\acer\Anaconda3\envs\MLOps\lib\site-packages\bentoml_cli\bentos.py", line 327, in build bento = Bento.create( File "C:\Users\acer\Anaconda3\envs\MLOps\lib\site-packages\simpledi__init__.py", line 139, in return func(_inject_args(bind.args), _inject_kwargs(bind.kwargs)) File "C:\Users\acer\Anaconda3\envs\MLOps\lib\site-packages\bentoml_internal\bento\bento.py", line 198, in create ctx_fs = fs.open_fs(build_ctx) File "C:\Users\acer\Anaconda3\envs\MLOps\lib\site-packages\fs\opener\registry.py", line 220, in open_fs _fs, _path = self.open( File "C:\Users\acer\Anaconda3\envs\MLOps\lib\site-packages\fs\opener\registry.py", line 177, in open open_fs = opener.open_fs(fs_url, parse_result, writeable, create, cwd) File "C:\Users\acer\Anaconda3\envs\MLOps\lib\site-packages\fs\opener\osfs.py", line 40, in open_fs osfs = OSFS(path, create=create) File "C:\Users\acer\Anaconda3\envs\MLOps\lib\site-packages\fs\osfs.py", line 141, in init raise errors.CreateFailed(message) fs.errors.CreateFailed: root path 'D:\Internship@Webcomm\ieee-fraud-detection\bentoml BentoML main examples-fraud_detection\Webcomm\ieee-fraud-detection\bentoml BentoML main examples-fraud_detection' does not exist

FirefoxMetzger commented 6 months ago

@sauyon What would be needed in a PR to make this issue dissapear, i.e., where would we have to make changes?

I ran into this issue today but setting BENTOML_HOME did not help. Both my home directory and my username (on MacOS) contain @ symbols. BENTOML_HOME helps with an @ in the path of the home directory, or at least I imagine it does, but bentoml will still try to add my username to the path which reintroduces the @ and breaks.

My workaround was to create a new user (called bentoml :D), to move my project to that user's home, and to su - bentoml in the shell I use to work with the project. That solved my getting started issues, but I can't say anything about long-term consequences of this.

If this is a small change (and you can point me to the location) I can perhaps resolve this as I have some spare time now during vacation season.

spyrosavl commented 5 months ago

@sauyon, I can have a look into this.

aarnphm commented 5 months ago

feel free to take this @spyrosavl

kobiche commented 1 month ago

ping

I also find this important :)

bentoml / BentoML