In unrooted Termux proot-chroot alias of Debian, installed: Modular, Max and Mojo, but had an error at very end of process

b9Joker108 commented 5 months ago

Bug description

I took care to follow the process. I am so excited. I have really been looking forward to doing it!

I don't understand why the script couldn't find the module max:

(base) {24-04-28 10:54}localhost:~/max/examples/inference/bert-python-torchscript@main root# l
drwxr-xr-x. root root 3.4 KB Sun Apr 28 10:49:43 2024  .
drwxr-xr-x. root root 3.4 KB Sun Apr 28 10:49:44 2024  ..
.rw-r--r--. root root  18 B  Sun Apr 28 10:49:43 2024  .gitignore
.rw-r--r--. root root 392 B  Sun Apr 28 10:49:43 2024  config.pbtxt
.rwxr-xr-x. root root 2.2 KB Sun Apr 28 10:49:43 2024  deploy.sh
.rw-r--r--. root root 1.5 KB Sun Apr 28 10:49:43 2024  README.md
.rw-r--r--. root root  60 B  Sun Apr 28 10:49:43 2024 󰌠 requirements.txt
.rwxr-xr-x. root root 1.0 KB Sun Apr 28 10:49:43 2024  run.sh
.rw-r--r--. root root 3.0 KB Sun Apr 28 10:49:43 2024  simple-inference.py
.rw-r--r--. root root 3.6 KB Sun Apr 28 10:49:43 2024  triton-inference.py
(base) {24-04-28 10:54}localhost:~/max/examples/inference/bert-python-torchscript@main root# bash run.sh
+ INPUT_EXAMPLE='Paris is the [MASK] of France.'
+ MODEL_PATH=../../models/bert-mlm.torchscript
++ dirname run.sh
+ cd .
+ python3 ../common/bert-torchscript/download-model.py -o ../../models/bert-mlm.torchscript --mlm
Downloading model...
config.json: 100%|████████████████████████████████████████████████████████████████████████████████| 570/570 [00:00<00:00, 1.05MB/s]
model.safetensors: 100%|████████████████████████████████████████████████████████████████████████| 440M/440M [01:27<00:00, 5.04MB/s]
Saving model in TorchScript format...
Converting the model to TorchScript format...
/root/miniforge3/lib/python3.10/site-packages/transformers/modeling_utils.py:4371: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
  warnings.warn(
Model saved.
+ python3 simple-inference.py --text 'Paris is the [MASK] of France.' --model-path ../../models/bert-mlm.torchscript
Traceback (most recent call last):
  File "/root/max/examples/inference/bert-python-torchscript/simple-inference.py", line 14, in <module>
    from max import engine
ModuleNotFoundError: No module named 'max'
(base) {24-04-28 10:59}localhost:~/max/examples/inference/bert-python-torchscript@main root# max --help
NAME
        max — CLI interface to MAX Engine.

SYNOPSIS
        max <command>

DESCRIPTION
        Allows interacting with Modular MAX Engine through the command line.

COMMANDS
        benchmark — Benchmarks a model with MAX Engine.
        visualize — Generates a file you can use to visualize a model with Netron.app.

OPTIONS
    Common options
        --help, -h
            Displays help information.

        --version, -v
            Displays version information.

Steps to reproduce

On this page: https://docs.modular.com/engine/get-started/ I followed the process sequentially, and everything seemed to progress well. Until, running the script:

bash run.sh

I saw the first line output:

input text: Paris is the [MASK] of France.

But, not the second and final line:

filled mask: Paris is the capital of France.

Note: I exited Termux. I started a new Zsh shell session in the Debian proot chroot. The error was the same.

BTW, I executed the script in an old-school CLI terminal emulator, not a whizbang graphical shell, if that makes a difference. But, I do have graphical shells in the chroot, as I am launching a Debian GUI, through Termux:X11. Though, at present, not without error.

I await your advice.

System information

(base) {24-04-28 11:14}localhost:~/max/examples/inference/bert-python-torchscript@main root# uname -a
Linux localhost 6.2.1-PRoot-Distro #1 SMP PREEMPT Mon Dec 4 10:05:43 UTC 2023 aarch64 GNU/Linux

(base) {24-04-28 11:21}localhost:~/max/examples/inference/bert-python-torchscript@main root# cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
NAME="Debian GNU/Linux"
VERSION_ID="12"
VERSION="12 (bookworm)"
VERSION_CODENAME=bookworm
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

(base) {24-04-28 11:22}localhost:~/max/examples/inference/bert-python-torchscript@main root# max -v
max 24.2.1 (58157dc0)
Modular version 24.2.1-58157dc0-release

(base) {24-04-28 11:23}localhost:~/max/examples/inference/bert-python-torchscript@main root# mojo -v
mojo 24.2.1 (58157dc0)

(base) {24-04-28 11:24}localhost:~/max/examples/inference/bert-python-torchscript@main root# modular -v
modular 0.7.2 (d0adc668)

ehsanmok commented 5 months ago

Thanks for reporting! It looks like your python can't find max in its system modules. As followed the getting started page, make sure to run these in a new venv. You can create one with python3 -m venv venv and activate it via source /path/to/venv/bin/activate and then

MAX_PATH=$(modular config max.path) \
  && python3 -m pip install --find-links $MAX_PATH/wheels max-engine

MAX_PATH=$(modular config max.path) \
  && BASHRC=$( [ -f "$HOME/.bash_profile" ] && echo "$HOME/.bash_profile" || echo "$HOME/.bashrc" ) \
  && echo 'export MODULAR_HOME="'$HOME'/.modular"' >> "$BASHRC" \
  && echo 'export PATH="'$MAX_PATH'/bin:$PATH"' >> "$BASHRC" \
  && source "$BASHRC"

also make sure to check which python3 is using the venv python.

b9Joker108 commented 5 months ago

I created the dedicated Python Virtual Environment and activated, and checked activation with which python, and it pointed within the newly created venv. I then went back to the Max Engine tests. I realised why they aren't working. My device is aarch64, and the chroot that I am using is a guest Debian proot chroot within the Termux host main environment on Android. Will Max Engine support aarch64 in the near future?

ehsanmok commented 4 months ago

We do support ARM on Graviton currently. Please see this FAQ for more details.

modularml / max