asmith26 commented 3 months ago

Hi, I'm trying to run a modal job/app using the pixi package manager. I'm essentially trying to run: https://modal.com/docs/examples/hello_world

Originally posted at https://github.com/prefix-dev/pixi/issues/1566, thought I'd ask here in case you are able to provide any insight regarding the inner workings of modal. Thanks for any help!

Reproducible Example

Install pixi following: https://pixi.sh/latest/#installation (e.g. run curl -fsSL https://pixi.sh/install.sh | bash).
Create pyproject.toml containing

[project]
name = "pixi-modal"
version = "0.1.0"
requires-python = ">= 3.12"
dependencies = ["modal"]

[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"

[tool.pixi.project]
channels = ["conda-forge"]
platforms = ["linux-64"]

[tool.pixi.pypi-dependencies]
pixi-modal = { path = ".", editable = true }

[tool.pixi.dependencies]
python = ">=3.12.4,<3.13"

Install this pixi environment locally by running pixi install.
Create main.py with the following and run pixi run modal run main.py::f:

import sys
import modal

app = modal.App("example-hello-world")

@app.function()
def f(i=2):
    if i % 2 == 0:
        print("hello", i)
    else:
        print("world", i, file=sys.stderr)
    return i * i

Unfortunately this yields:

Error: ''_Client' object has no attribute 'app'

If try to run this with the following conda/mamba environment it works (using the command modal run main.py::f)

environment.yaml

```yaml name: pixi-modal channels: - conda-forge dependencies: - _libgcc_mutex=0.1=conda_forge - _openmp_mutex=4.5=2_gnu - bzip2=1.0.8=hd590300_5 - ca-certificates=2024.7.4=hbcca054_0 - ld_impl_linux-64=2.40=hf3520f5_7 - libexpat=2.6.2=h59595ed_0 - libffi=3.4.2=h7f98852_5 - libgcc-ng=14.1.0=h77fa898_0 - libgomp=14.1.0=h77fa898_0 - libnsl=2.0.1=hd590300_0 - libsqlite=3.46.0=hde9e2c9_0 - libuuid=2.38.1=h0b41bf4_0 - libxcrypt=4.4.36=hd590300_1 - libzlib=1.3.1=h4ab18f5_1 - ncurses=6.5=h59595ed_0 - openssl=3.3.1=h4ab18f5_1 - pip=24.0=pyhd8ed1ab_0 - python=3.12.4=h194c7f8_0_cpython - readline=8.2=h8228510_1 - setuptools=70.1.1=pyhd8ed1ab_0 - tk=8.6.13=noxft_h4845f30_101 - tzdata=2024a=h0c530f3_0 - wheel=0.43.0=pyhd8ed1ab_1 - xz=5.2.6=h166bdaf_0 - pip: - aiohttp==3.9.5 - aiosignal==1.3.1 - aiostream==0.5.2 - annotated-types==0.7.0 - anyio==4.4.0 - attrs==23.2.0 - certifi==2024.7.4 - click==8.1.7 - dnspython==2.6.1 - email-validator==2.2.0 - fastapi==0.111.0 - fastapi-cli==0.0.4 - frozenlist==1.4.1 - grpclib==0.4.7 - h11==0.14.0 - h2==4.1.0 - hpack==4.0.0 - httpcore==1.0.5 - httptools==0.6.1 - httpx==0.27.0 - hyperframe==6.0.1 - idna==3.7 - jinja2==3.1.4 - markdown-it-py==3.0.0 - markupsafe==2.1.5 - mdurl==0.1.2 - modal==0.63.31 - multidict==6.0.5 - orjson==3.10.6 - protobuf==4.25.3 - pydantic==2.8.2 - pydantic-core==2.20.1 - pygments==2.18.0 - python-dotenv==1.0.1 - python-multipart==0.0.9 - pyyaml==6.0.1 - rich==13.7.1 - shellingham==1.5.4 - sigtools==4.0.1 - sniffio==1.3.1 - starlette==0.37.2 - synchronicity==0.6.7 - toml==0.10.2 - typer==0.12.3 - types-certifi==2021.10.8.3 - types-toml==0.10.8.20240310 - typing-extensions==4.12.2 - ujson==5.10.0 - uvicorn==0.30.1 - uvloop==0.19.0 - watchfiles==0.22.0 - websockets==12.0 - yarl==1.9.4 ```

mwaskom commented 2 months ago

Hi, thanks for the detailed reproduction recipe. I had to make one change to run on my development laptop, which is a Mac:

$ git diff
diff --git a/pyproject.toml b/pyproject.toml
index d13b160..721a0cc 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -10,7 +10,7 @@ build-backend = "setuptools.build_meta"

 [tool.pixi.project]
 channels = ["conda-forge"]
-platforms = ["linux-64"]
+platforms = ["osx-arm64"]

 [tool.pixi.pypi-dependencies]
 pixi-modal = { path = ".", editable = true }

Interestingly, it seems to run fine when I do that?

$ pixi run modal run main.py::f
✓ Initialized. View run at https://modal.com/michael-modal-labs/main/apps/ap-NPVJuNt5lSxhS0lzYkOV5K
✓ Created objects.
├── 🔨 Created mount /Users/mwaskom/scratch/pixi-issue/main.py
└── 🔨 Created function f.
hello 2
Stopping app - local entrypoint completed.
✓ App completed. View run at https://modal.com/michael-modal-labs/main/apps/ap-NPVJuNt5lSxhS0lzYkOV5K

asmith26 commented 2 months ago

Many thanks for your help and time @mwaskom - today I tried it and it seems to work!! As far as I can tell I didn't change anything, the only difference I can see is possibly a new modal version: https://pypi.org/project/modal/#history

Many thanks again!

modal-labs / modal-client

Modal doesn't work with `pixi` (but works with a conda/mamba environment) #1975

Reproducible Example