search

mirage-project / mirage

A multi-level tensor algebra superoptimizer
Apache License 2.0
262 stars 16 forks source link

Auto pip install #20

Closed jiakunw closed 1 week ago

jiakunw commented 1 month ago

I automated pip install but you have to manually set environmental variable.

jiakunw commented 1 month ago

The code I wrote ran successfully, but the later part did not becasue the cluster did not have cython installed. When I attempted to install cython I got permission errors: WechatIMG473 WechatIMG470

jiakunw commented 1 month ago

This time I added requirement.txt. When I ran python setup.py install I got the following errors: WechatIMG496 WechatIMG497 I also tried running pip.install mirage but I got some other errors: WechatIMG498 WechatIMG499 WechatIMG500

jiazhihao commented 4 weeks ago

This time I added requirement.txt. When I ran python setup.py install I got the following errors:

Make sure you compile on a GPU node (instead of the catalyst-cluster login node), since there is no GPUs on the login node. You can do so by running salloc --nodes 1 --gres=gpu:1 and then ssh to the assigned node, which should be one of catalyst-0-9 ... catalyst-0-15.

I also tried running pip.install mirage but I got some other errors:

I am not 100% how to resolve this issue.

jiakunw commented 4 weeks ago

I fixed some bugs but _cython is not in mirage ccc

jiazhihao commented 3 weeks ago

The issue is that setup.py does not find mirage/_cython, which is at mirage/python/mirage/_cython. I have pushed a change to setup.py to fix the dir path issue. Can you pull the changes in the main branch and try again?

jiakunw commented 3 weeks ago

I tried running the commands maually using the unmodified verison of setup.py as specified in INSTALL.md, and I got the same error message. Using pdb I realized the error message came from python return cythonize(ret, compiler_directives={"language_level" : 3}) WechatIMG610 could you please take a look at config_cython()?

jiakunw commented 2 weeks ago

does that mean it worked? WechatIMG341 I also have some questions about it. cuDNN and CUDA is not available as a Python package on PyPI. cuDNN is a GPU-accelerated library for deep neural networks provided by NVIDIA, and it should be installed via conda or directly from NVIDIA's website, not from PyPI. So I removed cuda and cudnn version from requirements.txt, but they are both installed, and it seemed working. Does that mean it's done or I should fix them??

jiazhihao commented 2 weeks ago

@jiakunw Thanks for the PR. One quick comment: you may not want to include your personal vscode config in this PR: vscode/sftp.json.

jiakunw commented 2 weeks ago

fixed!

jiazhihao commented 2 weeks ago

The PR looks great overall. One last thing: you can use git rm to remove the sftp.json file and run script/format.sh to auto-format your changes.

jiakunw commented 1 week ago

script/format.sh


scripts/format.sh: line 72: /Users/jackiexu/Development/mirage/.tools/clang-format-15-master-1d7ec53d: cannot execute binary file: Exec format error``` I got this error when I ran it
jiazhihao commented 1 week ago

I can correct format in another PR. Can you use git rm to remove your json file from this PR.

jiakunw commented 1 week ago

I removed it yesterday and there was no json file in the repo WechatIMG58 WechatIMG59

jiazhihao commented 1 week ago

It is still there in this PR: https://github.com/mirage-project/mirage/pull/20/files. Let me push a fix to address this issue.

jiakunw commented 1 week ago

I re-removed it, it should be good for now :)

jiazhihao commented 1 week ago

merged. Great work!