[Modular CLI]:Install bug

peterzyk commented 9 months ago

Issue description

Steps to reproduce

Include relevant code snippet that did not work as expected.
If applicable, add screenshots to help explain the problem.
Include anything else that might help us debug th mojo-bug.md e issue.

Version Info

- Provide Modular CLI version by pasting the output of `modular -v`
- What OS did you install modular CLI on ?
/❯ modular -v
modular 0.2.2 (95d42445)

Sharktheone commented 9 months ago

Is the "i tried many" related to modular install mojo or modular clean? If you havent triedmodular clean` yet, please run it. It should fix at least this warning:

==========================
Failure Information:
    - Creating symlink for /Users/xuan_kai/.modular/pkg/packages.modular.com_mojo/lib/mblack/_internal/Python3.framework/Versions/3.9/Python3 -> /Users/xuan_kai/.modular/pkg/packages.modular.com_mojo/lib/mblack/_internal/Python3 failed: File exists
    - Creating symlink for /Users/xuan_kai/.modular/pkg/packages.modular.com_mojo/lib/liblldb.18.0.0git.dylib -> /Users/xuan_kai/.modular/pkg/packages.modular.com_mojo/lib/liblldb.dylib failed: File exists
    - Creating symlink for /Users/xuan_kai/.modular/pkg/packages.modular.com_mojo/lib/mblack/_internal/Python3.framework/Versions/3.9/Python3 -> /Users/xuan_kai/.modular/pkg/packages.modular.com_mojo/lib/mblack/_internal/Python3.framework/Python3 failed: File exists
==========================

peterzyk commented 9 months ago

我运行了 modular clean，然后再次执行 modular install mojo，就会触发 pip 版本过低。

按照提示，我升级pip版本，再次运行 modular install mojo就会触发

Failure Information:
    - Creating symlink for /Users/xuan_kai/.modular/pkg/packages.modular.com_mojo/lib/mblack/_internal/Python3.framework/Versions/3.9/Python3 -> /Users/xuan_kai/.modular/pkg/packages.modular.com_mojo/lib/mblack/_internal/Python3 failed: File exists
    - Creating symlink for /Users/xuan_kai/.modular/pkg/packages.modular.com_mojo/lib/liblldb.18.0.0git.dylib -> /Users/xuan_kai/.modular/pkg/packages.modular.com_mojo/lib/liblldb.dylib failed: File exists
    - Creating symlink for /Users/xuan_kai/.modular/pkg/packages.modular.com_mojo/lib/mblack/_internal/Python3.framework/Versions/3.9/Python3 -> /Users/xuan_kai/.modular/pkg/packages.modular.com_mojo/lib/mblack/_internal/Python3.framework/Python3 failed: File exists

需要我modular clean，死循环。

peterzyk commented 9 months ago

I think it is caused by the low version of pip in the resources for installing mojo. If the official upgrades the pip in the resources, users can avoid this problem when installing.

Sharktheone commented 9 months ago

I think, your main problem is that the tests are failing. The pip version thing is only a note that there is a new version available. You can try running mojo -v and mojo nevertheless because it should be installed. I've seen failing tests basically all the time in these issues. I think, modular should provide more information on what failed.

smuniraj2018 commented 9 months ago

can you delete Modular Home dir and try again and let us know if this get resolved?

peterzyk commented 9 months ago

I think, your main problem is that the tests are failing. The pip version thing is only a note that there is a new version available. You can try running mojo -v and mojo nevertheless because it should be installed. I've seen failing tests basically all the time in these issues. I think, modular should provide more information on what failed.

This is not feasible, it says mojo is not installed

peterzyk commented 9 months ago

can you delete Modular Home dir and try again and let us know if this get resolved?

Executing modular clean will automatically delete Modular Home dir

peterzyk commented 9 months ago

I tried again today and still failed, falling into the endless loop I described before. The execution process of all commands is as follows

Last login: Tue Dec 12 22:41:15 on console
[oh-my-zsh] Random theme 'blinks' loaded

xuan_kai@MacBook-Pro-2 ~
  % date                                                                  !5872
2023年12月12日 星期二 22时43分40秒 CST

xuan_kai@MacBook-Pro-2 ~
  % modular clean                                                         !5873

xuan_kai@MacBook-Pro-2 ~
  % curl https://get.modular.com | sh - && \                              !5874
modular auth mut_d4b55d170dc74d4c81832372c67750de
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3885  100  3885    0     0   5185      0 --:--:-- --:--:-- --:--:--  5180
Updating Homebrew
Updated 3 taps (homebrew/services, homebrew/core and homebrew/cask).
==> New Formulae
argc            chisel-tunnel   drogon          open-simh       retire
cargo-sweep     direwolf        glbinding@2     redress
==> New Casks
anka-build-cloud-controller              screens-assist
anka-build-cloud-registry                truhu
garmin-basecamp
==> Outdated Formulae
aom                        go                         node
apktool                    gtksourceview4             nvm
c-ares                     libfido2                   openssl@1.1
ca-certificates            libgpg-error               pandoc
coreutils                  libksba                    protobuf
curl                       libpq                      python-certifi
enchant                    libpthread-stubs           python-cryptography
exploitdb                  libuv                      python@3.10
freetds                    libxml2                    python@3.12
gettext                    mysql                      ruby
gh                         mysql-client               ruby@2.7
git-gui                    nginx                      sqlmap
glib                       nikto                      telnet

You have 39 outdated formulae installed.
You can upgrade them with brew upgrade
or list them with brew outdated.
Installing/upgrading Modular using Homebrew
Warning: modularml/packages/modular 0.2.2 already installed
  __  __           _       _
 |  \/  | ___   __| |_   _| | __ _ _ __
 | |\/| |/ _ \ / _` | | | | |/ _` | '__|
 | |  | | (_) | (_| | |_| | | (_| | |
 |_|  |_|\___/ \__,_|\__,_|_|\__,_|_|

Welcome to the Modular CLI!
For info about this tool, type "modular --help".

To install Mojo🔥, type "modular install mojo".

For Mojo documentation, see https://docs.modular.com/mojo.
To chat on Discord, visit https://discord.gg/modular.
To report issues, go to https://github.com/modularml/mojo/issues.

xuan_kai@MacBook-Pro-2 ~
  % modular install mojo                                                  !5875
# Found release for https://packages.modular.com/mojo @ 0.6.0
# Installing to /Users/xuan_kai/.modular/pkg/packages.modular.com_mojo
# Downloading artifacts. Please wait...
# Downloads complete, setting configs...
# Configs complete, running post-install hooks...
Collecting find_libpython==0.3.0
  Using cached find_libpython-0.3.0-py3-none-any.whl (8.5 kB)
Collecting jupyter_client>=8.3.0
  Using cached jupyter_client-8.6.0-py3-none-any.whl (105 kB)
Collecting ipykernel>=6.25.0
  Using cached ipykernel-6.27.1-py3-none-any.whl (114 kB)
Collecting jupyter-core!=5.0.*,>=4.12
  Using cached jupyter_core-5.5.0-py3-none-any.whl (28 kB)
Collecting pyzmq>=23.0
  Using cached pyzmq-25.1.2-cp39-cp39-macosx_10_15_universal2.whl (1.9 MB)
Collecting python-dateutil>=2.8.2
  Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting traitlets>=5.3
  Using cached traitlets-5.14.0-py3-none-any.whl (85 kB)
Collecting importlib-metadata>=4.8.3
  Using cached importlib_metadata-7.0.0-py3-none-any.whl (23 kB)
Collecting tornado>=6.2
  Using cached tornado-6.4-cp38-abi3-macosx_10_9_x86_64.whl (431 kB)
Collecting comm>=0.1.1
  Using cached comm-0.2.0-py3-none-any.whl (7.0 kB)
Collecting ipython>=7.23.1
  Using cached ipython-8.18.1-py3-none-any.whl (808 kB)
Collecting nest-asyncio
  Using cached nest_asyncio-1.5.8-py3-none-any.whl (5.3 kB)
Collecting matplotlib-inline>=0.1
  Using cached matplotlib_inline-0.1.6-py3-none-any.whl (9.4 kB)
Collecting psutil
  Using cached psutil-5.9.6-cp36-abi3-macosx_10_9_x86_64.whl (246 kB)
Collecting appnope
  Using cached appnope-0.1.3-py2.py3-none-any.whl (4.4 kB)
Collecting packaging
  Using cached packaging-23.2-py3-none-any.whl (53 kB)
Collecting debugpy>=1.6.5
  Using cached debugpy-1.8.0-py2.py3-none-any.whl (5.0 MB)
Collecting zipp>=0.5
  Using cached zipp-3.17.0-py3-none-any.whl (7.4 kB)
Collecting typing-extensions
  Using cached typing_extensions-4.9.0-py3-none-any.whl (32 kB)
Collecting jedi>=0.16
  Using cached jedi-0.19.1-py2.py3-none-any.whl (1.6 MB)
Collecting decorator
  Using cached decorator-5.1.1-py3-none-any.whl (9.1 kB)
Collecting prompt-toolkit<3.1.0,>=3.0.41
  Downloading prompt_toolkit-3.0.42-py3-none-any.whl (385 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 386.0/386.0 KB 1.4 MB/s eta 0:00:00
Collecting pygments>=2.4.0
  Using cached pygments-2.17.2-py3-none-any.whl (1.2 MB)
Collecting stack-data
  Using cached stack_data-0.6.3-py3-none-any.whl (24 kB)
Collecting exceptiongroup
  Using cached exceptiongroup-1.2.0-py3-none-any.whl (16 kB)
Collecting pexpect>4.3
  Using cached pexpect-4.9.0-py2.py3-none-any.whl (63 kB)
Collecting platformdirs>=2.5
  Using cached platformdirs-4.1.0-py3-none-any.whl (17 kB)
Collecting six>=1.5
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting parso<0.9.0,>=0.8.3
  Using cached parso-0.8.3-py2.py3-none-any.whl (100 kB)
Collecting ptyprocess>=0.5
  Using cached ptyprocess-0.7.0-py2.py3-none-any.whl (13 kB)
Collecting wcwidth
  Using cached wcwidth-0.2.12-py2.py3-none-any.whl (34 kB)
Collecting executing>=1.2.0
  Using cached executing-2.0.1-py2.py3-none-any.whl (24 kB)
Collecting asttokens>=2.1.0
  Using cached asttokens-2.4.1-py2.py3-none-any.whl (27 kB)
Collecting pure-eval
  Using cached pure_eval-0.2.2-py3-none-any.whl (11 kB)
Installing collected packages: wcwidth, pure-eval, ptyprocess, find_libpython, appnope, zipp, typing-extensions, traitlets, tornado, six, pyzmq, pygments, psutil, prompt-toolkit, platformdirs, pexpect, parso, packaging, nest-asyncio, executing, exceptiongroup, decorator, debugpy, python-dateutil, matplotlib-inline, jupyter-core, jedi, importlib-metadata, comm, asttokens, stack-data, jupyter_client, ipython, ipykernel
Successfully installed appnope-0.1.3 asttokens-2.4.1 comm-0.2.0 debugpy-1.8.0 decorator-5.1.1 exceptiongroup-1.2.0 executing-2.0.1 find_libpython-0.3.0 importlib-metadata-7.0.0 ipykernel-6.27.1 ipython-8.18.1 jedi-0.19.1 jupyter-core-5.5.0 jupyter_client-8.6.0 matplotlib-inline-0.1.6 nest-asyncio-1.5.8 packaging-23.2 parso-0.8.3 pexpect-4.9.0 platformdirs-4.1.0 prompt-toolkit-3.0.42 psutil-5.9.6 ptyprocess-0.7.0 pure-eval-0.2.2 pygments-2.17.2 python-dateutil-2.8.2 pyzmq-25.1.2 six-1.16.0 stack-data-0.6.3 tornado-6.4 traitlets-5.14.0 typing-extensions-4.9.0 wcwidth-0.2.12 zipp-3.17.0
WARNING: You are using pip version 22.0.4; however, version 23.3.1 is available.
You should consider upgrading via the '/Users/xuan_kai/.modular/pkg/packages.modular.com_mojo/venv/bin/python -m pip install --upgrade pip' command.
[mojo] Testing: `mojo build test_mandelbrot.mojo` [FAIL]
[mojo] Testing: `mojo build test_python.mojo` [FAIL]
[mojo] Testing: `mojo test_python.mojo` [FAIL]
[mojo] Testing: `mojo repl` [FAIL]
[mojo] Some components may have been installed successfully, but others may not work as expected. Please submit an issue to https://github.com/modularml/mojo and include the full output of the command you just ran.
modular: error: failed to run python: 

xuan_kai@MacBook-Pro-2 ~
  % mojo -V                                                               !5876
zsh: command not found: mojo

xuan_kai@MacBook-Pro-2 ~
  % /Users/xuan_kai/.modular/pkg/packages.modular.com_mojo/venv/bin/python -m pip install --upgrade pip                                                           !5877
Requirement already satisfied: pip in ./.modular/pkg/packages.modular.com_mojo/venv/lib/python3.9/site-packages (22.0.4)
Collecting pip
  Using cached pip-23.3.1-py3-none-any.whl (2.1 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 22.0.4
    Uninstalling pip-22.0.4:
      Successfully uninstalled pip-22.0.4
Successfully installed pip-23.3.1

xuan_kai@MacBook-Pro-2 ~
  % modular install mojo                                                                                                                                          !5878
# Found release for https://packages.modular.com/mojo @ 0.6.0
# Installing to /Users/xuan_kai/.modular/pkg/packages.modular.com_mojo
# Downloading artifacts. Please wait...
modular: error: failure detected - please run `modular clean` and try again
==========================
Failure Information:
    - Creating symlink for /Users/xuan_kai/.modular/pkg/packages.modular.com_mojo/lib/mblack/_internal/Python3.framework/Versions/3.9/Python3 -> /Users/xuan_kai/.modular/pkg/packages.modular.com_mojo/lib/mblack/_internal/Python3 failed: File exists
    - Creating symlink for /Users/xuan_kai/.modular/pkg/packages.modular.com_mojo/lib/liblldb.18.0.0git.dylib -> /Users/xuan_kai/.modular/pkg/packages.modular.com_mojo/lib/liblldb.dylib failed: File exists
    - Creating symlink for /Users/xuan_kai/.modular/pkg/packages.modular.com_mojo/lib/mblack/_internal/Python3.framework/Versions/3.9/Python3 -> /Users/xuan_kai/.modular/pkg/packages.modular.com_mojo/lib/mblack/_internal/Python3.framework/Python3 failed: File exists
==========================

xuan_kai@MacBook-Pro-2 ~
  % modular clean                                                                                                                                                 !5879

xuan_kai@MacBook-Pro-2 ~
  % modular install mojo                                                                                                                                          !5880
# Found release for https://packages.modular.com/mojo @ 0.6.0
# Installing to /Users/xuan_kai/.modular/pkg/packages.modular.com_mojo
# Downloading artifacts. Please wait...
# Downloads complete, setting configs...
# Configs complete, running post-install hooks...
Collecting find_libpython==0.3.0
  Using cached find_libpython-0.3.0-py3-none-any.whl (8.5 kB)
Collecting jupyter_client>=8.3.0
  Using cached jupyter_client-8.6.0-py3-none-any.whl (105 kB)
Collecting ipykernel>=6.25.0
  Using cached ipykernel-6.27.1-py3-none-any.whl (114 kB)
Collecting importlib-metadata>=4.8.3
  Using cached importlib_metadata-7.0.0-py3-none-any.whl (23 kB)
Collecting pyzmq>=23.0
  Using cached pyzmq-25.1.2-cp39-cp39-macosx_10_15_universal2.whl (1.9 MB)
Collecting traitlets>=5.3
  Using cached traitlets-5.14.0-py3-none-any.whl (85 kB)
Collecting jupyter-core!=5.0.*,>=4.12
  Using cached jupyter_core-5.5.0-py3-none-any.whl (28 kB)
Collecting python-dateutil>=2.8.2
  Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting tornado>=6.2
  Using cached tornado-6.4-cp38-abi3-macosx_10_9_x86_64.whl (431 kB)
Collecting packaging
  Using cached packaging-23.2-py3-none-any.whl (53 kB)
Collecting psutil
  Using cached psutil-5.9.6-cp36-abi3-macosx_10_9_x86_64.whl (246 kB)
Collecting appnope
  Using cached appnope-0.1.3-py2.py3-none-any.whl (4.4 kB)
Collecting ipython>=7.23.1
  Using cached ipython-8.18.1-py3-none-any.whl (808 kB)
Collecting nest-asyncio
  Using cached nest_asyncio-1.5.8-py3-none-any.whl (5.3 kB)
Collecting matplotlib-inline>=0.1
  Using cached matplotlib_inline-0.1.6-py3-none-any.whl (9.4 kB)
Collecting debugpy>=1.6.5
  Using cached debugpy-1.8.0-py2.py3-none-any.whl (5.0 MB)
Collecting comm>=0.1.1
  Using cached comm-0.2.0-py3-none-any.whl (7.0 kB)
Collecting zipp>=0.5
  Using cached zipp-3.17.0-py3-none-any.whl (7.4 kB)
Collecting exceptiongroup
  Using cached exceptiongroup-1.2.0-py3-none-any.whl (16 kB)
Collecting pexpect>4.3
  Using cached pexpect-4.9.0-py2.py3-none-any.whl (63 kB)
Collecting prompt-toolkit<3.1.0,>=3.0.41
  Using cached prompt_toolkit-3.0.42-py3-none-any.whl (385 kB)
Collecting jedi>=0.16
  Using cached jedi-0.19.1-py2.py3-none-any.whl (1.6 MB)
Collecting stack-data
  Using cached stack_data-0.6.3-py3-none-any.whl (24 kB)
Collecting pygments>=2.4.0
  Using cached pygments-2.17.2-py3-none-any.whl (1.2 MB)
Collecting decorator
  Using cached decorator-5.1.1-py3-none-any.whl (9.1 kB)
Collecting typing-extensions
  Using cached typing_extensions-4.9.0-py3-none-any.whl (32 kB)
Collecting platformdirs>=2.5
  Using cached platformdirs-4.1.0-py3-none-any.whl (17 kB)
Collecting six>=1.5
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting parso<0.9.0,>=0.8.3
  Using cached parso-0.8.3-py2.py3-none-any.whl (100 kB)
Collecting ptyprocess>=0.5
  Using cached ptyprocess-0.7.0-py2.py3-none-any.whl (13 kB)
Collecting wcwidth
  Using cached wcwidth-0.2.12-py2.py3-none-any.whl (34 kB)
Collecting asttokens>=2.1.0
  Using cached asttokens-2.4.1-py2.py3-none-any.whl (27 kB)
Collecting executing>=1.2.0
  Using cached executing-2.0.1-py2.py3-none-any.whl (24 kB)
Collecting pure-eval
  Using cached pure_eval-0.2.2-py3-none-any.whl (11 kB)
Installing collected packages: wcwidth, pure-eval, ptyprocess, find_libpython, appnope, zipp, typing-extensions, traitlets, tornado, six, pyzmq, pygments, psutil, prompt-toolkit, platformdirs, pexpect, parso, packaging, nest-asyncio, executing, exceptiongroup, decorator, debugpy, python-dateutil, matplotlib-inline, jupyter-core, jedi, importlib-metadata, comm, asttokens, stack-data, jupyter_client, ipython, ipykernel
Successfully installed appnope-0.1.3 asttokens-2.4.1 comm-0.2.0 debugpy-1.8.0 decorator-5.1.1 exceptiongroup-1.2.0 executing-2.0.1 find_libpython-0.3.0 importlib-metadata-7.0.0 ipykernel-6.27.1 ipython-8.18.1 jedi-0.19.1 jupyter-core-5.5.0 jupyter_client-8.6.0 matplotlib-inline-0.1.6 nest-asyncio-1.5.8 packaging-23.2 parso-0.8.3 pexpect-4.9.0 platformdirs-4.1.0 prompt-toolkit-3.0.42 psutil-5.9.6 ptyprocess-0.7.0 pure-eval-0.2.2 pygments-2.17.2 python-dateutil-2.8.2 pyzmq-25.1.2 six-1.16.0 stack-data-0.6.3 tornado-6.4 traitlets-5.14.0 typing-extensions-4.9.0 wcwidth-0.2.12 zipp-3.17.0
WARNING: You are using pip version 22.0.4; however, version 23.3.1 is available.
You should consider upgrading via the '/Users/xuan_kai/.modular/pkg/packages.modular.com_mojo/venv/bin/python -m pip install --upgrade pip' command.
[mojo] Testing: `mojo build test_mandelbrot.mojo` [FAIL]
[mojo] Testing: `mojo build test_python.mojo` [FAIL]
[mojo] Testing: `mojo test_python.mojo` [FAIL]
[mojo] Testing: `mojo repl` [FAIL]
[mojo] Some components may have been installed successfully, but others may not work as expected. Please submit an issue to https://github.com/modularml/mojo and include the full output of the command you just ran.
modular: error: failed to run python: 

xuan_kai@MacBook-Pro-2 ~
  %                                                                                                                                                               !5885

Sharktheone commented 9 months ago

This is not feasible, it says mojo is not installed

It says: "Some components may have been installed successfully, but others may not work as expected."

From my experience, normally even some of the tests fail, mojo is installed, even it might not work completely how it is supposed to be. So I suggest, try it even though some tests are failing.

Executing modular clean will automatically delete Modular Home dir

No, modular clean only deletes the ~/.modular/pkg/ directory. So try, as suggested, rm -r ~/.modular

peterzyk commented 9 months ago

I tried this and still have the same problem. I'm curious, am I the only one having this problem? I have the latest version of brew, etc. running on the m1 chip. I still think it's the low pip version in the official sources that triggers the issue.

Sharktheone commented 9 months ago

I see these tests failing quite often in install issues. For example #1378

zbowling commented 9 months ago

@modocache

peterzyk commented 9 months ago

Please fix the issues with the modular install Mojo

In other issue branches, I have also seen some problems and methods. However, it is clear that there is an issue with the resource package used during the modular install process of Mojo.

From the screenshot, it can be seen that the warnings are generated by pip in the ./.modular/pkg/packages.modular.com_mojo/venv/bin/ path, and are unrelated to the user's local Python environment.

I know there are some other methods to solve this problem, but they are extremely unfriendly to beginners. This is a problem with the modular system, so why should beginners have to solve it themselves?

modularml / mojo