search

bodo-ai / Bodo-examples

Other
23 stars 11 forks source link

Installation/runtime Issues with jupyterlab 3.X #38

Open willyyang opened 2 years ago

willyyang commented 2 years ago

Issue

Hi, I'm having issues installing and using Bodo in JupyterLab 3.X.

I've tried installing bodo via conda but that doesn't work. I've tried the sample here: https://github.com/Bodo-inc/Bodo-examples/blob/master/docker/BodoNotebook.Dockerfile but still unable to resolve the conda bodo package. 

[+] Building 15.8s (5/5) FINISHED
 => [internal] load build definition from dockerfile                                                                                                                                        0.0s
 => => transferring dockerfile: 207B                                                                                                                                                        0.0s
 => [internal] load .dockerignore                                                                                                                                                           0.0s
 => => transferring context: 2B                                                                                                                                                             0.0s
 => [internal] load metadata for docker.io/jupyter/minimal-notebook:latest                                                                                                                  1.1s
 => CACHED [1/2] FROM docker.io/jupyter/minimal-notebook:latest@sha256:cfeab9b91dfce03d9be1683f9d4728860c30757c593f9b9277da4ce7d1a4e7f3                                                     0.0s
 => ERROR [2/2] RUN conda install -y bodo ipyparallel -c bodo.ai -c conda-forge                                                                                                            14.7s
------
 > [2/2] RUN conda install -y bodo ipyparallel -c bodo.ai -c conda-forge:
#5 0.684 Collecting package metadata (current_repodata.json): ...working... done
#5 5.036 Solving environment: ...working... failed with initial frozen solve. Retrying with flexible solve.
#5 5.037 Collecting package metadata (repodata.json): ...working... done
#5 13.57 Solving environment: ...working... failed with initial frozen solve. Retrying with flexible solve.
#5 13.58
#5 13.58 PackagesNotFoundError: The following packages are not available from current channels:
#5 13.58
#5 13.58   - bodo
#5 13.58
#5 13.58 Current channels:
#5 13.58
#5 13.58   - https://conda.anaconda.org/bodo.ai/linux-aarch64
#5 13.58   - https://conda.anaconda.org/bodo.ai/noarch
#5 13.58   - https://conda.anaconda.org/conda-forge/linux-aarch64
#5 13.58   - https://conda.anaconda.org/conda-forge/noarch
#5 13.58
#5 13.58 To search for alternate channels that may provide the conda package you're
#5 13.58 looking for, navigate to
#5 13.58
#5 13.58     https://anaconda.org
#5 13.58
#5 13.58 and use the search bar at the top of the page.
#5 13.58
#5 13.58
------

I then proceeded to install ipyparallel and bodo via pip and have tried setting up the jupyterlab as per instructions: https://docs.bodo.ai/latest/source/installation_and_setup/ipyparallel.html#ipyparallelsetup

However, after running the sample code there are import issues with the bodo package (see last notebook cell output) 

# input
import ipyparallel as ipp

import psutil; n = min(psutil.cpu_count(logical=False), 8)

rc = ipp.Cluster(engines='mpi', n=n).start_and_connect_sync(activate=True)
# result
Using existing profile dir: '/opt/app-root/src/.ipython/profile_default'

Starting 8 engines with <class 'ipyparallel.cluster.launcher.MPIEngineSetLauncher'>

100%

8/8 [00:12<00:00, 12.02s/engine]
# Input
%%px

from mpi4py import MPI

comm = MPI.COMM_WORLD

print(f"Hello World from rank {comm.Get_rank()}. total ranks={comm.Get_size()}")
# result
[stdout:0] Hello World from rank 0. total ranks=8

[stdout:1] Hello World from rank 1. total ranks=8

[stdout:7] Hello World from rank 7. total ranks=8

[stdout:4] Hello World from rank 4. total ranks=8

[stdout:6] Hello World from rank 6. total ranks=8

[stdout:2] Hello World from rank 2. total ranks=8

[stdout:5] Hello World from rank 5. total ranks=8

[stdout:3] Hello World from rank 3. total ranks=8
# input
import bodo

​

# result
---------------------------------------------------------------------------

ImportError                               Traceback (most recent call last)

<ipython-input-6-7b53d0178f92> in <module>

----> 1 import bodo

/opt/conda/lib/python3.8/site-packages/bodo/__init__.py in <module>

     20 from numba.core.types import List

     21 import bodo.libs

---> 22 import bodo.libs.distributed_api

     23 import bodo.libs.timsort

     24 import bodo.io

/opt/conda/lib/python3.8/site-packages/bodo/libs/distributed_api.py in <module>

     20 from numba.parfors.array_analysis import ArrayAnalysis

     21 import bodo

---> 22 from bodo.hiframes.datetime_date_ext import datetime_date_array_type

     23 from bodo.hiframes.datetime_timedelta_ext import datetime_timedelta_array_type

     24 from bodo.hiframes.pd_categorical_ext import CategoricalArrayType

/opt/conda/lib/python3.8/site-packages/bodo/hiframes/datetime_date_ext.py in <module>

     17 import bodo

     18 from bodo.hiframes.datetime_datetime_ext import DatetimeDatetimeType

---> 19 from bodo.hiframes.datetime_timedelta_ext import datetime_timedelta_type

     20 from bodo.libs import hdatetime_ext

     21 from bodo.utils.indexing import array_getitem_bool_index, array_getitem_int_index, array_getitem_slice_index, array_setitem_bool_index, array_setitem_int_index, array_setitem_slice_index

/opt/conda/lib/python3.8/site-packages/bodo/hiframes/datetime_timedelta_ext.py in <module>

     15 import bodo

     16 from bodo.hiframes.datetime_datetime_ext import datetime_datetime_type

---> 17 from bodo.libs import hdatetime_ext

     18 from bodo.utils.indexing import get_new_null_mask_bool_index, get_new_null_mask_int_index, get_new_null_mask_slice_index, setitem_slice_index_null_bits

     19 from bodo.utils.typing import BodoError, get_overload_const_str, is_iterable_type, is_list_like_index_type, is_overload_constant_str

ImportError: libmpi-badaf374.so.12.1.8: cannot open shared object file: No such file or directory

Packages

(2022-01-27 20:29:29) (/opt/conda/lib/python3.8/site-packages/bodo/libs)

-$ pip list | grep -i bodo

bodo 2021.10.1

(2022-01-27 20:30:37) (/opt/conda/lib/python3.8/site-packages/bodo/libs)

-$ pip list | grep -i ipyparallel

ipyparallel 8.1.0

(2022-01-27 20:30:44) (/opt/conda/lib/python3.8/site-packages/bodo/libs)

-$ pip list | grep -i jupyterlab

jupyterlab 3.2.8

sahil1105 commented 2 years ago

Hi @willyyang, thanks for trying Bodo and reporting this issue.

I wasn't able to replicate the Docker build issue:

 % docker build --file BodoNotebook.Dockerfile --tag bodo-notebook:latest .
[+] Building 454.8s (7/7) FINISHED                                                                                                          
 => [internal] load build definition from BodoNotebook.Dockerfile                                                                      0.0s
 => => transferring dockerfile: 193B                                                                                                   0.0s
 => [internal] load .dockerignore                                                                                                      0.0s
 => => transferring context: 2B                                                                                                        0.0s
 => [internal] load metadata for docker.io/jupyter/minimal-notebook:latest                                                             1.7s
 => [auth] jupyter/minimal-notebook:pull token for registry-1.docker.io                                                                0.0s
 => [1/2] FROM docker.io/jupyter/minimal-notebook:latest@sha256:cfeab9b91dfce03d9be1683f9d4728860c30757c593f9b9277da4ce7d1a4e7f3      81.4s
 => => resolve docker.io/jupyter/minimal-notebook:latest@sha256:cfeab9b91dfce03d9be1683f9d4728860c30757c593f9b9277da4ce7d1a4e7f3       0.0s
 => => sha256:cfeab9b91dfce03d9be1683f9d4728860c30757c593f9b9277da4ce7d1a4e7f3 743B / 743B                                             0.0s
 => => sha256:e659268a4bcf642996a505b2c1b64ed3fadf94f2f0727eb732ea566e8c9ef8a8 11.64kB / 11.64kB                                       0.0s
 => => sha256:ea362f368469f909a95f9a6e54ebe0121ce0a8e3c30583dd9c5fb35b14544dec 28.57MB / 28.57MB                                      11.9s
 => => sha256:41fcd07744d8934e689ba82ac81c7b2eed3f46484802efdd5c423a07885fd785 3.04kB / 3.04kB                                         0.0s
 => => sha256:1c2cc406fa63e6931b0cfb124173fdcbea06809c6c42d4cfb25b8874b1463a1e 35.26MB / 35.26MB                                      17.0s
 => => sha256:669dbb3f63668a4c5a267e3083fe974cec496c9a97e74944529b45919cdc5f1e 684B / 684B                                             0.4s
 => => sha256:3923fd8d3607224bdebba09e8de3a3819a41e9bd0eab252513da33fbf6dc9d07 1.92kB / 1.92kB                                         0.8s
 => => sha256:d5da8a60f05b52ed80577aab05f648559b50873d90a47175cbc679a0f60aa6be 4.54kB / 4.54kB                                         0.9s
 => => sha256:78722f61450a56f31e00e788c283bc8abc244f98f4871fbf4b7a76627316c2f8 162B / 162B                                             1.2s
 => => sha256:92fcb4043b45484602772354f77af6a231181f6bd530b19244d38c8f55715211 87.35MB / 87.35MB                                      33.6s
 => => extracting sha256:ea362f368469f909a95f9a6e54ebe0121ce0a8e3c30583dd9c5fb35b14544dec                                              1.1s
 => => sha256:e43cca32fa728bd1bbec630a2230571fff2ee1be338ce20fc55e8615bfab9ff4 88.56MB / 88.56MB                                      43.0s
 => => sha256:d6e059957f764282e558930ce0479d40d0862c68d5139bbe7080624bfe4f69d3 4.00kB / 4.00kB                                        17.0s
 => => extracting sha256:1c2cc406fa63e6931b0cfb124173fdcbea06809c6c42d4cfb25b8874b1463a1e                                              1.7s
 => => sha256:d4c1ac4b2556b688d1ee8bc220981c24028a4f2bb0677bfb4fb97060a4929956 991B / 991B                                            17.2s
 => => sha256:662cec5dfc0cdd6056b55d2a62f1e5a84a15a4a1489e6caaab672c1e5e4124d1 1.07kB / 1.07kB                                        17.5s
 => => sha256:2c1cefff5d33968205bf89bb11cee87f4c944712ea62ecacb0f1201cc67f8665 235.96MB / 235.96MB                                    69.0s
 => => extracting sha256:669dbb3f63668a4c5a267e3083fe974cec496c9a97e74944529b45919cdc5f1e                                              0.0s
 => => extracting sha256:3923fd8d3607224bdebba09e8de3a3819a41e9bd0eab252513da33fbf6dc9d07                                              0.0s
 => => extracting sha256:d5da8a60f05b52ed80577aab05f648559b50873d90a47175cbc679a0f60aa6be                                              0.0s
 => => extracting sha256:78722f61450a56f31e00e788c283bc8abc244f98f4871fbf4b7a76627316c2f8                                              0.0s
 => => sha256:759c927f13aa8f4534dc633ddb6e49a2de8316a91b360d0a9b4b8a28e16576e1 1.47kB / 1.47kB                                        33.9s
 => => extracting sha256:92fcb4043b45484602772354f77af6a231181f6bd530b19244d38c8f55715211                                              3.9s
 => => extracting sha256:e43cca32fa728bd1bbec630a2230571fff2ee1be338ce20fc55e8615bfab9ff4                                              5.0s
 => => extracting sha256:d6e059957f764282e558930ce0479d40d0862c68d5139bbe7080624bfe4f69d3                                              0.0s
 => => extracting sha256:d4c1ac4b2556b688d1ee8bc220981c24028a4f2bb0677bfb4fb97060a4929956                                              0.0s
 => => extracting sha256:662cec5dfc0cdd6056b55d2a62f1e5a84a15a4a1489e6caaab672c1e5e4124d1                                              0.0s
 => => extracting sha256:2c1cefff5d33968205bf89bb11cee87f4c944712ea62ecacb0f1201cc67f8665                                             11.9s
 => => extracting sha256:759c927f13aa8f4534dc633ddb6e49a2de8316a91b360d0a9b4b8a28e16576e1                                              0.0s
 => [2/2] RUN conda install -y bodo=2021.9 ipyparallel -c bodo.ai -c conda-forge                                                     364.3s
 => exporting to image                                                                                                                 7.4s 
 => => exporting layers                                                                                                                7.4s
 => => writing image sha256:e29c12b61ad24c167d27c4e2a17d4f0c9a3f513b131f20186735f0f1b4bbaa0d                                           0.0s
 => => naming to docker.io/library/bodo-notebook:latest                                                                                0.0s

From your logs, it seems like you didn't specify a Bodo Version in the conda install command in your Dockerfile.

https://github.com/Bodo-inc/Bodo-examples/blob/12a4f6e1afe942fdc2dd42dd0bc883c596bb3784/docker/BodoNotebook.Dockerfile#L4

Could you try doing this? Let us know if you face any issues.

willyyang commented 2 years ago

hey @sahil1105 thanks for the prompt response. Yes sorry I must've pasted incorrectly but still facing same issue.

docker build -f dockerfile . -t test
[+] Building 13.6s (5/5) FINISHED
 => [internal] load build definition from dockerfile                                                                                                                                        0.0s
 => => transferring dockerfile: 659B                                                                                                                                                        0.0s
 => [internal] load .dockerignore                                                                                                                                                           0.0s
 => => transferring context: 2B                                                                                                                                                             0.0s
 => [internal] load metadata for docker.io/jupyter/minimal-notebook:latest                                                                                                                  1.0s
 => CACHED [1/2] FROM docker.io/jupyter/minimal-notebook:latest@sha256:cfeab9b91dfce03d9be1683f9d4728860c30757c593f9b9277da4ce7d1a4e7f3                                                     0.0s
 => ERROR [2/2] RUN conda install -y bodo=2021.9 ipyparallel -c bodo.ai -c conda-forge                                                                                                     12.6s
------
 > [2/2] RUN conda install -y bodo=2021.9 ipyparallel -c bodo.ai -c conda-forge:
#5 0.732 Collecting package metadata (current_repodata.json): ...working... done
#5 2.979 Solving environment: ...working... failed with initial frozen solve. Retrying with flexible solve.
#5 2.980 Collecting package metadata (repodata.json): ...working... done
#5 11.58 Solving environment: ...working... failed with initial frozen solve. Retrying with flexible solve.
#5 11.58
#5 11.58 PackagesNotFoundError: The following packages are not available from current channels:
#5 11.58
#5 11.58   - bodo=2021.9
#5 11.58
#5 11.58 Current channels:
#5 11.58
#5 11.58   - https://conda.anaconda.org/bodo.ai/linux-aarch64
#5 11.58   - https://conda.anaconda.org/bodo.ai/noarch
#5 11.58   - https://conda.anaconda.org/conda-forge/linux-aarch64
#5 11.58   - https://conda.anaconda.org/conda-forge/noarch
#5 11.58
#5 11.58 To search for alternate channels that may provide the conda package you're
#5 11.58 looking for, navigate to
#5 11.58
#5 11.58     https://anaconda.org
#5 11.58
#5 11.58 and use the search bar at the top of the page.
#5 11.58
#5 11.58
------
executor failed running [/bin/bash -o pipefail -c conda install -y bodo=2021.9 ipyparallel -c bodo.ai -c conda-forge]: exit code: 1

This should be an exact c&p of https://github.com/Bodo-inc/Bodo-examples/blob/master/docker/BodoNotebook.Dockerfile

willyyang commented 2 years ago

And any feedback regarding the bodo pip installation then used in jupyterlab ? Fetching bodo 2021.10.1 via pip then running import bodo results in the above error.. Hoping that issue can be addressed, thanks.

sahil1105 commented 2 years ago

And any feedback regarding the bodo pip installation then used in jupyterlab ? Fetching bodo 2021.10.1 via pip then running import bodo results in the above error.. Hoping that issue can be addressed, thanks.

Hi, this issue is due to the mpi4py_mpich version and should be resolved in the future versions of our pip package. Thanks for bringing it to our attention. For now could you try it with pip install mpi4py-mpich==3.0.3 ?

sahil1105 commented 2 years ago

hey @sahil1105 thanks for the prompt response. Yes sorry I must've pasted incorrectly but still facing same issue.

docker build -f dockerfile . -t test
[+] Building 13.6s (5/5) FINISHED
 => [internal] load build definition from dockerfile                                                                                                                                        0.0s
 => => transferring dockerfile: 659B                                                                                                                                                        0.0s
 => [internal] load .dockerignore                                                                                                                                                           0.0s
 => => transferring context: 2B                                                                                                                                                             0.0s
 => [internal] load metadata for docker.io/jupyter/minimal-notebook:latest                                                                                                                  1.0s
 => CACHED [1/2] FROM docker.io/jupyter/minimal-notebook:latest@sha256:cfeab9b91dfce03d9be1683f9d4728860c30757c593f9b9277da4ce7d1a4e7f3                                                     0.0s
 => ERROR [2/2] RUN conda install -y bodo=2021.9 ipyparallel -c bodo.ai -c conda-forge                                                                                                     12.6s
------
 > [2/2] RUN conda install -y bodo=2021.9 ipyparallel -c bodo.ai -c conda-forge:
#5 0.732 Collecting package metadata (current_repodata.json): ...working... done
#5 2.979 Solving environment: ...working... failed with initial frozen solve. Retrying with flexible solve.
#5 2.980 Collecting package metadata (repodata.json): ...working... done
#5 11.58 Solving environment: ...working... failed with initial frozen solve. Retrying with flexible solve.
#5 11.58
#5 11.58 PackagesNotFoundError: The following packages are not available from current channels:
#5 11.58
#5 11.58   - bodo=2021.9
#5 11.58
#5 11.58 Current channels:
#5 11.58
#5 11.58   - https://conda.anaconda.org/bodo.ai/linux-aarch64
#5 11.58   - https://conda.anaconda.org/bodo.ai/noarch
#5 11.58   - https://conda.anaconda.org/conda-forge/linux-aarch64
#5 11.58   - https://conda.anaconda.org/conda-forge/noarch
#5 11.58
#5 11.58 To search for alternate channels that may provide the conda package you're
#5 11.58 looking for, navigate to
#5 11.58
#5 11.58     https://anaconda.org
#5 11.58
#5 11.58 and use the search bar at the top of the page.
#5 11.58
#5 11.58
------
executor failed running [/bin/bash -o pipefail -c conda install -y bodo=2021.9 ipyparallel -c bodo.ai -c conda-forge]: exit code: 1

This should be an exact c&p of https://github.com/Bodo-inc/Bodo-examples/blob/master/docker/BodoNotebook.Dockerfile

Hmm, this is very weird. As you can see in my previous reply, the same thing worked on my machine, and the base image seems to be exactly the same as well. What type of machine are you using? It might be related to the CPU generation.

willyyang commented 2 years ago

Looks like the pip install fixed things, will give it more of a thorough test thanks!

As for machine running the docker build, I've tried on both a 2019 mac running big sur and M1 running Monterey.

Quick question regarding the channel used for installation, should this package be available -c conda-forge ? I'm asking because I'm going to need to run this through my company conda channel mirrors.

ritwika314 commented 2 years ago

Hi @willyyang, thanks for the update! Would you be open to joining our community slack : https://join.slack.com/t/bodocommunity/shared_invite/zt-qwdc8fad-6rZ8a1RmkkJ6eOX1X__knA / to continue the conversation further there? Thanks!