search

thomasbrandon / mish-cuda

Mish Activation Function for PyTorch
MIT License
147 stars 67 forks source link

Install Guide for PyTorch #1

Open glenn-jocher opened 4 years ago

glenn-jocher commented 4 years ago

Hello, I'd like to try to use your implementation of Mish in our repo: https://github.com/ultralytics/yolov3

I'm not sure exactly how to implement Mish-CUDA. I haven't done any CUDA programming, I usually rely on publically available pre-made environments like GCP PyTorch images, Google Colab, or Nvidia Docker images like nvcr.io/nvidia/pytorch:19.10-py3. Could you provide the installation steps required to get this working on one of these? Thanks!

thomasbrandon commented 4 years ago

You should be able to just pip install the repo (i.e. pip install git+https://github.com/thomasbrandon/mish-cuda/). The main thing that's needed is correct environment variables for the CUDA toolkit which should be done in GPU enabled docker images. You can install on colab this way though when I checked a while ago colab was running on PyTorch 1.1 while 1.2 was needed but hopefully this has now been updated. You can also use autograd functions to reduce memory usage without requiring a custom CUDA kernel and with the JIT script feature can achieve similar speed to the CUDA kernel with reduced memory usage (actually I have not fully verified this on Mish, only Swish, I will look to update this repo after full testing). You can find JIT and autograd implementations of Mish/Swish here.

dempti commented 3 years ago

@thomasbrandon I am trying to install the same. However, it keeps failing.

This is the configuration of the machine

NAME="Amazon Linux"
VERSION="2"
ID="amzn"
ID_LIKE="centos rhel fedora"
VERSION_ID="2"
PRETTY_NAME="Amazon Linux 2"
ANSI_COLOR="0;33"
CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2"
HOME_URL="https://amazonlinux.com/"

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 460.73.01    Driver Version: 460.73.01    CUDA Version: 11.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla T4            On   | 00000000:00:1E.0 Off |                    0 |
| N/A   30C    P8     9W /  70W |     28MiB / 15109MiB |      0%   E. Process |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A     28924      C   nvidia-cuda-mps-server             25MiB |
+-----------------------------------------------------------------------------+

/etc/docker/daemon.json

{
    "runtimes": {
        "nvidia": {
            "path": "nvidia-container-runtime",
            "runtimeArgs": []
        }
    },
    "default-runtime": "nvidia"
}

Error Logs

Creating a virtualenv for this project...
Pipfile: /usr/src/app/Pipfile
Using /usr/bin/python3.6m (3.6.9) to create virtualenv...
⠴ Creating virtual environment...created virtual environment CPython3.6.9.final.0-64 in 346ms
  creator CPython3Posix(dest=/root/.local/share/virtualenvs/app-lp47FrbD, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/root/.local/share/virtualenv)
    added seed packages: pip==21.2.4, setuptools==58.0.4, wheel==0.37.0
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator

✔ Successfully created virtual environment! 
Virtualenv location: /root/.local/share/virtualenvs/app-lp47FrbD
Installing dependencies from Pipfile...
An error occurred while installing matplotlib! Will try again.
An error occurred while installing pycocotools! Will try again.
An error occurred while installing -e git+https://github.com/thomasbrandon/mish-cuda/#egg=mish-cuda! Will try again.
Installing initially failed dependencies...
[InstallError]:   File "/usr/local/lib/python3.6/dist-packages/pipenv/cli/command.py", line 253, in install
[InstallError]:       site_packages=state.site_packages
[InstallError]:   File "/usr/local/lib/python3.6/dist-packages/pipenv/core.py", line 2063, in do_install
[InstallError]:       keep_outdated=keep_outdated
[InstallError]:   File "/usr/local/lib/python3.6/dist-packages/pipenv/core.py", line 1312, in do_init
[InstallError]:       pypi_mirror=pypi_mirror,
[InstallError]:   File "/usr/local/lib/python3.6/dist-packages/pipenv/core.py", line 900, in do_install_dependencies
[InstallError]:       retry_list, procs, failed_deps_queue, requirements_dir, **install_kwargs
[InstallError]:   File "/usr/local/lib/python3.6/dist-packages/pipenv/core.py", line 796, in batch_install
[InstallError]:       _cleanup_procs(procs, failed_deps_queue, retry=retry)
[InstallError]:   File "/usr/local/lib/python3.6/dist-packages/pipenv/core.py", line 703, in _cleanup_procs
[InstallError]:       raise exceptions.InstallError(c.dep.name, extra=err_lines)
[pipenv.exceptions.InstallError]: Obtaining mish-cuda from git+https://github.com/thomasbrandon/mish-cuda/#egg=mish-cuda (from -r /tmp/pipenv-tde2a1rb-requirements/pipenv-6knl80qd-requirement.txt (line 1))
[pipenv.exceptions.InstallError]:   Updating /root/.local/share/virtualenvs/app-lp47FrbD/src/mish-cuda clone
[pipenv.exceptions.InstallError]: Running command git fetch -q --tags
[pipenv.exceptions.InstallError]:   Running command git reset --hard -q c54271c725d57af62968e960598ffedd4896ef94
[pipenv.exceptions.InstallError]:     ERROR: Command errored out with exit status -11:
[pipenv.exceptions.InstallError]:      command: /root/.local/share/virtualenvs/app-lp47FrbD/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/root/.local/share/virtualenvs/app-lp47FrbD/src/mish-cuda/setup.py'"'"'; __file__='"'"'/root/.local/share/virtualenvs/app-lp47FrbD/src/mish-cuda/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-0y_7qlol
[pipenv.exceptions.InstallError]:          cwd: /root/.local/share/virtualenvs/app-lp47FrbD/src/mish-cuda/
[pipenv.exceptions.InstallError]:     Complete output (0 lines):
[pipenv.exceptions.InstallError]:     ----------------------------------------
[pipenv.exceptions.InstallError]: WARNING: Discarding git+https://github.com/thomasbrandon/mish-cuda/#egg=mish-cuda. Command errored out with exit status -11: python setup.py egg_info Check the logs for full command output.
[pipenv.exceptions.InstallError]: ERROR: Could not find a version that satisfies the requirement mish-cuda (unavailable) (from versions: none)
[pipenv.exceptions.InstallError]: ERROR: No matching distribution found for mish-cuda (unavailable)
ERROR: Couldn't install package: mish-cuda
 Package installation failed...
The command '/bin/sh -c pipenv install --skip-lock' returned a non-zero code: 1

Dockerfile

#FROM nvidia/cuda:10.0-base-ubuntu18.04
#FROM mish:cuda
FROM mish-cuda:2

RUN apt-get update && \
    apt-get -y install \
   libsm6 \
   libxext6 \
   libxrender-dev \
   cuda-toolkit-10-0 \
   python3.7 \
   python3.7-dev \
   libavformat-dev \
   libavcodec-dev \
   libavdevice-dev \
   libavutil-dev \
   libswscale-dev \
   libswresample-dev \
   libavfilter-dev \
   libass-dev \
   libfreetype6-dev \
   libjpeg-dev \
   libtheora-dev \
   libtool \
   libvorbis-dev \
   libx264-dev \
   libpq-dev \
   libsm6 \
   libxext6 \
   libxrender-dev \
   python3-pip \
   pkg-config \
   zlib1g-dev \
   unzip \
   yasm \
   git

ENV LC_ALL=C.UTF-8
ENV LANG=C.UTF-8

WORKDIR /usr/src/app
COPY ./Pipfile .

RUN pip3 install pipenv

RUN pipenv install --skip-lock

Pipfile

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"
[dev-packages]
black = "*"
[packages]
configparser = "==5.0.0"
pydot = "==1.4.1"
numpy = "==1.19"
opencv-contrib-python = "==4.1.0.25"
imutils = "==0.5.3"
torch= "==1.6"
torchvision= "==0.7.0"
pandas = "==1.1.4"
pillow = "==6.2.2"
boto3 = "1.18.40"
matplotlib = "3.3.4"
pycocotools = "2.0.2"
tqdm= "4.62.2"
tensorboard= ">= 1.14"
pyyaml="5.4.1"
fastapi= "==0.63.0"
uvicorn="==0.13.3"
python-multipart="==0.0.5"
scikit-learn="==0.21.3"
pymilvus="==0.4.0"
redis="==3.5.3"
umap-learn="==0.4.6"
numba="==0.52.0"
hdbscan = "==0.8.26"
tensorflow-gpu= "==1.14"
keras="==2.2.5"
mish-cuda = {git = "https://github.com/thomasbrandon/mish-cuda/", editable = true}
[requires]
python_version = "3.6"
[pipenv]
allow_prereleases = true