Setup Environment for EndoNerf

[mabel-lr]

Git https://github.com/med-air/EndoNeRF

Data set https://drive.google.com/drive/folders/1zTcX80c1yrbntY9c6-EK2W2UVESVEug8?usp=sharing

[guibo-tech]

Error, I could not install the modified packed torchsearchsorted from the repo in the miniconda environment.

Solution ? import it as following: run_endonerf_helpers.py from torchsearchsorted.src.torchsearchsorted.searchsorted import searchsorted

benchmark.py and test.py from torchsearchsorted.src.torchsearchsorted.searchsorted import searchsorted from torchsearchsorted.src.torchsearchsorted.utils import numpy_searchsorted

Solution ? Can I install it from another source? conda install -c pytorch torchsearchsorted

[guibo-tech]

Error: CUDA and PyTorch versions for GPU Make sure to install the CUDA and PyTorch versions that are suitable for the GPU. They used CUDA 10.2, but it is not suitable for my GeForce RTX 3060

Solution: Check the suitable version here https://pytorch.org/get-started/locally/

[guibo-tech]

It is not working for me. Here my setup:

NVIDIA GeForce RTX 3060 Miniconda environment Python 3.6 cudatoolkit 11.1.1 PyTorch 1.9.1

I performed the following step. But it take a very short time, and got some error, I am sure it is not correct. There are problems with GPU/cuda

These folders were created.

[mabel-lr]

Successfully installed on Ubuntu, but next time change PyTorch install settings. I got the solution from this forum: https://discuss.pytorch.org/t/nvidia-nvidia-rtx-a5000-with-cuda-capability-sm-86-is-not-compatible-with-the-current-pytorch-installation/150593 In requirements.txt remove the version of torch and torchvision, this will install CUDA version 11.7 -> this is the recommended line given in PyTorch webpage: pip3 install torch torchvision torchaudio

[mabel-lr]

Downloading Drive data I have tried using wget -O folder https://googledrive.com/host/[ID] and didn't work. The ID is the long string of characters after "folders/" in the URL.

I have read that using gdrive download [ID] works, so I downloaded gdrive (version 2.1.1. IMPORTANT, the most updated version doesn't have download command) doing so:

wget -c https://github.com/prasmussen/gdrive/releases/download/2.1.1/gdrive_2.1.1_linux_386.tar.gz -O - | tar -xz

chmod +x gdrive

export PATH=$PATH:/data//

then gdrive is installed but we need an authentication code from google

[guibo-tech]

Setting up Nvidia GPU for Ubuntu

• To use a GPU on a virtual machine (VM), you need to ensure that your VM software supports GPU passthrough. GPU passthrough allows the VM to directly access and utilize the GPU hardware. It could be a trick process, I decided to install Ubuntu in a partitioned disk.
• Check if Nvidia GPU is available (command "nvidia-smi"), if not, install Nvidia drive. For me it was necessary to disable secure boot (BIOS setting) to install the drive.

example NVIDIA GeForce RTX 3060

[guibo-tech]

Installation packed torchsearchsorted

Error on my Ubuntu

Solutions:

• install CUDA, add to the path ? https://github.com/ashutoshIITK/install_cuda_cudnn_ubuntu_20 https://github.com/med-air/EndoNeRF/tree/master/torchsearchsorted
• in the workstation endonerf env there is no cuda library, maybe I need cuda only in the system, not in environment

It worked on workstation kirk

[guibo-tech]

Including dataset to the local repository

on my Ubuntu. command to copy folder from source path to destination path:

_cp -r /home/guibo/repos/BIMAP-EndoNeRF/endonerf_sampledatasets /home/guibo/repos/EndoNeRF/

The -r flag is used to copy the folder and its contents recursively.

On Kirk workstation

[guibo-tech]

To generate an SSH key to include in your GitHub account

1 - Run the following command to generate a new SSH key: ssh-keygen -t rsa -b 4096 -C "your_github_email@gmail.com" 2 - save the SSH key. Press Enter to accept the default location (~/.ssh/id_rsa) 3 - insert passphrase for extra layer of security or just enter enter to avoid it 4 - The ssh-keygen command will generate the SSH key pair: a private key (id_rsa) and a public key (id_rsa.pub). 5 - command to display the public key: cat ~/.ssh/id_rsa.pub 6 - copy the key in the GitHub "SSH and GPG keys".

[guibo-tech]

Installing CUDA, PyTorch, for NVIDIA RTX A5000

conda config --add channels nvidia conda install cudatoolkit=11.1 conda install torch==1.10.1+cu111 torchvision==0.11.2+cu111 torchaudio==0.10.1 -f https://download.pytorch.org/whl/cu111/torch_stable.html

conda install torch torchvision torchaudio cudatoolkit=11.1 -c pytorch

[guibo-tech]

torchsearchsorted tests

Failing in my working station

[guibo-tech]

Nvidia Drive, Cuda and Torch installation guide

A good guide, check it https://github.com/mingyen066/Cuda-PyTorch-Installation-Guide

Installing Nvidia drive https://www.youtube.com/watch?v=VP-R7LNSJXA

I read it is better to install the drive via SUDO, than downloading the drive from Nvidia homepage check the drive for the GPU in the homepage: https://www.nvidia.com/download/index.aspx then install via ubuntu terminal: sudo apt update sudo apt upgrade sudo ubuntu-drivers list sudo apt install nvidia-driver- than reboot system sudo reboot

How to disable Secure Boot (Lenovo Legion 5 Pro), it was necessary to make Ubuntu identify the GPU https://www.youtube.com/watch?v=n6OBZubyqmY

To check the cuda version for the Nvidia https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/index.html

Install cuda toolkit for the GPU Check what is the cuda version, download the correct one for GPU https://developer.nvidia.com/cuda-toolkit-archive commands: $ wget https://developer.download.nvidia.com/compute/cuda/12.0.1/local_installers/cuda_12.0.1_525.85.12_linux.run $ sudo sh cuda_12.0.1_525.85.12_linux.run or make the downloaded CUDA toolkit package executable chmod +x cuda_12.0.1_525.85.12_linux.run

Run the CUDA toolkit installation command with the appropriate options sudo ./cuda_12.0.1_525.85.12_linux.run

While you are installing cuda, a prompt will be displayed, asking you whether to install "Driver", "CUDA Toolkit", "CUDA Samples",... The "Driver" here means "Nvidia Graphics Driver" Since we have already installed Driver in the previous step, do NOT install Driver (i.e., unselect Driver)

Add Cuda to Path

This command will add the export PATH and export LD_LIBRARY_PATH lines to the end of the ~/.bashrc file. After executing this command, the two lines will be appended to the ~/.bashrc file, which ensures that the CUDA binaries directory is added to the system's PATH and the CUDA library path is properly set.

_echo >> ~/.bashrc ' export PATH=/usr/local/cuda/bin:$PATH export LD_LIBRARY_PATH=$LD_LIBRARYPATH:/usr/local/cuda/lib64'

for the changes to take effect in your current terminal session run source ~/.bashrc

Install CuDNN: Download from homepage with login (https://developer.nvidia.com/rdp/cudnn-download) cudnn-local-repo-ubuntu2004-8.8.0.121_1.0-1_amd64.deb

install command _sudo dpkg -i cudnn-local-repo-ubuntu2004-8.8.0.121_1.0-1amd64.deb

To install the key, run this command: sudo cp /var/cudnn-local-repo-ubuntu2004-8.8.0.121/cudnn-local-A9E17745-keyring.gpg /usr/share/keyrings/

No to the the path /var/cudnn-local-repo-ubuntu2004-8.8.0.121 open a terminal and install the 3 packages: _sudo dpkg -i libcudnn8_8.8.0.121-1+cuda12.0_amd64.deb sudo dpkg -i libcudnn8-dev_8.8.0.121-1+cuda12.0_amd64.deb sudo dpkg -i libcudnn8-samples_8.8.0.121-1+cuda12.0amd64.deb

Test Cudnn

Change the directory to: _cd /usr/src/cudnn_samplesv8/mnistCUDNN Clean the previous build: sudo make clean Build the mnistCUDNN program: sudo make Run the mnistCUDNN program: ./mnistCUDNN

I got an error and needed to install freeimage library sudo apt-get install libfreeimage-dev

Verify installation: After installing the FreeImage library, ensure that the FreeImage.h header file is available in a directory that is included in the compiler's search path. You can verify this by checking the following locations: /usr/include/ /usr/local/include/

Repeat again the previous steps, it will run correctly and Test Passed for using Cudnn and Cuda.

Install Pytorch by pip https://www.youtube.com/watch?v=c0Z_ItwzT5o

check command here, mark stable - linux - pip - Python - CUDA 11.7 There is no Pytorch for cuda 12.0, as I googled, the version Pythorch CUDA 11.7 works for CUDA 12.0 https://pytorch.org/get-started/locally/

pip3 install torch torchvision torchaudio

Check whether PyTorch, torchvision, torchaudio is installed: $ python3 Python 3.8.10 (default, Mar 13 2023, 10:26:41) [GCC 9.4.0] on linux Type "help", "copyright", "credits" or "license" for more information.

import torch torch.rand(3) tensor([0.6327, 0.5860, 0.3513])

torch.cuda.is_available() True torch.cuda.device_count() 1 torch.cuda.current_device() 0 torch.cuda.device(0) <torch.cuda.device object at 0x7f86d8d4a7f0> torch.cuda.get_device_name(0) 'NVIDIA GeForce RTX 3060 Laptop GPU'

print(torch.version) 2.0.1+cu117 import torchvision print(torchvision.version) 0.15.2+cu117 import torchaudio print(torchaudio.version) 2.0.2+cu117

Install git sudo apt update sudo apt install git git --version

Install Miniconda Dowload from homepage https://docs.conda.io/en/latest/miniconda.html

_cd ~/Downloads chmod +x Miniconda3-py38_23.3.1-0-Linux-x86_64.sh ./Miniconda3-py38_23.3.1-0-Linux-x8664.sh close and open terminal conda --version

Installing the requirements for the Project Delete these lines from the requirements.txt. It is better to use the torch, torchvision from the system level. torch>=1.4.0 torchvision>=0.2.1

_cd EndoNeRF to get the correct pytorch and cuda I needed to create environment with Python 3.8 conda create -n endonerf _2 python=3.8

conda create -n endonerf python=3.6 conda activate endonerf pip install -r requirements.txt cd torchsearchsorted pip install . cd .._

torchsearchsorted package It is using data type long for numpy and torch. Numpy newer than 1.20 uses data type int64 instead.

NumPy versions prior to 1.20 used the data type 'long' to represent 64-bit integers. Starting from NumPy version 1.20, the 'long' data type was deprecated, and the equivalent data type 'int64' should be used instead.

If you specifically need to use the 'long' data type, you can install an older version of NumPy that supports it. Versions 1.19.x or earlier should have the 'long' data type available. For example, you can install NumPy version 1.19.5, which is the latest release in the 1.19.x series:

pip install numpy==1.19.5

[guibo-tech]

My last set up on my computer

OS: Ubuntu 20.04.6 LTS Processor: AMD® Ryzen 5 5600h with radeon graphics × 12 Graphic processor: NVIDIA GeForce RTX 3060 Laptop GPU Nvidia drive: 525.116.04, CUDA version: 12.0 CUDA Toolkit 12.0.1(January 2023), installed cuda_12.0.1_525.85.12_linux.run

CuDNN: libcudnn8_8.8.0.121-1+cuda12.0_amd64.deb libcudnn8-dev_8.8.0.121-1+cuda12.0_amd64.deb libcudnn8-samples_8.8.0.121-1+cuda12.0_amd64.deb

Miniconda: Python 3.8 | Miniconda3 Linux 64-bit

Python 3.8 Miniconda3 Linux 64-bit

Nividia drive

CUDA

Cudnn

Test passed for Cudnn

Pytorch, torchvision, torchaudio installed and tested for cuda

torchsearchsorted package

benchmark.py

test.py

test_searchsorted.py

Sometimes all 324 tests passes, sometimes 1 fails, sometimes 2 fails.

[guibo-tech]

Installation torchsearchsorted

Even with the environment pretty well set up, it failed to install it with CUDA 12.0, due mismatch cuda 12.0 in system with Pytorch CUDA 11.7

I create env endonerf_2 python 3.8 and downgraded the CUDA 12.0 in system to the same version as PyTorch Cuda 11.7. Also installed Cudnn for cuda 11x

Is the Cuda 11.7 suitable for Geforce RTX 3060?

Yes, it worked and I could install the package downgrading to CUDA 11.7 But this package seems deprecated

[guibo-tech]

endonerf_2 environment

torch 2.0.1 torchaudio 2.0.2 torchsearchsorted 1.1 torchvision 0.15.2 imageio 2.28.1 imageio-ffmpeg 0.4.8 matplotlib 3.7.1 ConfigArgParse 1.5.3 tensorboard 2.13.0 tqdm 4.65.0 opencv-python 4.7.0.72 PyMCubes 0.1.4 trimesh 3.21.6 jupyter 1.0.0 lpips 0.1.4 open3d 0.17.0

[guibo-tech]

.

[guibo-tech]

OS: Ubuntu 20.04.6 LTS Processor: AMD® Ryzen 5 5600h with radeon graphics × 12 Graphic processor: NVIDIA GeForce RTX 3060 Laptop GPU 14GB NVIDIA-SMI 525.125.06 Driver Version: 525.125.06 CUDA Version: 12.0

CUDA Toolkit 11.7 Built on Wed_Jun__8_16:49:14_PDT_2022 Cuda compilation tools, release 11.7, V11.7.99 Build cuda_11.7.r11.7/compiler.31442593_0

conda environment torch 2.0.1 torchaudio 2.0.2 torchsearchsorted 1.1 torchvision 0.15.2 imageio 2.28.1 imageio-ffmpeg 0.4.8 matplotlib 3.7.1 ConfigArgParse 1.5.3 tensorboard 2.13.0 tqdm 4.65.0 opencv-python 4.7.0.72 PyMCubes 0.1.4 trimesh 3.21.6 jupyter 1.0.0 lpips 0.1.4 open3d 0.17.0