CUDA support on Windows

[cocoa-xu]

Let's try to bring CUDA support to Windows. This should address a previous issue #164.

• [x] CUDA support.
• [x] Precompiled binaries for x86_64-windows-msvc.

[cocoa-xu]

So now I can install CUDA 11.1.0, 11.4.3 and 11.8.0 with cocoa-xu/cuda-toolkit, forked from Jimver/cuda-toolkit.

- uses: cocoa-xu/cuda-toolkit@v0.2.9
  id: cuda-toolkit
  with:
    cuda: '11.8.0'

For dev versions, set these env vars in cmd

set EVISION_PREFER_PRECOMPILED="false"
set EVISION_ENABLE_CUDA="true"

And the compilation process is fine, but we need to either

• Copy all CUDA runtime DLL files to the priv/lib directory, which can be quite large in size (~1.89 GB, CUDA 11.8.0)

• Somehow detect the directory where all the CUDA runtime DLLs sit, and add it to the DLL search path.

  defmodule :evision_nif do
  @moduledoc false
  @on_load :load_nif
  def load_nif do
    require Logger
    nif_file = '#{:code.priv_dir(:evision)}/evision'
    :ok = 
      case :os.type() do
        {:win32, _} -> 
          DLLLoaderHelper.addDLLDirectory("#{:code.priv_dir(:evision)}")
          DLLLoaderHelper.addDLLDirectory("C:/PATH/TO/CUDA/DLLs/bin")
        _ -> :ok
      end
  
    case :erlang.load_nif(nif_file, 0) do
      :ok -> :ok
      {:error, {:reload, _}} -> :ok
      {:error, reason} -> Logger.warn("Failed to load nif: #{inspect(reason)}")
    end
  end
  end

[cocoa-xu]

It works either way, Windows no more complains about missing DLLs.

Since the CUDA support will be explicitly specified by the users, it seems to be okay to copy these DLLs to priv/lib (for users who use precompiled binaries).

For users who prefer compiling from source, maybe they can configure the path in config.exs, something like

config :evision, copy_cuda_runtime: false
config :evision, cuda_runtime_dir: "C:/CUDA"

and if they are okay with copying all the runtime DLLs to priv/lib, then

config :evision, copy_cuda_runtime: true

[cocoa-xu]

Impossible to download cudnn libraries on Windows runner.

[cocoa-xu]

Actually, I just came up with an idea to address the cudnn thing. More changes will be made in cocoa-xu/cuda-toolkit, and hopefully, the precompiled version for Windows can use cudnn.

[edwardzhou]

Actually, I just came up with an idea to address the cudnn thing. More changes will be made in cocoa-xu/cuda-toolkit, and hopefully, the precompiled version for Windows can use cudnn.

precompiled for windows is huge helpful. much painful to setup compilation environment on windows than on Linux/Mac.

[cocoa-xu]

v0.1.26 is shipped!

[edwardzhou]

@cocoa-xu update to 0.1.28 with GPU config

set EVISION_CUDA_VERSION=118
set EVISION_ENABLE_CUDA=true
set EVISION_ENABLE_CONTRIB=true

mix deps.get
mix compile

iex -S mix

iex(1)> Evision.CUDA.DeviceInfo.deviceInfo

15:54:33.532 [warning] Failed to load nif: {:load_failed, [70, 97, 105, 108, 101, 100, 32, 116, 111, 32, 108, 111, 97, 100, 32, 78, 73, 70, 32, 108, 105, 98, 114, 97, 114, 121, 32, 100, 58, 47, 101, 108, 105, 120, 105, 114, 95, 119, 111, 114, 107, 47, 101, 118, 105, 115, 105, 111, ...]}, d:/elixir_work/evision_demo/_build/dev/lib/evision/priv/evision ** (ErlangError) Erlang error: "cv::cuda::DeviceInfo::cuda_DeviceInfo not loaded" :erlang.nif_error("cv::cuda::DeviceInfo::cuda_DeviceInfo not loaded") (evision 0.1.28) lib/generated/evision_nif.ex:1741: :evision_nif.cuda_cuda_DeviceInfo_DeviceInfo/1 (evision 0.1.28) lib/generated/evision_cuda_deviceinfo.ex:77: Evision.CUDA.DeviceInfo.deviceInfo/0 iex:1: (file) iex(1)>

[cocoa-xu]

Hi, please set EVISION_CUDA_RUNTIME_DIR to the directory that contains CUDA + cudnn runtime DLL files. These files are not redistributed to reduce the size of the precompiled archive.

Related link: https://github.com/cocoa-xu/evision#important-note-for-windows-users