search

LuxDL / MLDataDevices.jl

Data Transfer Functionalities across Backends for Machine Learning Applications
https://lux.csail.mit.edu/stable/
MIT License
2 stars 3 forks source link

make this package framework agnostic? #51

Closed CarloLucibello closed 2 months ago

CarloLucibello commented 3 months ago

I am working on making GNN.jl Lux compatible and something that could simplify the process is to be able to do data transfer across devices in a framework-agnostic way. So I was wondering if this package could be untied from Lux and serve this purpose. In general, I think it would be convenient for the whole ecosystem to have a reference package for data movement.

It seems that there is not much to be done to achieve this, just some debranding, e.g. "LuxDeviceUtils.jl -> DeviceUtils.jl", "LuxCUDADevice -> CUDADevice". Other things that I could spot are

avik-pal commented 3 months ago

Yes, I am open to the idea. This package was created with the idea of unifying the data movement in a similar spirit to https://github.com/LuxDL/WeightInitializers.jl but predates that, so the "branding" was present.

We should probably change the name (if that helps for wider adoption), though I think DeviceUtils.jl might have objections from the wider Julia community (for being too generic a name). Let's solicit opinions about this on Slack/zulip/discourse.

Certain things I would like to keep:

  1. LuxCore dep, as you mentioned, is very light, and it doesn't make much sense to move it to extensions
  2. The "- LuxCUDA.jl for NVIDIA CUDA Support." line can be changed to "- LuxCUDA.jl (alternatively both CUDA.jl and cuDNN.jl) for NVIDIA CUDA Support." -- I want the "LuxCUDA" in the text to avoid inconveniencing end users from loading both CUDA and cuDNN (and worse, missing out on one of the imports).

does full CUDA functionality require LuxCUDA? Maybe the LuxCUDA extension could become a CUDAcuDNN one?

Yeah, loading can be done via CUDA + cuDNN. The reason I mention LuxCUDA everywhere is to avoid problems like https://github.com/FluxML/Flux.jl/issues/2440.

CarloLucibello commented 3 months ago

Good! I can take care of this. Should it be done here or should we fork the package?

avik-pal commented 3 months ago

Do it here itself. I was anyways moving towards a 1.0 tag, so might as well rename it as a part of that change.