search

JuliaLang / julia

The Julia Programming Language
https://julialang.org/
MIT License
45.75k stars 5.49k forks source link

Proposal for the JuliaDL umbrella organizations #13533

Closed pluskid closed 9 years ago

pluskid commented 9 years ago

Sorry if this should really go to the julia-users mailing list. But it is easier to at authors of existing packages here on github. I want to initiate the discussion of the possibility of creating a JuliaDL umbrella organization, if enough people are interested in.

Currently we have several packages in Julia dedicated to deep learning (please comment if I missed any):

I think given the number of stars Mocha.jl received on github (3rd most star-ed Julia package, following IJulia and Gadfly.jl), there is no doubt that a fair amount of Julia users and developers have some interest in deep learning, not to mention that deep learning is becoming extremely important in machine learning and data analysis outside the Julia world.

So I'm expecting there will be further exciting development in deep learning related stuff in Julia, and I think it might be useful to have an umbrella organization. I think the goal of such an organization will be several folds:

Just for examples, I include several pointers here

tbreloff commented 9 years ago

I actually thought there was a machine learning org already, but maybe not. I propose the name JuliaLearn, and I'd like to be included (@tbreloff). I also think it should include more than just "deep learning"... Possibly a superset of the functionality in python's scikit-learn?

On Saturday, October 10, 2015, Chiyuan Zhang notifications@github.com wrote:

Sorry if this should really go to the julia-users mailing list. But it is easier to at authors of existing packages here on github. I want to initiate the discussion of the possibility of creating a JuliaDL umbrella organization, if enough people are interested in.

Currently we have several packages in Julia dedicated to deep learning (please comment if I missed any):

I think given the number of stars Mocha.jl received on github (3rd most star-ed Julia package, following IJulia and Gadfly.jl), there is no doubt that a fair amount of Julia users and developers have some interest in deep learning, not to mention that deep learning is becoming extremely important in machine learning and data analysis outside the Julia world.

So I'm expecting there will be further exciting development in deep learning related stuff in Julia, and I think it might be useful to have an umbrella organization. I think the goal of such an organization will be several folds:

  • Keep the mature existing packages within the umbrella, or at least have a centralized indexing into existing packages. I have to say none of the names in the packages mentioned above are very informative as their relation to deep neural networks.
  • Have a centralized place to discuss general plans and roadmaps for deep learning in Julia. Maybe also encourage collaboration and knowledge sharing, for a better future solution of deep learning in Julia. And when people want to jump in and contribute they will have an idea of the current status and identify interesting future projects.
  • It could also possibly raise more attention when we want to discuss about better collaboration with other parts of Julia. For example, deep learning is closely related to @JuliaStats https://github.com/JuliaStats (or maybe even should be a sub-category), as well as @JuliaGPU https://github.com/JuliaGPU, and maybe @JuliaOpt https://github.com/JuliaOpt.

Just for examples, I include several pointers here

  • Since almost all the non-trivial deep learning experiments need to run on GPUs, having a feature-complete Array type that lives on GPU memory is very important for both library developer and end users. Currently, at least Matlab, Python (e.g. via pycuda) and Lua (e.g. cutorch https://github.com/torch/cutorch) have mature support for manipulating arrays on GPU directly. Currently we have CUDArt.jl https://github.com/JuliaGPU/CUDArt.jl, and related packages CUBLAS.jl https://github.com/JuliaGPU/CUBLAS.jl. But a lot of operations are still not supported. Julia already have built-in support for distributed array types. I think a good GPU Array type will be very nice to people even outside the deep learning world.
  • Theano http://deeplearning.net/software/theano/ is a very popular python tool for building deep learning libraries. It has automatic differentiation, so that you simply need to write your network in standard numpy expressions, and it will get compiled into computation code that could be executed on GPU very efficiently. It is very flexible and convenient to use, especially for researchers who needs to try a lot of non-standard operations and architectures. Quite a while (OK, 3 years) ago, there was a discussion in the mailing list https://groups.google.com/forum/#!msg/julia-dev/21JSSm8Z-gQ/IbrjGwRfZOYJ about implementing a Theano equivalent in Julia. Julia seem to be an ideal platform to implement such stuff with its support for powerful macros and meta-programming, as well as LLVM backends. But yet it is not happening yet. I just want to keep this pointer here, so that when some people with relevant knowledge and interest could pick up this if the y want t o contribute. Just to mention, there is a successor project of Theano, called CGT http://rll.berkeley.edu/cgt/, though still in Python.
  • Several month ago, another very exciting thread https://groups.google.com/forum/#!searchin/julia-dev/maleadt/julia-dev/doQUqQabC8A/mbDe8DSd2ykJ was posted in the mailing list about some preliminary support of compiling Julia code directly to GPU devices. This means we could write GPU kernels (functions that runs on GPU) directly in Julia. For example, currently a large amount of code in Mocha.jl is still written in C++, because we need to write CUDA kernels in C++. I'm not blaming C++, I think it could be enjoyable experience once Cxx.jl https://github.com/Keno/Cxx.jl is stable enough to run on Julia releases. But currently everything needs to go through a C API layer and it becomes painful when you want to support type generics for different floating point sizes, for example. Also, just to mention, you can already compile python function to GPU http://docs.continuum.io/numbapro/CUDAJit with CUDAJit now.

— Reply to this email directly or view it on GitHub https://github.com/JuliaLang/julia/issues/13533.

tkelman commented 9 years ago

This isn't the right place for this. There's already a machine learning meta/roadmap issue somewhere. Not saying you shouldn't go ahead with this (though DL is not a universally unambiguous acronym for deep learning that people outside the field would recognize), but I am closing this as not actionable within base julia.

IainNZ commented 9 years ago

https://github.com/JuliaStats/Roadmap.jl

tkelman commented 9 years ago

ref https://github.com/JuliaStats/Roadmap.jl/issues/11

Go ahead and start an org with an empty repo for discussion where interested parties can have a more focused discussion, and advertise it via a link from here, that issue, and/or the mailing lists.

pluskid commented 9 years ago

@tkelman For general machine learning, we already have @JuliaStats.

stokasto commented 9 years ago

Hey, @pluskid good collection of links, I think a discussion can/should definitely be had about this. As @tkelman said this is is probably not the right place for it :) . Maybe just start a discussion repo, copy your comment and reference it here and in the JuliaStats roadmap. I'll be happy to add my 2 cents to the discussion later on.

pluskid commented 9 years ago

@IainNZ @tkelman @stokasto Thanks! I already copy & pasted the text to the JuliaStats roadmap issue.

StefanKarpinski commented 9 years ago

though DL is not a universally unambiguous acronym for deep learning that people outside the field would recognize

I assumed it stood for "Julia on the Down Low" and thought someone was proposing opening a Julia-themed speakeasy. Needless to say, I would be in favor of such a proposal.