search

hughperkins / DeepCL

OpenCL library to train deep convolutional neural networks
Mozilla Public License 2.0
865 stars 199 forks source link

motive for DeepCL #68

Closed NEELMCW closed 8 years ago

NEELMCW commented 8 years ago

Just curious to know what DeepCL is for ? does it stand as a counterpart to NVIDIA's cuDNN library ?

Dexdev08 commented 8 years ago

For me i was looking for an open cl deep learning library -- how did you stumble upon the project?

On Mon, 16 May 2016 at 15:34 NEELMCW notifications@github.com wrote:

Just curious to know what DeepCL is for ? does it stand as a counterpart to NVIDIA's cuDNN library ?

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/hughperkins/DeepCL/issues/68

hughperkins commented 8 years ago

@NEELMCW It's a neural network framework, similar to caffe, but targeted specifically at OpenCL.

NEELMCW commented 8 years ago

How different is deepCL compared to cltorch, clnn etc of yours and OpenCL-caffe from AMD?
Does it outperform all of these ? Is there an equivalent of cuDNN out there ? I was looking for an OpenCL support for Tensorflow that's how I bumped into this thread

hughperkins commented 8 years ago

I was looking for an OpenCL support for Tensorflow that's how I bumped into this thread

Ah. Well. Tensorflow is compilable with gpucc, per my understanding, and it has pluggable backends to compile as cuda, opencl, etc. Currently I dont think there is a pluggable backend for opencl, but google has one planned, per slides from november. It probably wouldnt be hard to write one.

Is there an equivalent of cuDNN out there ?

Well, right, even if tensorflow is built as opencl, it uses cudnn as its convolution library, so one would need to slot in something else instead of that. What hardware(s) are you targeting?

How different is deepCL compared to cltorch, clnn etc of yours and OpenCL-caffe from AMD?

Well...

NEELMCW commented 8 years ago

I am targeting AMD compute

hughperkins commented 8 years ago

I am targeting AMD compute

Ah. Much as it pains me to say it, probably the fastest convolutional backend for AMD right now is by Fabian Tschopp @naibaf7 . Since he works almost exclusively with AMD hardware, and works closely with AMD guys. (I made a decision to buy a laptop with an NVIDIA card, which has certain advantages for porting, but obviously sucks for optimizing for AMD :-P)

You can probably get hold of his convolutional backend by raising an issue at:

https://github.com/BVLC/caffe/tree/opencl

NEELMCW commented 8 years ago

Agree with you :-).

Oflate I have working in porting few of these frameworks in particular torch, caffe onto AMD's newly promoted HCC language over their new ROCM stack.

You may find my efforts @ https://bitbucket.org/multicoreware/hcc_torch. Its still under development though

hughperkins commented 8 years ago

Yes, hcc does seem to be the future, I concur. I dont think I'll be switching my own projects onto hcc, since opencl has its place too. But for AMD hardware, which is the main competitor for NVIDIA, in the discrete GPU space, as far as I know, hcc will plausibly be the way forward.

hughperkins commented 8 years ago

https://bitbucket.org/multicoreware/hcc_torch.

Whoa, cool :-)

NEELMCW commented 8 years ago

I am now looking at HCCizing Tensorflow. I reckon there are lot of dependencies the likes of Eigen, cuDNN that might refrain. I am currently investigating the efforts in this direction. any help from your side would be great

hughperkins commented 8 years ago

Ok. Dont suppose... for hcc-torch, do you mind linking also to cltorch? I think we can agree that hcc is a great way forward, so seems no need to hide alternative implementations :-)

NEELMCW commented 8 years ago

Sorry Didn't quite get what you meant here "linking also to cltorch" ???

hughperkins commented 8 years ago

In this bit:

"This repository hosts the HCC backend implementation project for torch7. To know what HCC is please refer here. Torch7 framework currently has a CUDA backend support in the form of cutorch and cunn packages. The goal of this project is to develop hctorch and hcnn packages that would functionally behave as HCC counterparts for existing cutorch and cunn packages. This project mainly targets the linux platform and makes use of the linux-based HCC compiler implementation hosted here. "

hughperkins commented 8 years ago

I mean, you dont have to. Its all good :-)

NEELMCW commented 8 years ago

:-) OK shall add references to cltorch and clnn

hughperkins commented 8 years ago

Ok. And you're waiting for .... ? If you're waiting for me to say that I will referrence hctorch from cltorch, I already wrote it in, just waiting for your reference, then I'll commit and push :-D

# Related projects

The OpenCL is enabled by using the following two projects, which are installed implicitly by this distro:
* [cltorch](https://github.com/hughperkins/cltorch)
* [clnn](https://github.com/hughperkins/clnn)

An hcc implementation for Torch is in progress here:
* [hctorch](https://bitbucket.org/multicoreware/hcc_torch)
hughperkins commented 8 years ago

(PS hctorch is much easier to say than hcctorch , and I notice that one of your modules is called hctorch, hence I've put hctorch, but I can put hcc_torch if you prefer :-) )

NEELMCW commented 8 years ago

I have made a reference to cltorch and clnn.... Agree with you the names got to change. Presently I am hosting both hctorch and hcnn, counterparts to cutorch and cunn under a single repo hcc_torch

NEELMCW commented 8 years ago

Repo name is now changed to HcTorch https://bitbucket.org/multicoreware/hctorch/overview

NEELMCW commented 8 years ago

Hugh,

The repo URL has changed. You may want to change it in cltorch Wiki here

https://github.com/hughperkins/cltorch

hughperkins commented 8 years ago

Ok updated :-)

FuriouslyCurious commented 8 years ago

Hugh - sorry to tag comment on a closed thread but didn't know how to reach Neelmcw.

Neel, I noticed your are working on hctorch. Thank you for that - I am also looking forward to a strong alternative to NVidia cuDNN from AMD.

I would like to request you to perform a benchmark and share how the library compares to cuDNN and fbnn. Can you please run these two benchmarks with hctorch and share the results?

  1. Soumith Chintala's benchmark: https://github.com/soumith/convnet-benchmarks
  2. Justin Johnson's benchmark: https://github.com/jcjohnson/cnn-benchmarks

Thank you!

naibaf7 commented 8 years ago

@campbx I'm currently also working on an alternative to cuDNN, namely libDNN (supposed to be autotuned vendor-independently at some point (https://github.com/naibaf7/libdnn)).

hughperkins commented 8 years ago

I'm currently also working on an alternative to cuDNN, namely libDNN (supposed to be autotuned vendor-independently at some point (https://github.com/naibaf7/libdnn)).

Hmmm, I should slot this into cltorch sooner rather than later really... I was going to wait until my winograd port was presentable, but that seems considerably harder than I thought, so maybe I'll add your library in now-ish.

FuriouslyCurious commented 8 years ago

@naibaf7 Thanks Fabian - the libDNN repo looks amazing. It will make it much easier to buy the cheapest card on market and run same Torch codebase on it.

What are your thoughts on AMD's new effort to de-CUDAify the code through HIP compiler? Early benchmarks I tested on my machine have been really good.
https://github.com/GPUOpen-ProfessionalCompute-Tools/HIP

@NEELMCW

naibaf7 commented 8 years ago

@campbx While HIP is really good, and helping developers to target more hardware, it naturally can't fix performance issues or help to get close to cuDNN, since that needs device specific optimizations.

bhack commented 8 years ago

@hughperkins You can always contribute winongrad to libdnn (if we can consider it upstream) later. ;)