Implement neural networks

[sjwhitworth]

Not sure if we want to stick with plain vanilla neural nets, or if we want to do deep learning. Do you have any experience with this @macmania ?

[macmania]

yup - I took machine learning and we had to implement neural nets, i don't have a phd on it but if I get stuck - i know a couple of people that can help out :)

[sjwhitworth]

Sorry, I meant do you have experience with deep learning?

[macmania]

i don't have experience with deep learning but i can get up to speed with it :)

[sjwhitworth]

How are you getting on with this @macmania - have you had any spare time to look at it? I may make a start on it, if you want to work together.

[macmania]

Sure, im sorry about that. I've been busy with finals here :(

[sjwhitworth]

I think that's a pretty good excuse :)

[geraldstanje]

Hi guys,

Ryuichi has implemented a pure golang neural network lib: https://github.com/r9y9/nnet Do you guys want to reuse his code and integrate it to golearn? I could also help you in implementing deep learning... since we need train the model on the gpu, the following lib seems promising: https://github.com/barnex/cuda5 It would be great to use golearn for deep learning rather than pylearn2...

Thanks, Gerald

[Sentimentron]

I was considering this in the context of Word2Vec which is a technique for projecting words into a semantic space. It's do-able, but it requires some additional work on data representation (specifically support for relational attributes) before it could be realistically implemented.

[geraldstanje]

how is is Word2Vec related to deep learning? shouldnt the lib be more generic? Word2Vec seems to be a special case of text classification/sentiment analsis? i would like to see that deep learning can also be used for the following task: http://www.image-net.org/challenges/LSVRC/2014/ ...

[lazywei]

I don't think we need to integrate with text mining / NLP. We should keep the library more generic and focus on numeric data only.

[Sentimentron]

The guys at Stanford basically use word2vec and then layer a recursive neural network on top to do sentence parsing, so I'd argue that replicating those kind of applications is a pretty good place to start when thinking about API design.

[r9y9]

Hi,

As @geraldstanje mentioned above, I implemented neural network algorithms (nnet) that include multi-layer perceptron, binary-binary restricted boltzmann machines (RBM), gaussian-binary RBMs and deep belief nets in the past. I'm not currently planning to add more features such as GPU powered training and other type of neural networks, but it might be helpful for someone to start writing neural networks in golang.

[sjwhitworth]

Hey there. Your project looks great. Would you be willing to integrate your code with golearn?

[r9y9]

Thanks, but I’m afraid I don’t have time to do that.

[sjwhitworth]

Sorry, I probably wasn't clear. Would you be willing to allow your code to be used in golearn, if we were to be the ones that are porting it over? I can understand if not.

[r9y9]

Sorry, I misunderstood. Sure. That would be great.

[lazywei]

So are we going to "integrate" these two packages, or are we going to just create a wrapper for nnet in golearn?

[geraldstanje]

let me know what you want to do, i can help you with some integration work...

[jonhkr]

Hello guys,

I have been speaking with @geraldstanje about integrating my project gobrain (referenced above). As part of the integrating process I think it would be good to create a corporation, maybe we can name golearn, so we can create a team to work on the project. With a corporation we can also use github pages to make a nice website, which will bring more visibility to the project.

What do you think?

Cheers, Jonas Trevisan

[lazywei]

You mean a Github organization, right? I totally agree with that. However, I also think that we should integrate into one package. As in python, I can utlize almost any algorithms, models and other goodness in sciki-learn, i.e. in one united package.

[jonhkr]

Yes, I meant a Github Organization.

[Sentimentron]

Just until we decide what we want to do: #63 adds very basic support for feed-forward multi-layer networks.