search

YPares / porcupine

Express parametrable, composable and portable data pipelines
89 stars 11 forks source link

Idea: work together on an example for hasktorch #64

Open tscholak opened 5 years ago

tscholak commented 5 years ago

Hi @YPares, @austinvhuang mentioned tweag and your name on the https://github.com/hasktorch slack channel recently, and I got curious and had a look at tweag's open source haskell libraries and promptly found porcupine. It appears to me that hasktorch and porcupine could be a good fit when it comes to building an end-to-end machine learning pipeline, starting from preprocessing raw inputs to tensors, via training or running a model, to postprocessing tensors into the desired output format. In order to test this hypothesis, I'd like to propose a small collaboration in which we together build a small but complete example of such an application. An ideal candidate would be solving MNIST, see, for instance, https://github.com/goldsborough/examples/tree/cpp/cpp/mnist. We've already started working on a toy model implementation, see https://github.com/hasktorch/hasktorch/pull/196, but it would be great if we could pair this with an example of a principled data pipeline. Let us know if you are interested, Cheers, Torsten

YPares commented 5 years ago

Hi @tscholak :) yep interesting idea! Let me ping @MMesch who has worked a bit with hasktorch and who'll be interested also

tscholak commented 5 years ago

@YPares glad to hear that, @idontgetoutmuch has shown interest in working on this as well.

YPares commented 5 years ago

@tscholak if you are interested, I'll be presenting porcupine at Haskell Exchange just this Friday morning: https://skillsmatter.com/conferences/11741-haskell-exchange-2019#program the talks will be uploaded just after.

tscholak commented 5 years ago

@YPares I saw, yes, I'm looking forward to the recording of your talk :)

tscholak commented 5 years ago

I just finished watching your talk, @YPares. Amazing presentation of an awesome library! I definitely want to see how this could be used with hasktorch. I will try to find your Bayesian inference example. From the sound of it, it comes closest to what I have in mind. The data visualization part of it is also very relevant.

YPares commented 5 years ago

@tscholak Thank you :) The bayesian inference example is https://github.com/tweag/porcupine/tree/master/porcupine-core/examples/example-radon . It could do with some comments, but it follows a blog post by Thomas Wiecki (link in a comment in example's source), although for now it implements only the flat linear regression. I should improve it with more visualisations of estimated parameters and their uncertainty intervals. It uses monad-bayes

YPares commented 5 years ago

@tscholak btw, I invite you to join the gitter https://gitter.im/tweag/porcupine?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge if you have questions, that's easier for conversations :)