[WIP] Random distributions

eaplatanios / tensorflow_scala

TensorFlow API for the Scala Programming Language

http://platanios.org/tensorflow_scala/

Apache License 2.0

937 stars 95 forks source link

[WIP] Random distributions #9

Closed sbrunk closed 6 years ago

sbrunk commented 7 years ago

To get to know the codebase better, I've added ops for creating random uniform/normal distribution tensors.

Open questions:

The Python API has a graph-level random seed attribute. Should we do the same and add it to Graph?
Better type safety for parameters (Might make sense to revisit when the data types API is final).
- I wonder if it's possible to restrict the parameter values dataType to e.g. float16/32/64 at compile time somehow.
- For now the functions take just untyped Tensors even for scalar parameters like mean. I tried overloading with float/double instead but that doesn't work in combination with default parameters.

I know you're still traveling so no hurries. Perhaps you could have a look when you're back.

eaplatanios commented 7 years ago

That's great! :) I will have a look at the code once I get a chance during my trip.

Regarding your questions:

Yes, I think we should, but we should look into how and where the seed is serialized (e.g., MetaGraphDef or otherwise), when a graph is saved to disk, and replicate the behavior. I can look into that when I'm back and make the necessary changes.
That is currently a big open problem I have in my mind. I have thought quite a lot about it, but I think this may not be the right place to open this discussion. I will create a new issue detailing my thoughts and we can see if we can find an elegant solution.

eaplatanios commented 7 years ago

Issues #4 and #3 are related to your second question. :)

sbrunk commented 7 years ago

A thanks, could have found them myself. ;) I'll try to wrap my head around typed outputs a bit.

eaplatanios commented 6 years ago

@sbrunk I implemented a simple solution for a graph seed that gets serialized along with the graph and committed a cleaner solution for these ops. Would you like to check it out and tell me if you're happy with them? If yes, we can go ahead and close this pull request.

I will add unit tests later for these ops as for all other ops. I've been prioritizing adding support for ops, for now, rather than unit tests, in order to make the library usable in general ML contexts.

sbrunk commented 6 years ago

Great! Works for me except one small issue with the shape. I'll add a PR for that.

And thanks for adding the NN layers. :) That was actually the motivation for me to implement the random ops.

eaplatanios commented 6 years ago

@sbrunk Thanks for catching the bug!

I added most NN ops aside from convolution and pooling ops. I'm also working on a nice functional API for building neural networks and training them. I'll update you on this once it's in a usable state. :)

sbrunk commented 6 years ago

That's awesome. I was thinking of building something similar to Keras on top of your your TF API myself, but I guess I will have a look at yours first now when it's ready.

eaplatanios commented 6 years ago

@sbrunk That's kind of why I started this library in Scala in the first place! I'll let you know once I have updates! :)