Hi, I am Faruk. I found your repo via someone who was discussing some topic with me on Github. Special Thanks to Github for this.
I have been also developing a deep learning library in .NET 5 for 9 months and have developed the project enough to be considered as mature. I see you achieved NeuralNetwork.NET your own. That's a huge effort really. Thank you for that as well.
Your library seems to be similar to tensorflow. Do you have static or dynamic computation graph in the background?
My network type is dynamic graph like in pytorch. My aim is/was to develop a library to give the scientist the best experience and the most flexible library. To do so, I created multi-dimensional layers (unless you specify that it shouldn't be) and multi-inputs for an RNN layer etc. Because RNN layers are processed dynamically, there are many things you can do to control the flow of the network while it goes forward. So, how the flow will go may depend on the output of the current layer for example, or depend on the index (current time step value which is also multidimensional).
The layers that are ready to be used:
I separated the project into two subproject named: DeepLearningFramework and PerformanceWork.
Tensor, memory management and kernel implementations are in PerformanceWork. Other things like layers, terms, training procedures etc are in DeepLearningFramework.
The layers have terms inside them. The terms are the basic units of a graph. Terms can be thought as each box in the RNN representation. So, for each time step in a layer, there is a term corresponds to that time step. The time step is not just a number, but an n-dimensional tuple, because the network is multi-dimensional.
Example code for XOR model creation:
Example code for MNIST model creation:
The training procedure and loading data to the model are not ready for users. So, I semi-manually do that currently.
There are many things that I consider for future development such as Encoder-Decoder models and Multi-Task Learning. The library should be enough flexible to enable users to develop those models and the models that are not implemented yet in the world maybe.
Serialization is another topic that I give thoughts about. There are trade-offs between how flexible library and how easy to reload a model that is saved.
To sum up, my aim is to create the most flexible library, optimization is important of course but not the main target here. There are also distributed learning approaches for multi-GPU instances that I forgot to mention. It's pretty messed up thinking of those all together. Serialization, multi-GPU, encoder-decoder, multi-task learning... These are the things that are to be considered together while designing the library.
However, I have no strength to support my project. I believe that I have gained a lot of experience while developing the library. I also want to create it and propose to scientists and to my university maybe. Do you have any idea about how should I keep developing or should I? I'm trying to collaborate with people, but no one is coming. Every time, I write a lot of messages like this to explain the library. How did you manage to develop the library on your own? I mean it, it's really huge work. Congrats! Also, if you have enough time, you can check out my project. If you'd like to help me, I gladly accept that 😄
faruknane
commented
4 years ago
Hi, I am Faruk. I found your repo via someone who was discussing some topic with me on Github. Special Thanks to Github for this.
I have been also developing a deep learning library in .NET 5 for 9 months and have developed the project enough to be considered as mature. I see you achieved NeuralNetwork.NET your own. That's a huge effort really. Thank you for that as well.
Your library seems to be similar to tensorflow. Do you have static or dynamic computation graph in the background?
My network type is dynamic graph like in pytorch. My aim is/was to develop a library to give the scientist the best experience and the most flexible library. To do so, I created multi-dimensional layers (unless you specify that it shouldn't be) and multi-inputs for an RNN layer etc. Because RNN layers are processed dynamically, there are many things you can do to control the flow of the network while it goes forward. So, how the flow will go may depend on the output of the current layer for example, or depend on the index (current time step value which is also multidimensional).
The layers that are ready to be used:
I separated the project into two subproject named: DeepLearningFramework and PerformanceWork.
Tensor, memory management and kernel implementations are in PerformanceWork. Other things like layers, terms, training procedures etc are in DeepLearningFramework.
The layers have terms inside them. The terms are the basic units of a graph. Terms can be thought as each box in the RNN representation. So, for each time step in a layer, there is a term corresponds to that time step. The time step is not just a number, but an n-dimensional tuple, because the network is multi-dimensional.
Example code for XOR model creation:
Example code for MNIST model creation:
The training procedure and loading data to the model are not ready for users. So, I semi-manually do that currently.
There are many things that I consider for future development such as Encoder-Decoder models and Multi-Task Learning. The library should be enough flexible to enable users to develop those models and the models that are not implemented yet in the world maybe.
Serialization is another topic that I give thoughts about. There are trade-offs between how flexible library and how easy to reload a model that is saved.
To sum up, my aim is to create the most flexible library, optimization is important of course but not the main target here. There are also distributed learning approaches for multi-GPU instances that I forgot to mention. It's pretty messed up thinking of those all together. Serialization, multi-GPU, encoder-decoder, multi-task learning... These are the things that are to be considered together while designing the library.
However, I have no strength to support my project. I believe that I have gained a lot of experience while developing the library. I also want to create it and propose to scientists and to my university maybe. Do you have any idea about how should I keep developing or should I? I'm trying to collaborate with people, but no one is coming. Every time, I write a lot of messages like this to explain the library. How did you manage to develop the library on your own? I mean it, it's really huge work. Congrats! Also, if you have enough time, you can check out my project. If you'd like to help me, I gladly accept that 😄