robertleeplummerjr
commented
5 years ago I'm not really having an issue with brain.js but I was just curious if there was a way to visualize a neural network like the image below.
RNN's get pretty big pretty quick. Sometime back, when I was debugging the implementation, I wrote https://github.com/BrainJS/rnn-viewer, which is not anything like your image, but a way to visualize them. Likely the code is a bit out of date, as it was a tool to fix what was not working yet. Of which those issues were closed sometime back.
I was also wondering if there's any news on training LSTMs with a GPU, if I recall I was told it was coming soon back in March and so far I've heard nothing, it has become a big issue as my datasets get larger and larger.
It has been a bit longer than anticipated, and I ty for the prod, but it is under very active development. I think the problem is the scope of v2 has grown over time. I did go through all the issues in GPU.js, where the GPU implementation is being done, and have finally set a limit as to what is going into v2. It stopped at bitwise operators, of all things. As it was requested several times prior to the initial node proof of concept.
A couple of really big concepts finally made their way into GPU.js v2. For example, I wanted to pull out a lot of the code that does type checking, so it could be done once, outside the kernel, rather than N lookups inside the kernel. This lead to kernels being able to raise an event that asks for a kernel to recompile, and then cache that beside the existing kernel.
Then there was the big issue of float for loop handling not working in node on some GPU's, which lead to either upgrading Google Angle to the latest, or writing a complete type inference system, of the two, the latter was chosen.
Then there was returning array values from kernels, and as well multiple render targets, which is an absolute must for brain.js to have performant backpropagation. This lead to having to learn and debug c++ and headlessgl, to add those parts, unit test them thoroughly, join their development team, and get it released.
Then there was breaking up the argument and constant handlers, so they reused the same code, and did the type checking to raise said events about kernels in the previous paragraph.
We have also added TONS more unit tests. I believe we are nearing 20k assertions, testing the kernels from different angles, and ensuring all tests pass that are supported in Node, Chrome, Firefox, and Safari.
I think I could go on and on. And I apologize for it taking so long. The most important thing, is, I believe we are down to the wire, and the implementation will not be rushed, it will be right.
Once I cannot find anymore bugs in GPU.js, v2 will land, and I will take a proper look at all open issues for brain. If I do find anymore bugs in GPU.js, I will add those unit tests, fix them, and continue iterating. If anyone would like to join in the fun, they are invited to jump in.
CamK06
samfultonjr
mubaidr
Comments
I'm not really having an issue with brain.js but I was just curious if there was a way to visualize a neural network like the image below.
I was also wondering if there's any news on training LSTMs with a GPU, if I recall I was told it was coming soon back in March and so far I've heard nothing, it has become a big issue as my datasets get larger and larger.
And finally, a while back I noticed an issue someone posted (#377) which I was also having and he wasn't able to get help so I am just going to add it onto this issue; When I train a network and save it to a JSON file I would like to be able to do
network.fromJSON();to load it then using that network, add more training on top of its existing training data since retraining everything with the old data takes forever without GPU acceleration (I'm sure it would still be an issue WITH GPU acceleration). Is this in any way possible, and if not, could it be added?