Use for neural networks (ANN)

[UniqueFool]

Just a heads-up: There's recently been some discussion on reusing arrayfire-js as the back-end in the BrainJS and Synaptic frameworks:

https://github.com/cazala/synaptic/issues/70 https://github.com/cazala/synaptic/issues/12

[unbornchikken]

Yeah, AF.js works well for this case, but you gotta expect some serious API changes (sync -> async) because of https://github.com/arrayfire/arrayfire-js/issues/9.

[UniqueFool]

Hi, thank you for responding - given that these refactorings are basically about implementing functional-a style interface, do you think you could provide a functional-style set of APIs in the form of map/filter and reduce that basically deal with arrays and callbacks directly ?

That would make it much easier for other ANN projects to use arrayfire in a higher-level fashion while isolating them from lower-level API changes.

For instance, here's the response from @cazala (synaptic lead developer), where he specifically stated that the changes required for adopting arrayfire would be fairly self-contained once we can access/use arrayfire in a map/reduce fashion: https://github.com/cazala/synaptic/issues/70#issuecomment-222581144

Thank you

[unbornchikken]

@UniqueFool Please note https://github.com/arrayfire/arrayfire-js/issues/9. Resolving that issue will lead fundamental API changes for sure. I'm going back to this thread once the new version stabilizes, and then I can review your above suggestion.

[robertleeplummerjr]

any headway?

[unbornchikken]

Unfortunately I couldn't come up with a solution in Node.js that is better (faster) than we have in the current version. ArrayFire's design could not fit very well with the Nature of Node's event queue, so it turned out, my first approach is the best that I can come up with. The good news is that already near feature complete, just check out the examples folder.

[robertleeplummerjr]

So in https://github.com/arrayfire/arrayfire-js/blob/master/examples/es6/machine-learning/ann.js#L15 I see a log of synchronous examples, for(var i = essentially. What if we inverted the logic so that we are transforming a vector, rather then a matrix? In short (taken from: https://github.com/gpujs/gpu.js/issues/67#issuecomment-301217809)

We could implement something more like:

function relu(value) {
  return Math.max(0, value[this.thread.y][this.thread.x]);
}

function add(left, right) {
  return left[this.thread.y][this.thread.x] + right[this.thread.y][this.thread.x];
}

function multiply(left, right) {
  var sum = 0;
  for (var i=0; i<512; i++) {
    sum += left[this.thread.y][i] * right[i][this.thread.x];
  }
  return sum;
}

const gpu = GPU({ mode: 'webgl' });
const layerForward = gpu
  .addFunction(relu)
  .addFunction(add)
  .addFunction(multiply)
  .createKernel(function(weightMatrix, inputMatrix, transitionMatrix, previousOutputMatrix, biasMatrix) {
    return relu(
      add(
        add(
          multiply(
            weightMatrix,
            inputMatrix
          ),
          multiply(
            transitionMatrix,
            previousOutputMatrix
          )
        ),
        biasMatrix
      )
    );
  });