Speed up by ~15x

[IainNZ]

Hi @Andy-P, I read your slide deck and found it pretty interesting.

The design right now is not exactly Julian, but I thought there might be some easy performance wins.

I set a seed at the top of example.jl and then ran for maxiters to warm up, then timed the second maxiters. First commit improves run time for this second bunch from

elapsed time: 9.979846892 seconds (4465378136 bytes allocated, 18.29% gc time)

to

elapsed time: 0.671702053 seconds (378648984 bytes allocated, 21.77% gc time)

I'm hoping to get some more! I'm working just on Julia 0.3, no idea about 0.4

[IainNZ]

Little bit of type typing removes type ambiguity in solver.jl, now

elapsed time: 0.506166683 seconds (301476984 bytes allocated, 18.79% gc time)

[IainNZ]

Devectorizing some of those operations got me some additional wins, down to

elapsed time: 0.357143495 seconds (196616696 bytes allocated, 21.77% gc time)

[IainNZ]

Save some matrix creations

elapsed time: 0.267663482 seconds (123987192 bytes allocated, 17.42% gc time)

[IainNZ]

OK, I think I'm done with the current design. We can talk about an different take on the design in another issue if you'd like. I can also rebase this before merging, I left it as individual commits so you can see the steps I took.

[Andy-P]

Iain,

Thanks so much for this. I found a few these but hadn't pushed my own code. Your implementation is much better. Learning so much from this!

Andre

[IainNZ]

I see you just added the first commit, but now I can't merge this. I have to go now, but I'll rebase later (I'd like it to be merged with my name on it if its going to be used)

[Andy-P]

Sorry, about that. I'm still new to Github collaboration. I did something wrong, but I am not sure what. My intention was to just accept your pull request with your name on it. Still trying to figure out how to fix what I've done

[Andy-P]

I think I understand what happened. I made changes to recurrent.jl a few days ago but didn't push it to Github.

Specifically, I changed this:

    w::Array # matrix of weights
    dw::Array # matrix of gradients

to this

    w::Array{Float64,2} # matrix of weights
    dw::Array{Float64,2} # matrix of gradients

When I tried to accept your pull I received a merge conflict. As it turns out my last commit and you 1st commit made the same change. So, I'll just wait for you to rebase and then redo it to ensure it is tagged with your name.

[Andy-P]

Found a easier way to do this. Just rolled back to when you forked and then merged. Thanks again

[IainNZ]

No problem!