Speed up simulation

[chriskelly]

Running 5000x takes ~15sec (not including generating returns). Improving this would allow far more combinations to be tried.

[chriskelly]

Seems numpy arrays are faster for accessing, lower memory, and allow vector operations. Lists seem to win for append speed though. Could be a useful to go through simulator code and change lists to numpy arrays where it makes sense

[chriskelly]

We should add a benchmark timer within simulator so we can understand how fast it is now.

[10CDizzle]

Yep, my uncommitted changed in the speed_enhancements branch include that. I'll push in a minute.

[10CDizzle]

I ran it through a profiler to see where we could get the most gains - the results are below:

I'm going to look at bracket_math() and get_taxes() - looks like we're making a lot of calls to those

[chriskelly]

Just made a commit that should cut down on deepcopy's time. It's been moved out of the monte carlo loop and now only runs once.

[10CDizzle]

Your latest commit had these profiling results on my system:

[chriskelly]

Interesting. On mine, annuity updating balance was the highest

[chriskelly]

Made improvements though that cut down annuity.update_balance time dramatically by maintaining a single balance rather than recreating a list of balances each time.

[chriskelly]

@10CDizzle I'm going to merge with the main branch because I think we're going to start having some conflicts that I'd like to address now rather than later. Can you commit any changes you're working on and let me know when everything's sync'd?

[10CDizzle]

Let's do the merge now - we already have a bunch of improvements and my stuff on bracket_math didn't yield great results. Will likely go back to the drawing board for that.

On Mon, Nov 28, 2022 at 12:30 PM Chris Kelly @.***> wrote:

@10CDizzle https://github.com/10CDizzle I'm going to merge with the main branch because I think we're going to start having some conflicts that I'd like to address now rather than later. Can you commit any changes you're working on and let me know when everything's sync'd?

— Reply to this email directly, view it on GitHub https://github.com/chriskelly/LifeFInances/issues/45#issuecomment-1329726577, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMBJLDNAZPGNT2ZARWOXD7TWKUI77ANCNFSM6AAAAAAQGFQ2GA . You are receiving this because you were mentioned.Message ID: @.***>

-- Christopher E. Davis Massachusetts Institute of Technology | Class of 2014 Department of Chemical Engineering GenOne Technologies - Principal

[chriskelly]

Alright, everything's merged now. Your params.json should update when you run simulator, but if you have problems, copy from the default folder and let me know.

[10CDizzle]

The latest commit to speed_enhancements has a kludge where it doesn't run the computationally-expensive calls to bracket_math if the quarterly income is zero (ant therefore taxes are zero). Cuts down the time it takes to make calls to get_taxes by roughly a factor of two. It doesn't look like it is the bottleneck from the latest profiling run

[chriskelly]

Nice! I went a little further and stored the bracket sums in the constants so now it's a simpler look-up. Down to 2.5% cumulative time (pink box).

It's very curious that we see pretty different profiles. I suppose it's down to our specific params.json.

[10CDizzle]

One final optimization run - I'm attempting to make the classes in /models/ compile directly to machine code using numba

https://numba.pydata.org/

Jury's still out on whether it will make things faster across the board. Experimenting in speed_improvements.

[chriskelly]

I'm going to close this issue and delete the branch now, and instead make individual issues to track improvements