search

OutlierVentures / QTM-Interface

GNU General Public License v3.0
28 stars 18 forks source link

UI Performance Improvements #47

Closed achimstruve closed 9 months ago

achimstruve commented 1 year ago

I did some performance tests in our streamlit interface with the following results:

Run empty script: <1s Simulation only: 33s Simulation + Postprocessing: 43s

This clearly shows that the calculation work is somehow done in streamlit, which makes it quite slow compared to the pure Python execution.

This streamlit blog talks about different tips to improve the streamlit performance. Someone can read this to see how we can apply them.

In this issue we can discuss/collect all UI performance related topics.

BlockBoy32 commented 1 year ago

@achimstruve how are you identifying those numbers? I went though the article and I am not sure if it helps. I am going to keep researching but having the way you got those numbers will be helpful

BlockBoy32 commented 1 year ago

We could potentially cache the simulation and run it in the background I am just not sure if that actually solves the problem. I think the size of the data we are pulling seems like its probably an issue too.

achimstruve commented 1 year ago

I literally took my phone and stopped the times manually from clicking on the button until the simulation finished. Note that those times might be different on your local machine.

Cached data is only helpful if we work with pre-processed data, such as stated in issue #45.

The blog post section 3 is talking about "..., bring your model and other assets to the production machine, and you can get orders-of-magnitude better startup time." I am wondering if this can help us.

BlockBoy32 commented 1 year ago

instead of running the file as a sub process + script I just called it as a function. Timing should be good now.

achimstruve commented 1 year ago

@BlockBoy32 not sure why, but for me the simulation now takes for ever (>5 min). Could you clear your cache and see if it is still solved?

BlockBoy32 commented 1 year ago

It was working for me, could it be with that new push that @alex put in?

BlockBoy32 commented 1 year ago

I am also now getting a new pool error? Not sure if thsts just local to my device right now

achimstruve commented 1 year ago

When I run the ./Model/tests/test_stage.py everything runs fine as before. So all the changes from @alex-damjanovic should be good. Also I went through all of his changes and there was nothing that will impact the performance or logic of the model.

BlockBoy32 commented 1 year ago

@achimstruve I am still struggling with this speed issue, I created a panel app to see if it was a streamlit but getting the same slow speed when I click the button to run the simulation. Its just weird because when I encode a timer on the interface it outputs 10-11 seconds but visually its a minute. If we have any talent with front end experience it would be extremely helpful.

achimstruve commented 1 year ago

I just created one simple UI interface with dash plotly but it is even slower than streamlit. On my machine it requires 71s to run the simulation + post processing vs. 43s with streamlit.

achimstruve commented 1 year ago

Now I built a minimalistic streamsync app app to test the simulation + post processing duration. It requires 66s on my machine and is therefore a bit faster than dash plotly, but still slower than streamlit.

BlockBoy32 commented 1 year ago

I can push my panel app if you want but its around the same time frame. its just weird because when I time the interface on the on actual app.py it shows it as super quick. So could it maybe be a rendering thing? I feel like if we had someone who understood front ends and could lend advise this could be a very quick fix for something that we are overlooking. I even tried to use the dev tools and still couldnt figure it out

achimstruve commented 1 year ago

The introduction of if-statements before for-loops over agents brought a slight improvement to the simulation time in pure python console testing. However, there is no real performance improvement for the streamlit UI.

Also I tried to remove any database operations in a dedicated test run. This didn't change anything as well.

Here are the current stats on my local machine for different simulation configurations via streamlit UI:

This means that it is the simulation itself that requires so much computational load via streamlit.

achimstruve commented 1 year ago

I created a new streamlit forum post on the UI performance, but no one answered, yet.

achimstruve commented 9 months ago

I close this issue for now as we don't have any tangible items left that we can work on to improve the UI performance further.

On my machine a 60 months simulation takes 12s and a 120 months simulation takes 19s, which is okay for the current state of the project.