Plan tasks for all group members to work through during Herbie meetings---stress testing

herbie-fp / odyssey

A platform for exploring floating-point expressions :boat:

https://herbie-fp.github.io/odyssey/

MIT License

17 stars 0 forks source link

Plan tasks for all group members to work through during Herbie meetings---stress testing #142

Closed elmisback closed 6 days ago

elmisback commented 1 month ago

First draft:

[x] sqrt(x+1) - sqrt(x): everyone submit at the same time, everyone hit Improve at the same time
[x] sqrt(x+1.(...)) - sqrt(x): everyone submit at the same time, everyone hit Improve at the same time (defeat any server caching)
[ ] if server gets knocked over, does cache get flushed, or is the work saved? might be good for avoiding slowdown, might be bad for keeping buggy output?
[ ] sqrt(x+1.(...)) - sqrt(x): everyone submit at the same time, then click Improve ASAP (mixture of types of calls)
[ ] Some larger expression that takes longer to improve in Herbie? Anything from Ben's examples?
[x] Rapidly clicking on different points to submit analysis requests
[ ] Some people running FPTaylor tool

elmisback commented 1 month ago

Check on this in RA meeting Thursday.

elmisback commented 1 month ago

Rapidly clicking points while many improve jobs were running resulted in slowdown and eventual page crash(!).

Could mitigate slowdown by dedicating a thread to analysis calls?

Not sure about page crash.

I crashed the page using sqrt(x+1.(...)) - sqrt(x) and a single Improve call with lots of clicking while the server was under load.

elmisback commented 1 month ago

Crash had error code 5

elmisback commented 1 month ago

Zach: page falls over if Improve submitted over and over -- should probably just block the user from submitting more than one at a time

Pavel: Maybe track number of jobs open on the server, use pop up or whatever to apologize if server is under load (also acts as a debounce)

zaneenders commented 1 month ago

The server does not cache to disk right now. So if the server topples over the cache is lost.

elmisback commented 1 month ago

Stress testing is ongoing. We need to test with a more recent version of main, I'm not sure if the current version that fell over at the end of Ben's talk had threads or not. Server commit has is currently cd05a706932a72a9634d93a31fadff41edbacc11

elmisback commented 3 weeks ago

We need to resolve action items from the data we've collected. This probably involves writing unit tests and identifying specific weaknesses to test for, with the goal of validating a server version for the feature freeze.

elmisback commented 2 weeks ago

No progress this week due to generals, but I should have some time to work on this on Thursday/Friday.

elmisback commented 1 week ago

Resolved action items:

Prohibit new requests when there is a pending request. I've hacked in solutions for this for point selection => local error and for Improve calls.

elmisback commented 6 days ago

I'm going to close this now, it is what it is.