enso-org / enso

Hybrid visual and textual functional programming.
https://enso.org
Apache License 2.0
7.34k stars 321 forks source link

Introduce throttling to expression updates #7087

Open hubertp opened 1 year ago

hubertp commented 1 year ago

Occasionally bugs in instrumentation (like #7070, #4090, #3977) will lead to an increased number of expression updates. The amount of such updates can be so big that either instrumentation or IDE will give up and appear to freeze. In order to workaround this scenario, we could introduce in instrumentation updates throttling (per expression). Updates would be sent on two occasions: 1) a specified time window has passed and execution still continues 2) execution has finished and some updates haven't yet been sent

This should help with the impression that the execution continued as normal while giving us some leeway into fixing the underlying problem with less priority. Additional logging that monitors excessive updates must be added as well. The throttling would need to be carefully designed in order that IDE does not miss any crucial updates or creates inconsistencies, thus making things worse than before.

JaroslavTulach commented 1 year ago

The desire for rationalistic solutions inside of us provides impetus to solve these issues properly by annotating nodes with AvoidIdInstrumentationTag. Incrementally we are supposed to improve our avoid instrumentation capabilities and we do it as following issues demonstrate:

Ultimately we don't need any throttling! Step by step we'll make annotation of our nodes better and better and everything will be fast and smooth at the end. However the question is: What happens when things go wrong before we get to such an ideal state? Clearly we are not there as issues like

constantly remind us. We haven't reach the nirvana yet. What happens in such situations? The performance drastically decreases and poor Enso users get far from ideal experience. Can we do something about such known unknowns?

Yes, throttling is the solution. Such a pragmatic approach gives us a safety net to provide relatively acceptable user experience even in situations where avoiding instrumentation isn't done correctly. There is no reason why Enso users shall suffer for mistakes that we make - we should have the throttling safety net in place whenever we create a release.

Of course such pragmatic approach clashes with our desire for proper rationalistic solution. That's normal. However the answer is not to avoid the pragmatic choice - the answer is to design it in a way to rescue us, but give us enough information to incrementally improve our code - e.g. to place the AvoidIdInstrumentationTag where needed:

Are sample ways to allow us to act pragmatically, but strive for rationalism.

enso-bot[bot] commented 6 months ago

Dmitry Bushev reports a new STANDUP for yesterday (2024-02-20):

Progress: Started working on the task. Debugging the issue with the issue with the suggestions not being sent to gui. Figured out that during the initialization the suggestion deserializing jobs were not started. Implemented the fix to the unexpected warning in the runner script. It should be finished by 2024-02-26.

Next Day: Next day I will be working on the #7087 task. Continue working on the task

enso-bot[bot] commented 6 months ago

Dmitry Bushev reports a new STANDUP for today (2024-02-21):

Progress: Continue working on the task. Continue looking into the issue with the suggestions. Tried re-enabling the background processing but the issue remain in some cases. Switched to yjs task. Implemented the events interface. It should be finished by 2024-02-26.

Next Day: Next day I will be working on the #7087 task. Continue working on the task

enso-bot[bot] commented 6 months ago

Dmitry Bushev reports a new STANDUP for yesterday (2024-02-22):

Progress: Continue working on the task. Finished working on the regression with the suggestions synchronization. Tested in gui. Created the PR. It should be finished by 2024-02-26.

Next Day: Next day I will be working on the #7087 task. Continue working on the task

enso-bot[bot] commented 6 months ago

Dmitry Bushev reports a new STANDUP for today (2024-02-23):

Progress: Continue working on the task. Updated the PR with the unexpected logging. Addressed the review comments. Refactored the WebSocket server functionality into the WebSocket class. Implemented rest of the methods used in the server setup. It should be finished by 2024-02-26.

Next Day: Next day I will be working on the #7087 task. Continue working on the task

enso-bot[bot] commented 6 months ago

Dmitry Bushev reports a new STANDUP for yesterday (2024-02-26):

Progress: Continue working on the task. Finishing the Yjs server. Fixed an issue with NPE in the onconnect callback. Implemented the creation of WebSocket object per connection. Created a simple demo to verify that the server works with multiple clients. It should be finished by 2024-02-26.

Next Day: Next day I will be working on the #7087 task. Continue working on the task