Accessing indexed-db in an async call blocks main thread

[steininge]

Describe the bug Writing data to indexed-db using the storage module and calling it by call_async from the non-blocking module seems to block the main thread so that the app becomes non-responsive.

EDIT: it seems all expensive operations causes lag when run async (test app updated). Is it then a cpu load issue? I was hoping the switching between threads would keep the UI responsive.

Version 2.7.1

To Reproduce

I created an app that demonstrates this.

Public link: https://terrific-quirky-yak.anvil.app

Clone link: https://anvil.works/build#clone:5ZALETKYA3KHGJWW=QLFGAKJW5724PZ4BC7KHO4EH

Expected behavior Expected to be able to read and write to indexed-db in the background without the user noticing any lag in the UI responsiveness.

I am considering making a PR for this issue No

[s-cork]

thanks for the clone

the behaviour you report is as expected i've added some print statements so you can see how the control flow works

https://anvil.works/build#clone:ZGJI3PEYD2DCR3UK=WOGBFBGRYHFYFMBIN54OHAHP

as soon as the asynchronous call does something inherently asynchronous control is given back to the event loop and the next line can proceed.

with expensive operation

PERFORMING OPERATION
PERFORM EXPENSIVE OP CALLED, ABOUT TO SLEEP
ASYNC CALL GAVE BACK CONTROL
DONE SLEEPING, ABOUT TO INIT LARGE DB
INIT LARGE DATA DONE

with write_db

PERFORMING OPERATION
WRITE DB CALLED
ABOUT TO GET LARGE DATA
Time taken by create large data: 1.740000 seconds
DATA GOT, ABOUT TO WRITE TO STORE
ASYNC CALL GAVE BACK CONTROL
DONE WRITE TO STORE

The async call gives back control at the first async operation and allows the print("ASYNC CALL GAVE BACK CONTROL") to execute (which it wouldn't without the use of call_async)

in the first version, we get back control during the sleep call which is an async operation. In the second version, we get back control when we write to the db, which is also an async operation.

Your mock getting large data is a synchronous operation so does not yield control back to the event loop.

Since JavaScript is single threaded, if you have a synchronous operation it will block the main and only thread, regardless if you use call_async or not

---

You need a new thread to not block the main thread when using synchronous operations. You may want to explore the anvil-labs web worker module for this use case.

Web workers allow you to write code that executes in a separate javascript thread.

[steininge]

Thank you, Stu, for your quick reply and the great explanation! That was very helpful.

The control is returned when we write to db, as you say, but it takes almost as long as the writing operation itself before the control is returned. Is the storage module performing synchronous operations before finally performing an async operation when writing to indexed-db?

Is this as expected, and I should use web-workers to write local data without the user experiencing lag?

Start async operation PERFORMING OPERATION WRITE DB CALLED Time taken by async call: 2.923000 seconds ASYNC CALL GAVE BACK CONTROL DONE WRITING TO STORE Time taken by write data to store: 3.176000 seconds

https://anvil.works/build#clone:KNPDG4QS2BZILKBI=MPQOHPTRG7ZLFH6I5MJXXA2I

[s-cork]

It depends what object you're storing in indexed db and how large they are.

There are quite a few layers of abstraction going on here.

It could be the writing of the data to indexed db. It could be the library that is wrapping indexed db. It could be the conversion from Python objects to JavaScript objects and giving those to the library that is wrapping indexed db. It could be an issue with the anvil-extras wrapper preparing the object to be sent to indexed-db.

It could be all of the above.

Doing all the above in a web worker would mean the operation was running in a different thread and so the main event loop would be free for user interaction.

The way to evaluate this is to look at the performance tab using browser dev tools. But it can take a bit of getting used to, and knowing what you're looking for.

[steininge]

Web worker seems easiest, but the docs says:

A worker module, like fib_worker above, can only import libraries from python’s standard lib.

So it is not possible to import anvil_extras.storage.indexed_db and run the db writing as web worker?

[s-cork]

Yes we implemented support for anvil extras indexed db for this reason. But worth noting that anvil labs is experimental so using it in production apps isn't recommended.

If you plan to use it in production best to use a local clone.

[steininge]

Perfect! I tried it out, and it worked like a charm!

Thank you for your help.