Generic Async Task runner

[dlipicar]

Description

We've got some repeated patterns in the status-go Wallet Activity and Collectibles modules with regards to management of async tasks triggered by the client. We can probably develop a common"task manager" module that handles:

• Client registration (no need to hardcode the requestID in the Client)
• TaskID handler registration (distribute handlers of each taskID across different modules)
• Common structure for response events (for easily identifying on the client side which response event comes from which task run), no need to use different events for each new task.
• Single generic API function for Task execution.

We're now manually managing task distribution between threads (1 scheduler -> 1 thread). Is this desired, or do we want something more flexible like a thread pool? There are some thread pool implementations available.

[stefandunca]

...

• Client registration (no need to hardcode the requestID in the Client)

I tried to use the Nim instance address but it didn't work for some reason. We could generate an ID for each request on the backend instead. However, this might be covered better with a more advanced generic subscription mechanism that can improve the wallet events also.

• TaskID handler registration (distribute handlers of each taskID across different modules)

If we unify then this is required indeed.

• Common structure for response events (for easily identifying on the client side which response event comes from which task run), no need to use different events for each new task.

We already use Wallet event and I think we should move away from that and have only the request-identity common and payload different for each client.

• Single generic API function for Task execution.

Not sure here what do you mean.

We're now manually managing task distribution between threads (1 scheduler -> 1 thread). Is this desired, or do we want something more flexible like a thread pool?

I blocked only one thread because of the finding that the sqlcipher has contention issues found by colleagues from messaging. Not sure if this changed with the version upgrade. For sure we should do some benchmarks and allow few more in parallel for reading and serialize for writing.

There are some thread pool implementations available.

AFAIK goroutines are already efficient enough and run on a thread pool. Unless we have some long-running tasks that don't use the go sync primitives for cooperative multitasking, we should be ok by spanning as many as we need.