Multithread the server so multiple calls can be carried out at once

KordingLab / llm4papers

Apache License 2.0

22 stars 0 forks source link

Multithread the server so multiple calls can be carried out at once #5

Open j6k4m8 opened 1 year ago

j6k4m8 commented 1 year ago

Configurably, we should multithread the polling server, multiplexing across documents (but not across edits in one document, in case one changes the context of another)

wrongu commented 1 year ago

How about multiprocess rather than multithread here? Proposal:

one main process that infrequently polls projects to see if there are any "live" changes happening
...if so, spawn a paper-specific process
the per-paper processes shut themselves down when a paper goes quiet for a while

j6k4m8 commented 1 year ago

Totally — love it. Each process can be super lightweight too since 99% of the lifecycle is waiting for an HTTP round-trip...

j6k4m8 commented 1 year ago

...actually pretty low-importance until #10 since git rate limits will prevent this from actually being useful to us.

wrongu commented 1 year ago

would be nice if polling strat was separate for each paperremote. currently, if one paper is being actively edited, all papers get polled more frequently.