GabeDottl / NineSlashNine-PythonAutocomplete

Other
0 stars 0 forks source link

Explore parallelization #13

Open GabeDottl opened 5 years ago

GabeDottl commented 5 years ago

Pretty complex due to dependency graph. Also made messy by Python's concurrency model.

Could perhaps do a minimal version w/async + module loading?

E.g. determine all module dependencies, create a producer of modules which processes modules as soon as all dependencies are met.

Could become a memory nightmare left unchecked - would be good to add in some restrictions to prevent consuming all CPU or memory..

Or perhaps only run this way when we care about maximal throughput? (e.g. on servers)

GabeDottl commented 5 years ago

Also interesting to consider whether there's a way we could make it so all modules essentially became independent of each other. The greater degree to which we can rely on stubs + lazy modules, the more true this is..