search

deislabs / wagi

Write HTTP handlers in WebAssembly with a minimal amount of work
Apache License 2.0
889 stars 44 forks source link

This caches the modules once, and then uses the cached version #151

Closed technosophos closed 2 years ago

technosophos commented 2 years ago

This changes the way Wagi caches modules.

In this version, Wagi compiles the modules once and then caches them in memory. Each subsequent request for that module uses the cached module instead of regenerating the module.

In testing, this dropped the average page load time of Bartholomew from 560ms to 36ms. Tested with siege, too, and saw similar performance gains across both Bartholomew and Fileserver.gr

Note that what we give up here is file-based caching. So if we want to keep that, we might need to rework this a bit.

Signed-off-by: Matt Butcher matt.butcher@fermyon.com

itowlson commented 2 years ago

Would it be possible just to tuck the precompiled module in a case (perhaps now the only case) of a WasmModuleSource?

Or is it not practical to have an Engine available at the time we would need to precompile them?

technosophos commented 2 years ago

We might be able to do that. I'll look tomorrow.

technosophos commented 2 years ago

Thinking about this... I don't think there is any reason why we couldn't just load all of the modules once at startup. I mean, this PR could be refined to use an LRU cache so that Wagi could hold LOTS of modules. But that's not really a likely use case right now. So maybe the better approach would be to load all of the modules right at startup.

technosophos commented 2 years ago

Closing in favor of #153.