search

mozilla / Snappy-Symbolication-Server

https://wiki.mozilla.org/Snappy_Symbolication_Server
Mozilla Public License 2.0
3 stars 12 forks source link

Switch I/O processing to Tornado framework #7

Closed walac closed 9 years ago

walac commented 9 years ago

Move the I/O processing from "one thread per connection" to Tornado coroutines. This allows us to process multiple connections in a single thread.

The symbolication code has been moved to a separate process so new connections get accepted fast. This is makes better use of multiple cores.

r? @vdjeric

vdjeric commented 9 years ago

Can you make the default gOptions platform-independent:

"mruSymbolStateFile": "/tmp/snappy-mru-symbols.json",

vdjeric commented 9 years ago

Don't you want to break up the symbolication with some yields for better responsiveness? FetchSymbolsFromURL for example

walac commented 9 years ago

hrm, with async calls to S3, we could have several simultaneous fetch requests, making better usage of the bandwidth, we could potentially speed up requests a bit. I am going to do that. This actually showed to be quite tricky, because there is no IOLoop object running on the subprocess.

vdjeric commented 9 years ago

You should have a yield between the fetching of .sym files from S3, the parsing of the .sym file, and the local caching of the parsed results on local disk (this last one is future work).

walac commented 9 years ago

The double import os was fixed.