minusdavid
commented
3 years ago I used swagger-cli to bundle the swagger.json file into 1 file of 10891 lines instead of 53 files. (Alas, it still has 741 internal $ref pointers, although it's better than the 1151 $ref pointers we had before) and now it's taking about 70 seconds to do 763 brk syscalls... and no mmap/munmap calls.
That said... when compared en masse... it's taking about the same amount of time for the total 120 processes to startup. About 3 minutes from first to last.
I don't know that it's directly related to the spec file though since we're running a JSON::Validator->bundle() beforehand to load and validate the spec (before dynamically adding to it at runtime) and it's going super fast.
As I noted at https://github.com/jhthorsen/json-validator/issues/246, I'm having an issue where launching 120 processes that use Mojolicious::Plugin::OpenAPI is causing huge server load and slow application start time (about 3 minutes from start to finish instead of 30 seconds).
After a bit of experimenting, I've isolated the cause of the load and latency to be Mojolicious::Plugin::OpenAPI. Without it, the processes start rapidly and there's no load.
I need to do more in-depth profiling, but at a glance when I run a strace on one of the processes, I notice about 30 seconds is spent on about 300 brk syscalls, 1 mmap, and 1 munmap. I have 10 CPUs, which is way more than enough to handle the running workload, but it's not enough to cope with this load.
I suppose one answer would be to allocate more CPUs, but it would be overkill for the workload. It would just help with the startup time I imagine.
I'm wondering if the memory allocation is necessary.
I haven't profiled Mojolicious before so I'll have to look into that...