Closed tisnik closed 6 years ago
I am already looking into this.
My findings and observations:
component-analysis level
There was one issue when we were initializing Selinon and Celery over and over again in basically every user request. This could sometime take multiple seconds and therefore very significantly prolong response time. I partially fixed the issue, but we still need to initialize celery in each request - this needs further investigation.
We are scheduling "bookkeeping" flow for each successful request. It runs asynchronously and it doesn't block the request. But if the only purpose of this flow is to store the bookkeeping data in RDS, then doing this directly in the context of request could save us some precious worker resources that can be used for stack analyses instead. And talking to RDS directly could be less time consuming than scheduling flows.
TL;DR:
@msrb - +1. During our very initial discussion with Slavek, it was more leaning towards separation of server and worker, but I guess we should re-think that now, since stack-analyses gets the data directly from RDS, and I see no reason why it can't directly persist the same in RDS as book-keeping. cc @samuzzal-choudhury
@msrb @miteshvp database operation will add up to the total response time. That was reason behind doing bookkeeping async way.
Description
Update current logging/tooling to be able to identify bottlenecks in the component analysis, both for known component(s) and for unknown (non-existing) components as well
Acceptance criteria
Related issues
Task for the epic #1144