Closed koalaeagle closed 2 years ago
Hello, I am experiencing the same issue as well. I have increased my worker pool to 20 works and it has not helped.
Hello @koalaeagle and @andrewnorthern,
I think it's better to discuss that on Slack, since OpenCTI ingestion performances is something we need to qualify (depending on use cases, environments, etc.).
Please reach @richard-julien or myself directly on Slack. In the mean time, if you has not read this yet, I advise you to read carefully this one: https://medium.com/luatix/opencti-platform-performances-e3431b03f822.
Kind regards, Samuel
Close this due to inactivity.
Description
I am attempting to get started with OpenCTI version 4.5.2. When importing data from Alienvault OTX, I am seeing a processing rate of 3 messages per second. This will take a significant amount of time to import 2 million messages. Additionally, the rate seems to eventually drop down to 0.2 messages / second. I get similar results when disabling Alienvault and importing from a local MISP instance.
I had to increase my server to 16 vCPUs and 32GB of RAM to prevent it from running out of resources. To me it seems like the system needs a huge amount of resources and is still extremely slow to import any data (it would be months before the system had decent threat intelligence).
For now I just wanted to check in and see if anyone else has experienced similar issues or is it just me? Since it is a docker-based configuration and I have not changed anything from the example (aside from adding more feeds) I would assume my environment/performance is similar to others.
If this issue is just on my end, I can provide some more information.
Environment