PaloAltoNetworks / minemeld-ansible

Ansible playbook for installing MineMeld on Linux
Apache License 2.0
48 stars 48 forks source link

Performance problem when stdlib.listIPv4Generic node contains a large number of addresses #61

Closed pgerencser closed 5 years ago

pgerencser commented 5 years ago

We would like to use a stdlib.listIPv4Generic miner with more than 25000 addresses. These addresses has been imported from a texfile using minemeld-sync.py. My problem is that it takes around 20 seconds to list the indicators (500 pages, 50 indicators per page). During the process minemeld generates the paginator component, one CPU core is used at 100%, but other cores are not used. When I add a new IP, after ~10 seconds I get a green message saying "Changes saved". When I refresh the indicator list, I can find the recently added address. During the add process one core of the CPU works at 100% for several sconds. So at only one address it works, just a bit slow. When I add 3-4 IPs one-by-one using the web gui (my typing speed is around 3 seconds per IP), I get a red message: "Error adding indicator: Timeout". At this case recently added addresses has not got in the list.