search

cloudfoundry-community / logsearch-for-cloudfoundry

A Logsearch addon that customises Logsearch to work with Cloud Foundry data
Apache License 2.0
49 stars 57 forks source link

logsearch for cloudfoundry scalability #268

Closed marcus-sds closed 7 years ago

marcus-sds commented 7 years ago

Hi all. I'm using logsearch for cf release to gather the application log. I generated about 10,000 message/sec with cf app and there are many missing values at logsearch. Can you provide the scaling and performance tuning guide? We are using 2 ingestor, 2 parser and 2 es data instances and firehose to gather the log. Ingestor's cpu util is the highest one. If I use 2 firehose, the message can be duplicated?

Infra-Red commented 7 years ago

@marcus-sds The messages will be divided equally between Ingestors with the same firehose subscription id.

Infra-Red commented 7 years ago

Closing due to inactivity. Feel free to reopen if you need more help.