search

gchq / stroom-stats

Other
4 stars 4 forks source link

Too many calls to stroom database during large stats load #34

Open at055612 opened 7 years ago

at055612 commented 7 years ago

When running a bulk load of stat events for a single stat config, there is a lot of activity on the stroom database (as seen in docker-compose stats) suggesting the caching of the stat config entity is not working in some way. The cache should be holding a copy of the one stat config that is in use in the load and thus no more DB queries are required.

Investigation needed to see what DB interactions are happening.

Run the COUNT_LARGE feed and monitor the DB queries.

at055612 commented 7 years ago

Not sure this is stats related as the only interaction stats has with the stroomDb is to fetch the stat config entity, which is cached so is only fetched once every 3mins or so.