Closed grooverdan closed 6 years ago
Before change mongotop was showing the read/total ghcrawler.commit time in the 1000s of ms. After:
ns total read write 2018-06-07T04:39:42Z ghcrawler.commit 3ms 3ms 0ms admin.system.namespaces 0ms 0ms 0ms admin.system.roles 0ms 0ms 0ms admin.system.version 0ms 0ms 0ms config.system.namespaces 0ms 0ms 0ms config.system.sessions 0ms 0ms 0ms ghcrawler.admin.system.version 0ms 0ms 0ms ghcrawler.commits 0ms 0ms 0ms ghcrawler.contributors 0ms 0ms 0ms ghcrawler.deadletter 0ms 0ms 0ms
Thanks @grooverdan. The change looks simple. Anything else need to be done? Do you want to do a PR for it just to make sure the change goes where you expect etc? (not a lot of Mongo expertise available on the team right now)
I haven't looked at where in the code this is/should be. I'll take a look.
This is my first look at Mongo too. A runaway CPU consumption aids a bit of rapid learning :-)
Crawling a largish tree like MariaDB generated a rising CPU profile on the server for the mongodb process:
mongodb process was niced just before the fix below.
A log at the profiler shows a column scan looking for a
_metadata.url
Adding an index on this cased a rather quick drop in CPU as shown in the end of the graph:
If this could be default that would be most useful.