matus-chochlik / ctcache

Cache for clang-tidy static analysis results
Boost Software License 1.0
84 stars 30 forks source link

Feature Request: Consider adding a `clang-tidy-cache --show-stats` option #3

Closed devjgm closed 1 year ago

devjgm commented 3 years ago

ccache has a --show-stats flag that's useful to see the value its providing. https://ccache.dev/manual/4.2.html

Typical usage is like:

ccache --zero-stats
... do the build using ccache
ccache --show-stats

It may be nice if clang-tidy-cache had similar functionality when run in local mode.

What should the output be? Cache hits, cache misses, etc. In case it helps, the output from ccache looks like the following:

cache directory                     /h/.cache/ccache
primary config                      /h/.config/ccache/ccache.conf
secondary config (readonly)         /etc/ccache.conf
stats updated                       Mon Apr 12 16:33:48 2021
stats zeroed                        Mon Apr 12 16:32:03 2021
cache hit (direct)                   971
cache hit (preprocessed)              89
cache miss                             0
cache hit rate                    100.00 %
cleanups performed                     0
files in cache                      2039
cache size                         157.6 MB
max cache size                       5.0 GB

Of course, the output doesn't need to exactly match this, but it's something to consider and draw inspiration from.

matus-chochlik commented 3 years ago

The idea was that in the local mode the cache client should have as less overhead as possible. To gather that kind of stats additional synchronization is needed to keep the data consistent when doing parallel builds (and clang-tidy checks), besides what is necessary to touch the hash file. In the client/server mode the server gathers some such statistics and provides them through a REST API and a visual dashboard.

matus-chochlik commented 3 years ago

Hi, I've added the printout of some stats, but for now it works only with the server.

devjgm commented 3 years ago

Thanks.

Our builds run in a completely headless environment. They run in docker containers on servers and we only see the text logs. In such a case, I'm thinking the ctcache server would not be a big advantage for us and so I'm using local mode. Does this thinking sound right to you?

Also, no urgency on this FR. You can close this issue, or file it away to maybe do someday or not. It's up to you.

matus-chochlik commented 3 years ago

If the system running in the container has systemd, then it is pretty straightforward to (autostart and) run the server as a systemd user service under the same account. With the server, you get more features, the cached hashes are kept in memory, and saved in a zipped file, so depending on your storage HW the server may even be faster. It also takes care of the hash cleanup so you don't have to do that manually and it tracks some statistics. Also there is a dockerfile so you can run the server itself as a docker service. With something like docker-compose it should be easy to integrate into your environment.

Zitrax commented 1 year ago

I had the same use case, thus running only in local mode (just like ccache).

I made a quick not yet well tested addition here: https://github.com/matus-chochlik/ctcache/compare/main...Zitrax:ctcache:local_stats

matus-chochlik commented 1 year ago

Hi, it looks promising. If testing confirms that these changes don't break anything, please open a PR. Thank you.

Zitrax commented 1 year ago

Thanks, I can come back with a PR once I have tested it a bit more.