[Feature Request] whitelist.conf / blacklist.conf / query-bulk support

[druchoo]

• query-bulk

# It's more effective, but python-carbon 0.9.13 (or latest from 0.9.x branch) is required
# See https://github.com/graphite-project/carbon/pull/132 for details
#CARBONLINK_QUERY_BULK = True

• whitelist.conf

# This file takes a single regular expression per line
# If USE_WHITELIST is set to True in carbon.conf, only metrics received which
# match one of these expressions will be persisted. If this file is empty or
# missing, all metrics will pass through.
# This file is reloaded automatically when changes are made
.*

• blacklist.conf

# This file takes a single regular expression per line
# If USE_WHITELIST is set to True in carbon.conf, any metrics received which
# match one of these expressions will be dropped
# This file is reloaded automatically when changes are made
^some\.noisy\.metric\.prefix\..*

[stv-io]

+1

[AlexAkulov]

carbon-c-relay?

[bzed]

I think @AlexAkulov is right, for white/blacklisting using carbon-c-relay is easy (and I guess you need it anyway :smiley_cat:

[ra-dft]

Any updates on this? Does go-carbon support white/black lists at all? This is a big decision maker as we heavily use blacklists right now, however we're in the process of building out a new cluster and migrating services over to it. If we can get the speed of go-carbon and full compatibility with carbon-c-relay that would be awesome.

[lomik]

go-carbon doesn't support white/black list. But you can use blacklist in carbon-c-relay (send to blackhole)

[ybizeul]

That would be really handy. I did a simple in-place replacement of carbon with go-carbon and just found out about missing support for blacklist.conf. In my use case that's a pretty big deal. Implementing carbon-c-relay is quite some work to integrate in my build pipeline.

Any chance we see that implemented eventually ?

[deniszh]

@ybizeul : as far as I know developers have no intention to implement such feature. But patches are welcome. Similar functionality exists in carbon-clickhouse - https://github.com/lomik/carbon-clickhouse/pull/51 - maybe someone can adapt this code or write new one. If you need practical solution right now and don't like C code in your pipeline - you can try nanotube or graphite-relay-ng - both have support for filtering and aggregation.

[ybizeul]

Thanks Denis, I'll start working on bundling carbon-c-relay in my ova then, it's probably my best option knowing this !