Abc-Arbitrage / infix

MIT License
38 stars 6 forks source link

Usage of rule update_tag_value.go #12

Open guro23 opened 2 years ago

guro23 commented 2 years ago

We are using this project to change some of the values in our influxdb. Renaming tags works fine.

When we try to use the update_tag_value rule, we get an error for an unregistrated rule.

We tried adding it to registry.go. But no success of registrating it.

The update_tag_value.go is missing a code snippet for configuration or is it used in another way inside the TOML configuration file?

oktal commented 2 years ago

Hello.

You are right, the UpdateTagValue rule was not properly plugged in the configuration system.

I implemented it in af40eefb027456882cdf21a7966b5bf760d91bdb

Could you try and let me know how it goes ?

guro23 commented 2 years ago

Thank you very much. It works if you change the config from example.

[[rules.update-tag-value]] to="NewValue" [rules.update-tag-value.measurement.strings] equal="Measurement" [rules.update-tag-value.key.strings] equal="KeyValue" [rules.update-tag-value.value.strings] equal="OldValue"

Unluckily it is not changing values inside our files. It shows changed Values for our rules but result is not noticeable inside our queries.

oktal commented 2 years ago

Thanks, I'll update the example accordingly !

Unluckily it is not changing values inside our files. It shows changed Values for our rules but result is not noticeable inside our queries.

Which kind of index are you using ? memory or tsi ? If you are using the TSI, make sure to rebuild the index

hiroshi-kawazoe commented 2 years ago

As far as I tried, I needed to change the FilterKey() in UpdateTagValueRule to return true, although I'm not sure if this is the right modification. https://github.com/Abc-Arbitrage/infix/blob/master/rules/update_tag_value.go#L68

Also, when comparing measurement names, equal did not work well, while contains seemed to work, though not perfectly.

[[rules.update-tag-value]]
    to="NewValue"
    [rules.update-tag-value.measurement.strings]
        contains="Measurement"
    [rules.update-tag-value.key.strings]
        equal="KeyValue"
    [rules.update-tag-value.value.strings]
        equal="OldValue"
mayeranalytics commented 2 years ago

When I build a test database with content like this

insert linux.test,region=amazon-us value=1 1439856000

the unaltered example in UpdateTagValueRule just seems to delete everything. (I'm running Influxdb 1.6, which doesn't seem to have influx_inspect for rebuilding the index, yet.) I would caution against using UpdateTagValueRule without thorough testing in a copy of the target database.

pencil commented 1 year ago

I tried running infix with the following rule (to change the tag value of "instance" from "wlan0" to "wl0-ap0" on all measurments):

[[rules.update-tag-value]]
        to="wl0-ap0"
        [rules.update-tag-value.measurement.pattern]
           pattern=".*"
        [rules.update-tag-value.value.strings]
           equal="wlan0"
        [rules.update-tag-value.key.strings]
           equal="instance"

This did not appear to change any data.

I then implemented the change suggested by @hiroshi-kawazoe:

As far as I tried, I needed to change the FilterKey() in UpdateTagValueRule to return true, although I'm not sure if this is the right modification. https://github.com/Abc-Arbitrage/infix/blob/master/rules/update_tag_value.go#L68

The script took a lot longer to run and it ended with an output indicating that the tag values were changed as expected:

[UpdateTagValueRule] Updating tag for measurement 'interface_rx' instance=wlan0 to instance=wl0-ap0: #148 actions [UpdateTagValueRule] Updating tag for measurement 'interface_tx' instance=wlan0 to instance=wl0-ap0: #148 actions [UpdateTagValueRule] Updating tag for measurement 'iwinfo_value' instance=wlan0 to instance=wl0-ap0: #185 actions