influxdata / influxdb

Scalable datastore for metrics, events, and real-time analytics
https://influxdata.com
Apache License 2.0
28.82k stars 3.55k forks source link

[feature request] RENAME TAG #4157

Open beckettsean opened 9 years ago

beckettsean commented 9 years ago

From https://github.com/influxdb/influxdb/issues/828#issuecomment-83224779

e.g. RENAME TAG foo TO bar ON measurement

Would have to update all CQs, too.

victorhooi commented 9 years ago

Out of curiosity, what's a CQ?

otoolep commented 9 years ago

Continuous query.

victorhooi commented 9 years ago

Aha, thanks =).

yujinqiu commented 9 years ago

+1, I love this feature

mitar commented 8 years ago

+1

mitar commented 8 years ago

What about changing the tag value?

toni-moreno commented 8 years ago

:+1:

We need both , tag key change and also tag value change !

Lots of thanks for this awesome tool.

sthorsen commented 8 years ago

+1

kb1ibt commented 8 years ago

@toni-moreno I have a perfect use case as to why I need to be able to change values. I use collectd to pull in my SNMP values. I performed a firmware upgrade on a line of switches and the ifDesc field name changed on all 8 switches. So now all of the type_instance tag values are different. For example for interface 32 on switch 1 the ifDesc used to be "TenGigabitEthernet 1_0_32" but now that interface is "TenGigabitEthernet 0_32" So the graphs and data retention are broken. Being able to alter/rename/update all tags from before the firmware update would allow the data to be linked up.

piaseckim commented 8 years ago

+1

scambra commented 7 years ago

I have same use case about SNMP values and waiting for long time for this feature. In the meantime, I dump series, rename tag value with sed and import again, but it's very slow process when I have many interfaces changes.

hraftery commented 7 years ago

+1. I can't find a workaround. I can workaround the measurement and field rename, but not the tag.

Eg. SELECT "old_field_key" AS "new_field_key" INTO new_measurement FROM old_measurement GROUP BY "old_tag_key" renames field key and measurement, but cannot be extended to rename the tag key.

sandervandegeijn commented 7 years ago

I'm having the exact same problem as hraftery piecing together data from two databases that contain data about the same measurement but with different names. Another solution would something like

Select sum(value) as value, counter=blub into newmeasurement

so I can tag the new data that going into newmeasurement.

apollotonkosmo commented 7 years ago

Have anybody found a proper way or a workaround? I am also having the same issue. I renamed a host and now I need to merge the old tagged value of the host with the new one. Is that possible? Thanks George

mribichich commented 7 years ago

same here is there a workaround? In my case the scheme changed, and I need to rename a tag like renaming a column name

max3163 commented 7 years ago

+1 must have feature

discoduck2x commented 7 years ago

@beckettsean any update on this?

curio77 commented 7 years ago

+1 (really not seeing how this has not been implemented yet, essential as it seems dealing with historical data)

rjain11 commented 6 years ago

Any update on this? We have a use case of updating existing tag values as well.

dandv commented 6 years ago

How about updating a tag for points matching a query?

In the Key Concepts sample data, imagine that one of the scientists mistakenly recorded the location for measurements between timeA and timeB as 1 instead of 2. How can we update those points? Overwriting with the same (measurement name, time, tag set) tuple can only be used to update fields, not tags.

NubeDev commented 6 years ago

+1

pmorch commented 6 years ago

Newbie alert: I'm a little unclear about the scope of this feature request. Does it cover two related but different capabilities?:

  1. Be able to rename a tag key for all series in a measurement. E.g. for Key Concepts sample data, rename "location" -> "place"
  2. Be able to change the a tag value for a one or more series in a measurement. E.g. for Key Concepts sample data, UPDATE census SET "scientist"="perpetua" WHERE "scientist"="miller" (e.g. person changed their last name)

The RENAME TAG foo TO bar ON measurement suggestion from the OP description seems to describe the first situation but not the second.

For our purposes, the second situation, rename a tag value is by far the most important.

Should this issue really be split in two? Is it the same implementation to support both tag key and tag value changes or are the implementations independent? Which is the most difficult to achieve?

hraftery commented 6 years ago

This issue is definitely about the tag key. It's one of a group of issues about renaming keys that used to be one issue.

As dandv and you say, changing tag values is also a challenge, but deserves its own issue because the implications are different. As discussed on StackOverflow and Google Groups, there's no good solution to that either, short of re-inserting all the data and dropping the old. The implication is that tag values form series, so changing a tag value could change the database cardinality. While changing a tag key only affects the metadata.

If you want to go down the dump all, modify, re-import path (brutal, but effective), this comment might help.

scotloach commented 6 years ago

This would be an awesome feature. Right now migrating data when a tag value changes is not really possible.

MuadDib81 commented 6 years ago

+1

yves-bourgeois commented 6 years ago

+1

YEMEAC commented 6 years ago

+1 waiting :)

mecton commented 6 years ago

+1

galanCA commented 6 years ago

+1

As well as changing the tag value.

nativb commented 6 years ago

+1

As well as changing the tag value.

otherplace commented 6 years ago

+1

czhao0378 commented 6 years ago

+1

dandv commented 6 years ago

If you're about to leave a "+1" comment, please stop. Click the emoji icon of one of the messages in this thread that you agree with, and choose :+1: there.

mferjani commented 5 years ago

Hi There, Any news about this issue ?

willowsbrain commented 5 years ago

+1

painhardcore commented 5 years ago

Any chance that it will be in 2.x?

lydon42 commented 5 years ago

+1

bussyjd commented 5 years ago

+1

yangxianbo commented 5 years ago

+1

lexmag commented 5 years ago

Rather than +1 in comments it is way better to react with 👍 under the initial comment. There are many people who have subscribed to this issue to receive a notification when it will be closed (hopefully implemented), and those +1 only create unnecessary noise for everyone.

liamdiprose commented 5 years ago

SELECT "old_field_key" AS "new_field_key" INTO new_measurement FROM old_measurement GROUP BY "old_tag_key"

The GROUP BY syntax seems the most intuitive to me.

In the meantime, a nice workaround I've found is to use Python and the aioinflux package. To rename the host tag to host2:

async with aioinflux.InfluxDBClient(host='influxdb', db='mydb', output='dataframe') as client:
    df = await client.query("SELECT host as host2,* FROM measurement")
    delete df['host']
    await client.write(df, measurement="measurement_new", tag_columns=["host2"])

I've got a Jupyter Notebook instance running on the same machine as influxdb, which makes workarounds like this quite painless.

prczsf commented 4 years ago

+1

neilee commented 4 years ago

+1

oktal commented 4 years ago

Hello,

Altering metadata is a major limitation of InfluxDB. Since InfluxQL does not support alter query types for measurements, tags or fields, I decided to implement a tool as a workaround.

The tool supports a RenameTag rule to rename a tag key from a measurement. It is an offline tool, so the node must be taken down first (the exact same way you would run influx_inspect buildtsi).

Please check it and let me know if it helps.

https://github.com/Abc-Arbitrage/infix

Any feedback is also welcome.

justinnamilee commented 1 year ago

+1 plz

bcutter commented 1 year ago

Seeing this request is 8 years old I have to summarize: this project is likely dead, isn't it.

drewyh999 commented 1 year ago

Interesting enough, this feature is still open...

ipksupport commented 1 year ago

At the moment this is the way I do it: Updating tags and fields at will: In influx select the data you need to change and copy it in to a new database, you can select the data you want to change to limit the amount of data: SELECT INTO datatochange99999..:MEASUREMENT FROM original99999.././ WHERE "siteid" =~ /ALM_HQ|AMS_DC/ AND "sitename" !~ /ALM_HQ|AMS_DC/ AND time > now() - 365d GROUP BY *

Then export the new db in influx line protocol: sudo influx_inspect export -datadir /var/lib/influxdb/data -waldir /var/lib/influxdb/wal -database datatochange99999 -compress -lponly -out /home/localadmin/tagchange/test.out.gz

Use sed etc to change whatever you need in test.out.gz and import this file again with for example curl, you can use the linux split tool to split the output into manageble sizes to use curl or other to import the changed data.

If you want you can pipe it all together in a oneliner and "curl" it in :-), I can provide an example if needed. After import you need to delete the "old" incorrect data from the database. Best is to make a backup before you start as allways...