Closed melson-jao closed 8 years ago
jwilder
commented
8 years ago @melson-jao Can you update this issue with the required information we need for bug reports as described in our issue template? There is not enough information here to help to you.
If you get a panic, please include the full trace, not just the first line. We also need to know what version of the server your are running. If you have way for us to reproduce the issue, that will greatly speed up our ability to help you.
melson-jao
commented
8 years ago Hi,
I update some information and crash log. Please let me know if anything is still not clear.
And about the reproduce procedure, I guess the point is some old format data is still existed at /var/lib/influxdb/data/ after drop measurement. I do not know why. Appreciate for any explanation about what drop measurement will really do, ex delete recent data or all history data of this measurement. And then new format data comes and saved without problem because new measurement will be created with new format. When influxdb restart to load data, it is confused and crash.
jwilder
commented
8 years ago @melson-jao Thanks. I think what might have happened is that a compaction was running while the drop measurement ran and the compaction completed writing the old int64 type. New data was stored as float64. When a subsequent compaction ran, it had two files with different types.
melson-jao
commented
8 years ago Basically, I stop writing data to measurement before I drop measurement xxx via web GUI 8083. I try to avoid drop measurement and write data happen at the same time. After drop measurement and star to write dat again and as I said, at that moment it is fine : new measurement is created and new data with new format is saved. The bad thing happens at influxdb restart. It means crash might happen days later and none can immediately link to the error cause to data format change days ago. This is a risk.
jwilder
commented
8 years ago I just realized you're running 0.10.0. Are you able to reproduce this with 0.12.2 or latest nightlies?
melson-jao
commented
8 years ago Hi jwilder,
I have tried v0.12.2. After 2-3 times test, I cannot reproduce this issue anymore. I think we can close this issue first. It might be fixed at 0.11.x or 0.12.x
Thanks, Melson
Revision: 0.10.0 Reproduce procedure:
Crash Log: Please check the attachment. influxdb-crash-log.txt
BTW, I also reproduce the same issue at v0.9.5.
Here is the story:
I update influxdb-java lib and then some data are saved to influxdb as INT64 but there is existed FLOAT64 data there. So, I try to drop existed measurement and all new data are saved to new created measurement as INT64 without any error, looks fine.
But, when I restart influxdb, it is panic with message like "panic: interface conversion: tsm1.Value is tsm1.Int64Value, not tsm1.FloatValue" And I have to remove several file under /var/lib/influxdb/data/[db-name]/default/XXX/xxx.tsm1 to make influxdb come back to service again.
Can anyone provide some advice or document about how to update measurement if some data format is changed? Or how to clean remove the old measurement data to avoid influxdb load it at startup? Look like "drop measuremetn xxx" is not clean enough.
BTW, after check some related discussion, I decide to update my code with some type cast to force to save all number data to influxdb in float data type to make thing simple and uniform. Is it suggested? Welcome for any better idea or suggestions.
Finally, how about just skip to load unexpected format old data with warning message instead of panic or crash directly? Everything goes fine in running but fail at influxdb restart is a risk. Someone, like me, will think everything is done after measurement recreated but actually it will crash in days or months when influxdb restart happens.