ytyou / ticktock

TickTockDB is an OpenTSDB-like time series database, with much better performance.
GNU General Public License v3.0
72 stars 8 forks source link

tags does not support boolean or numeric values #53

Closed andreish closed 12 months ago

andreish commented 12 months ago

POST /api/put

fails to insert datapoint if tag is boolean or numeric

ylin30 commented 12 months ago

Hi @andreish, could you pls share your repro? I can't repro it. Please refer to my test case below.

  1. Running TT v0.12.0

    [t460p ticktock (main)]$ ./bin/tt -c conf/tt.conf &
    [1] 1978
    [t460p ticktock (main)]$  TickTockDB v0.12.0,  Maintained by
    Yongtao You (yongtao.you@gmail.com) and Yi Lin (ylin30@gmail.com).
    This program comes with ABSOLUTELY NO WARRANTY. It is free software,
    and you are welcome to redistribute it under certain conditions.
    For details, see <https://www.gnu.org/licenses/>.
    Writing to log file: /home/ylin30/ticktock/log/ticktock.log
  2. Add a data point with two tags, one with bool, anther with numeric value.

  3. Then query the data point.

    [Yi-MBP ticktock (migrate)]$ curl -v -XPOST 'http://192.168.1.52:6182/api/put' -d $'put test.cpu.usr 1633412175 125 host=true cpu=1'
    Note: Unnecessary use of -X or --request, POST is already inferred.
    *   Trying 192.168.1.52...
    * TCP_NODELAY set
    * Connected to 192.168.1.52 (192.168.1.52) port 6182 (#0)
    > POST /api/put HTTP/1.1
    > Host: 192.168.1.52:6182
    > User-Agent: curl/7.64.1
    > Accept: */*
    > Content-Length: 47
    > Content-Type: application/x-www-form-urlencoded
    >
    * upload completely sent off: 47 out of 47 bytes
    < HTTP/1.1 200 OK
    < Content-Length: 0
    < Content-Type: text/plain
    <
    * Connection #0 to host 192.168.1.52 left intact
    * Closing connection 0
    [Yi-MBP ticktock (migrate)]$ curl -v 'http://192.168.1.52:6182/api/query?start=1600000000&m=avg:test.cpu.usr'
    *   Trying 192.168.1.52...
    * TCP_NODELAY set
    * Connected to 192.168.1.52 (192.168.1.52) port 6182 (#0)
    > GET /api/query?start=1600000000&m=avg:test.cpu.usr HTTP/1.1
    > Host: 192.168.1.52:6182
    > User-Agent: curl/7.64.1
    > Accept: */*
    >
    < HTTP/1.1 200 OK
    < Content-Type: application/json
    < Content-Length: 106
    <
    * Connection #0 to host 192.168.1.52 left intact
    [{"metric":"test.cpu.usr","tags":{"cpu":"1","host":"true"},"aggregateTags":[],"dps":{"1633412175":125.0}}]* Closing connection 0
    [Yi-MBP ticktock (migrate)]$
andreish commented 12 months ago

Use "cpu": 1 instead of "cpu" : "1"

I used Python json.dumps to create body of post request.

I am mobile now. If neded will update later.

ylin30 commented 12 months ago

yes, please update with your python example.

ylin30 commented 12 months ago

I think I can repro it. Did you use json format instead of plain put format in your request?

[Yi-MBP python (migrate)]$ curl -v -XPOST 'http://192.168.1.52:6182/api/put' -d '[{"metric":"testM2","value":2.33266,"timestamp":1514779734000,"tags":{"host":1}}]'
...
< HTTP/1.1 200 OK
< Content-Length: 24
< Content-Type: application/json
<
* Connection #0 to host 192.168.1.52 left intact
{"success":0,"failed":0}* Closing connection 0
[Yi-MBP python (migrate)]$
[Yi-MBP python (migrate)]$
[Yi-MBP python (migrate)]$
[Yi-MBP python (migrate)]$ curl -v 'http://192.168.1.52:6182/api/query?start=1500000000&m=avg:testM2'
...
< HTTP/1.1 200 OK
< Content-Type: application/json
< Content-Length: 2
<
* Connection #0 to host 192.168.1.52 left intact
[]* Closing connection 0
ylin30 commented 12 months ago

Looks TT always assumes string type in tag value up to v0.12.0. You may hit the problem in json format requests. It will be fixed soon. Maybe a patch v0.12.1.

FYI. We are having a big refactor right now to improve query perf and the next release is not backward compatible. But we will provide a data migration tool to migrate your existing data.

ylin30 commented 12 months ago

@andreish A fix has been checked into dev branch. You can try the dev branch if you build TT from source. It may take a few days to officially release v0.12.1 for the patch due to many tests.

Here is a manual test:

[Yi-MBP python (dev)]$ curl -v -XPOST 'http://192.168.1.41:6182/api/put' -d '[{"metric":"testM1","value":2.33266,"timestamp":1514779734,"tags":{"host":true,"cpu":1}},{"metric":"testM2","value":2.33266,"timestamp":1514779734,"tags":{"host":true,"cpu":1}}]'
Note: Unnecessary use of -X or --request, POST is already inferred.

...
< HTTP/1.1 200 OK
...
{"success":2,"failed":0}* Closing connection 0
[Yi-MBP python (dev)]$ curl -v 'http://192.168.1.41:6182/api/query?start=1500000000&m=avg:testM2'
...
< HTTP/1.1 200 OK
...
[{"metric":"testM2","tags":{"cpu":"1","host":"true"},"aggregateTags":[],"dps":{"1514779734":2.3326600000000002}}]* Closing connection 0
[Yi-MBP python (dev)]$
andreish commented 12 months ago

dev branch comit 5ebb832fd471123a5a9efcf18a7df07ecb1a8ceb fixes it , closing issue

ylin30 commented 11 months ago

@andreish FYI. The fix has been released in v0.12.1.