shell909090 / influx-proxy

Other
487 stars 272 forks source link

Write body received by handler一直未写入 #107

Open XuankuF opened 4 years ago

XuankuF commented 4 years ago

你好,我写两条数据,一直未写入,命令如下: curl -i -X POST 'http://192.168.33.33:8086/write?db=monitor' --data-binary 'temperature,host=s2 value=0.6499999' curl -i -X POST 'http://192.168.33.33:8086/write?db=monitor' --data-binary 'temperature,host=s3 value=0.88888881212211221' 日志一直打印:

2020/08/11 14:53:35.043490 http.go:140: Write body received by handler: temperature,host=s2 value=0.6499999
temperature,host=s3 value=0.88888881212211221
,the client is 192.168.33.33:51389
2020/08/11 14:53:35.244816 http.go:140: Write body received by handler: temperature,host=s2 value=0.6499999
temperature,host=s3 value=0.88888881212211221

查询: curl -v -X GET 'http://192.168.33.32:8086/query?db=monitor' --data-urlencode "q=SELECT host FROM temperature where time< now()" 日志打印:

*   Trying 192.168.33.32...
* TCP_NODELAY set
* Connected to 192.168.33.32 (192.168.33.32) port 8086 (#0)
> GET /query?db=monitor HTTP/1.1
> Host: 192.168.33.32:8086
> User-Agent: curl/7.60.0
> Accept: */*
> Content-Length: 66
> Content-Type: application/x-www-form-urlencoded
> 
* upload completely sent off: 66 out of 66 bytes
< HTTP/1.1 400 Bad Request
< X-Influxdb-Version: 1.1
< Date: Tue, 11 Aug 2020 06:54:32 GMT
< Content-Length: 11
< Content-Type: text/plain; charset=utf-8
< 
* Connection #0 to host 192.168.33.32 left intact
shell909090 commented 4 years ago

呃...32是啥?33是啥?

XuankuF commented 4 years ago

做的两台主机的HA 以下是config.py的主体:

BACKENDS = {
    'local': {
        'url': 'http://localhost:8086', 
        'db': 'monitor', 
        'zone':'local', 
        'interval': 1000,
        'timeout': 10000, 
        'timeoutquery':600000, 
        'maxrowlimit':10000,  
        'checkinterval':1000, 
        'rewriteinterval':10000,
    },
    'local2': {
        'url': 'http://192.168.33.32:8086',
        'db': 'monitor',
        'interval': 200,
        'zone':'local'
    }
}

KEYMAPS = {
    'cpu': ['local'],
    'temperature': ['local2'],
    'default': ['local','local2']
}

NODES = {
    'l1': { 
        'listenaddr': ':8086',
        'db': 'monitor',
        'zone': 'local',
        'interval':10,
        'idletimeout':10,
        'writetracing':1,
        'querytracing':1,
    }
}

DEFAULT_NODE = {
    'listenaddr': ':8086'
}
shell909090 commented 4 years ago

哦。 那influxdb的地址呢?

XuankuF commented 4 years ago

需要定义么,我以为是http://192.168.33.32:8086(主机ip:8086端口)

shell909090 commented 4 years ago

呃,不是,等会,我有点晕。 http://192.168.33.32:8086上面,到底跑的是influx-proxy的程序,还是influxdb的程序

XuankuF commented 4 years ago

是influx-proxy,两台主机上装了influxdb但是没起服务

shell909090 commented 4 years ago

呃,可能我在文档里说的不是很清楚。influx-proxy是通过将请求代理到不同influxdb的方式来进行分片的。如果你弄了两台influx-proxy,却没有启influxdb。我能想到的正常结果就是。influx-proxy将请求循环发送给自己导致死循环。 这个听起来配置有点问题。

shell909090 commented 4 years ago

举例来说,如果你有机器A和B,希望做高可用同时分片。你可以这么配置。 A:8087 influxdb A B:8087 influxdb B A:8086 influx-proxy A B:8086 influx-proxy B

BACKENDS = { 'influxA': { 'url': 'http://A:8087', 'db': 'monitor', 'zone':'local', 'interval': 1000 }, 'influxB': { 'url': 'http://B:8087', 'db': 'monitor', 'interval': 1000, 'zone':'local' } }

XuankuF commented 4 years ago

多谢指正。 更改完后向monitor库写入数据,返回204,没有报错,但是influx上没有monitor库的,我手动去两个influx服务上分别建了monitor库,再写write

curl -XPOST 'http://192.168.33.32:6666/write?db=monitor' --data-binary 'temperature,host=s2 value=0.6499999'

从influx-proxy日志中可以看到写入和查询日志,但是两个influx服务的日志中没有打印:

2020/08/12 09:47:24.529942 http.go:140: Write body received by handler: temperature,host=s1 value=0.6499999,the client is 192.168.33.34:44877
2020/08/12 09:47:31.100366 cluster.go:465: new measurement: influxdb.cluster
2020/08/12 09:47:37.034263 http.go:140: Write body received by handler: temperature,host=s3 value=0.6499999,the client is 192.168.33.34:44878
2020/08/12 09:47:41.100215 cluster.go:465: new measurement: influxdb.cluster
2020/08/12 09:47:51.100209 cluster.go:465: new measurement: influxdb.cluster
2020/08/12 09:47:52.567998 http.go:90: the query is select host from temperature where time< now(),the client is 192.168.33.34:44879
2020/08/12 09:48:01.100356 cluster.go:465: new measurement: influxdb.cluster

查询返回:

curl -GET 'http://192.168.33.32:6666/query?db=monitor' --data-urlencode "q=select host from temperature where time< now()"
{"results":[{"statement_id":0}]}
shell909090 commented 4 years ago

KEYMAPS有对应改过来么?

XuankuF commented 4 years ago

现在是: (192.168.33.32:8086和192.168.33.33:8086是两个influxdb)

BACKENDS = {
    'influx1': {
        'url': 'http://192.168.33.32:8086', 
        'db': 'monitor', 
        'zone':'local', 
        'interval': 1000,
        'timeout': 10000, 
        'timeoutquery':600000, 
        'maxrowlimit':10000,  
        'checkinterval':1000, 
        'rewriteinterval':10000,
    },
    'influx2': {
        'url': 'http://192.168.33.33:8086',
        'db': 'monitor',
        'zone':'local', 
        'interval': 1000,
        'timeout': 10000, 
        'timeoutquery':600000, 
        'maxrowlimit':10000,  
        'checkinterval':1000, 
        'rewriteinterval':10000,
    }
}

KEYMAPS = {
    'cpu': ['influx1'],
    'temperature': ['influx2'],
    'default': ['influx1']
}
shell909090 commented 4 years ago

日志非常奇怪。从Write body received by handler来看,你的请求确实进了系统。但是从new measurement: influxdb.cluster来看,识别出来的measurement是influxdb.cluster,而且时间隔的太长了。建议在 https://github.com/shell909090/influx-proxy/blob/master/backend/cluster.go#L447 加一行调试语句,看看分发之前的记录到底是什么样子的。