search

monasca / monasca-docker

Docker files and setup for Monasca
Apache License 2.0
24 stars 47 forks source link

persister components crash #493

Open zhangjianweibj opened 5 years ago

zhangjianweibj commented 5 years ago

hello,monasca persister components crashed after run a long time.

image

error:InfluxDBClientError: 400: {"error":"unable to parse points"}

is metrics format error?

zhangjianweibj commented 5 years ago

u'df.inodes.total,_region=useast,_tenant_id=47bb0f0c63064fc7b777a5da50b3c0fc,availableZone=cn-north-3a,endpoint=salt-haproxy,fstype=ext4,instance_name=test,mount=/,service=vm,serviec_type=ecs,step=30,tenant_id=d8e805fc-bf4f-4d2d-93a5-3a8a988cb2f2,type=GAUGE,uuid=3143c43a-f794-4685-8a99-87f466bfc6c6 value=2485504.0,value_meta="{}" 1548321460000']:

matrixik commented 5 years ago

Hmm, never encountered such error.

Could you search for points in influxdb logs? What version of influxdb you are using?

matrixik commented 5 years ago

What's more strange that after looking to influxdb code this error should return more information and not end after points.

matrixik commented 5 years ago

Other problem could be that this repo contain quite old monasca-persister, I should probably push more to have https://github.com/openstack/monasca-persister/tree/master/docker pushed to docker hub. But I will be on vacations next week.

zhangjianweibj commented 5 years ago

very thanks.i guess too many metrics that belong influxdb resolve limitation.before monasca-persister crash, i find hundreds of metrics fetch from kafka.i am debugging the components.before find reason i hava two proposal.first one,write metrics to storage failed in some situtation,monasca persister can abandon those metrics, not reasonable to run os.exit(). image

second on : monasca persister can fetch metrics from kafka with limited metrics.at present," if len(self._data_points) >= self._batch_size:" can not achieve the goal.

matrixik commented 5 years ago

This issue tracker is wrong place to report this. We are looking here mostly for docker part. If you want to report problems with Monasca code itself then OpenStack Story Board is proper place. Here is board with Monasca bugs: https://storyboard.openstack.org/#!/worklist/213

You will need to fill new story and assign monasca-persister as a relevant project. Then tag it as a bug.

Here some introduction to StoryBoard: https://www.youtube.com/watch?v=n_PaKuN4Skk They are showing how to fill story around 5:30 min.

Here example of reported bug for monasca-persister: https://storyboard.openstack.org/#!/story/2003202

zhangjianweibj commented 5 years ago

hello,@matrixik i fix this issue:

https://review.openstack.org/#/c/638589/ https://storyboard.openstack.org/#!/story/2005069