pinpoint-apm / pinpoint

APM, (Application Performance Management) tool for large-scale distributed systems.
https://pinpoint-apm.gitbook.io/
Apache License 2.0
13.29k stars 3.75k forks source link

How to get the collector and web binaries that support uri statistics and hbase2? #10045

Open toushi1st opened 1 year ago

toushi1st commented 1 year ago

Hello, I deployed pinpoint-hbase2-collector-boot-2.5.2.jar and pinpoint-hbase2-web-boot-2.5.2.jar. But uri statistics does not seem to be supported. Non-base2 versions have dedicated binaries, such as Pint-collector-starter-boot-2.3.2.jar. Do collector and web for hbase2 support uri statistics? How to get the collector and web binaries that support uri statistics and hbase2?

emeroad commented 1 year ago

The url statistics feature requires an apache pinot.

Please refer to the uri stat guide. https://pinpoint-apm.gitbook.io/pinpoint/documents/uri_statistics

toushi1st commented 1 year ago

Thank you for your reply. I have deployed apache pinot and kafka, created tables and topics by document. The current problem is that after starting pinpoint-hbase2-collector-boot-2.5.2.jar, the collector and kafka are not connected, and no message is generated to uri-stat topic. Comparison pinpoint-collector-starter-boot-2.3.2.jar, pinpoint-hbase2-collector-boot-2.5.2.jar also does not have files such as pinot-java-client-0.11.0.jar,pinot-jdbc-client-0.11.0.jar,etc. so there are questions in the title. If it is convenient for you, please help me to check whether there is any problem with the following configuration.Thank you.

kafka version:3.4.0

collector Start command:

nohup java -jar -Xmx8g -Xms8g -Xmn2g -XX:+UseConcMarkSweepGC -XX:ParallelGCThreads=8 -XX:+CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=70 pinpoint-collector-starter-boot-2.5.2.jar - -spring.config.additional-location=./config/collector.properties > /opt/pinpoint/logs/collector-console.log 2>&1 &

Two collector.properties were used separately

No 1:

pinpoint.zookeeper.address=zookeeper01:2181,zookeeper02:2181,zookeeper03:2181 flink.cluster.enable=true flink.cluster.zookeeper.address=zookeeper01:2181,zookeeper02:2181,zookeeper03:2181 flink.cluster.zookeeper.sessiontimeout=3000

pinpoint.pinot.jdbc.url=jdbc:pinot://pinot:9000 pinpoint.pinot.jdbc.username=userId pinpoint.pinot.jdbc.password=password

pinpoint.metric.kafka.bootstrap.servers=kafka01:9092,kafka02:9092,kafka03:9092 pinpoint.collector.type=ALL collector.stat.uri=true

No 2:

spring.profiles.active=release pinpoint.zookeeper.address=zookeeper01:2181,zookeeper02:2181,zookeeper03:2181 flink.cluster.enable=true flink.cluster.zookeeper.address=zookeeper01:2181,zookeeper02:2181,zookeeper03:2181 flink.cluster.zookeeper.sessiontimeout=3000

pinpoint.pinot.jdbc.url=jdbc:pinot://pinot:9000 pinpoint.pinot.jdbc.username=userId pinpoint.pinot.jdbc.password=password

pinpoint.metric.kafka.bootstrap.servers=kafka01:9092,kafka02:9092,kafka03:9092 pinpoint.collector.type=ALL collector.stat.uri=true kafka.uri.topic=uri-stat pinpoint.pinot.tenantId=DefaultTenant

toushi1st commented 1 year ago

Append: 1) When pinpoint-collector-starter-boot-2.5.2.jar was used, the configuration information about kafka and uriStat would appear in the startup log

06-15 10:51:02.002 [ main] DEBUG c.n.p.c.c.CollectorConfiguration : 41 -- boolean metricJmxEnable @Value("${collector.metric.jmx:false}") = false 06-15 10:51:02.002 [ main] DEBUG c.n.p.c.c.CollectorConfiguration : 41 -- String metricJmxDomainName @Value("${collector.metric.jmx.domain:pinpoint.collector.metrics}") = pinpoint.collector.metrics 06-15 10:51:02.002 [ main] DEBUG c.n.p.c.c.CollectorConfiguration : 41 -- boolean uriStatEnable @Value("${collector.stat.uri:false}") = true 06-15 10:51:02.002 [ main] DEBUG c.n.p.c.c.CollectorConfiguration : 41 -- boolean statisticsAgentStateEnable @Value("${collector.statistics.agent-state.enable:false}") = true 06-15 10:51:02.002 [ main] INFO c.n.p.p.k.KafkaConfiguration : 39 -- kafka bootstrap.servers:kafka01:9092,kakfa02:9092,kafka03:9092 06-15 10:51:02.002 [ main] DEBUG c.n.p.p.k.KafkaConfiguration : 40 -- kafka config:KafkaProperties{bootstrapServers='kafka01:9092,kakfa02:9092,kafka03:9092', keySerializer='org.apache.kafka.common.serialization.StringSerializer', valueSerializer='org.springframework.kafka.support.serializer.JsonSerializer', partitionerClass='org.apache.kafka.clients.producer.internals.DefaultPartitioner', acks='1', compressionType='zstd'} 06-15 10:51:02.002 [ main] DEBUG uration$$EnhancerBySpringCGLIB$$6d0770ae: 23 -- pinot tenantId:DefaultTenant 06-15 10:51:02.002 [ main] INFO c.n.p.c.h.g.GrpcAgentStatHandlerV2 : 44 -- GrpcMetricHandler:com.navercorp.pinpoint.collector.handler.grpc.metric.AgentMetricHandler@77dba4cd 06-15 10:51:02.002 [ main] INFO c.n.p.c.h.g.GrpcAgentStatHandlerV2 : 44 -- GrpcMetricHandler:com.navercorp.pinpoint.collector.handler.grpc.metric.AgentMetricBatchHandler@2525a5b8

2)When pinpoint-hbase2-collector-boot-2.5.2.jar was used, the same log as above was not seen.

So,does pinpoint-hbase2-collector-boot-2.5.2.jar support uri statistics? The reasons described above makes me feel confused. If not, which jar files could be used for hbase2?

NobleTuna commented 11 months ago

Is this resolved? I seem to have the same issue. I made necessary preparations such as pinot and kafka, but it seems that the uristat function cannot be used with pinpoint-hbase2-collector-boot-2.5.2.jar.

toushi1st commented 11 months ago

Is this resolved? I seem to have the same issue. I made necessary preparations such as pinot and kafka, but it seems that the uristat function cannot be used with pinpoint-hbase2-collector-boot-2.5.2.jar.

Haven't solved yet.

aayushdgkd commented 5 months ago

we are also experiencing the same issue, is there any planned support of uristats with hbase2 compatible jars?

ga-ram commented 5 months ago

hbase2 will be used for images in v3.0.0