search

pinpoint-apm / pinpoint

APM, (Application Performance Management) tool for large-scale distributed systems.
https://pinpoint-apm.gitbook.io/
Apache License 2.0
13.42k stars 3.76k forks source link

is it a bug for mongodb? #5144

Closed shotgun88 closed 5 years ago

shotgun88 commented 5 years ago

i can detective mysql、redis but not mongodb. could u explain why?thank u a lot image

RoySRose commented 5 years ago

Hello, @shotgun88

There is a critical bug in mongodb. #5141 But still, I don't think it will effect the detection. Are you using mongodb java driver?

shotgun88 commented 5 years ago

yes,i am using mongodb java driver. and also i am using the latest version 1.8.1 of pinpoint. so i see that the lastest version can detect mongodb,but actually it seems not work out. could u please let me know when will be fixed up?

RoySRose commented 5 years ago

@shotgun88 It's planned to be released in 1.8.2 I don't think the bug effects the detection. What you are facing might be something else.

here is the fixed version of agent if you'd like to try. It would be great if you can help with the testing. https://github.com/RoySRose/pinpoint/releases/download/temporary/pinpoint-agent-1.8.2.jar

shotgun88 commented 5 years ago

Sorry,the link was wrong? image

shotgun88 commented 5 years ago

i got the jar files image

but please let me know where i put it on?

RoySRose commented 5 years ago

@shotgun88 Sorry, I didn't make it public.

https://github.com/RoySRose/pinpoint/releases/tag/temp

also changed the download just like an ordinary release. you can replace with 1.8.1

RoySRose commented 5 years ago

@shotgun88 let me know if you have any updates

shotgun88 commented 5 years ago

i unzip files and renew configuration,but my application did turn up image image image and is it mongo's version problem before?beacause my mongodb's version is 3.0.15 that lead to cant not able to detected?

shotgun88 commented 5 years ago

also i change mongodb's version is 3.6.7,it also cant not able to detected image it seems mongo's plugin didnt work out that cant not able to be detected image

RoySRose commented 5 years ago

@shotgun88

The version of Mongodb itself has nothing to do with detecting. plugins should be 1.8.2-SNAPSHOT, not 1.8.1 maybe 1.8.2-SNAPSHOT is not applied. please check the version again

RoySRose commented 5 years ago

@shotgun88 What is the version of Pinpoint-Collector and Pinpoint-Web?

shotgun88 commented 5 years ago

for previous 1.8.2-SNAPSHOT can not able to use,even can not able to turn up my application. so i change back to 1.8.1 which the latest version for release 1.8.1 of collect、web but my question is why can not to detect mongodb?even i change mongo's version to 3.6.7(i guess is mongo's version problem before) now i think it's maybe somewhere to block the detecting mongo,it didnt make sense why i can detect mysql、redis not mongo could u explain why this happen?

RoySRose commented 5 years ago

I think, with Pinpoint-Agent : 1.8.2-SNAPSHOT Pinpoint-Collector : 1.8.1 Pinpoint-Web : 1.8.1 There shouldn't be any problem.

I can't point out the reason why detecting mongodb fails in your environment. I have insufficient information on your environment to guess the reason for detection problem.

shotgun88 commented 5 years ago

maybe i can provide some information to pick up the probelm. what do you want to show you something else?

RoySRose commented 5 years ago

Version of Pinpoint-Web and Pinpoint-Collector must be 1.8.1+. The bug in plugin 1.8.1 might be the cause of the detection problem. (even it has a small probability) So, I suggest you change the Pinpoint-Agent to 1.8.2 first.

If you are facing some problem with upgrading the version of Pinpoint-Agent. Just replace the pinpoint-mongodb-driver-plugin-1.8.1 with pinpoint-mongodb-driver-plugin-1.8.2-SNAPSHOT which is in the file of the link above. If you are doing this, be sure to change the name of file from pinpoint-mongodb-driver-plugin-1.8.2-SNAPSHOT -> pinpoint-mongodb-driver-plugin-1.8.1. when replacing.

and restart your apllication

shotgun88 commented 5 years ago

i just did what u told,but it still not work out image so what can i provide some clues to u in next step?

RoySRose commented 5 years ago

@shotgun88 I think we can start from here.

  1. What the version of mongodb-java-driver?
  2. change the log level to debug. And please check the log when your application start-up, and check if there isn't any error around it. It should be around the log as below 
    2019-01-17 18:37:35 [DEBUG](c.n.p.p.i.c.URLClassLoaderHandler  :100) add Jar:file:/Users/user/IdeaProjects/pinpoint/agent/target/pinpoint-agent-1.8.2-SNAPSHOT/plugin/pinpoint-mongodb-driver-plugin-1.8.2-SNAPSHOT.jar
2019-01-17 18:37:36 [DEBUG](c.n.p.p.o.AutoBindingObjectFactory :95 ) Create instance by constructor public com.navercorp.pinpoint.plugin.mongo.interceptor.MongoDriverGetDatabaseInterceptor(com.navercorp.pinpoint.bootstrap.context.TraceContext,com.navercorp.pinpoint.bootstrap.context.MethodDescriptor), with arguments [com.navercorp.pinpoint.profiler.context.DefaultTraceContext@7f34a967, {className=com.mongodb.MongoClient, methodName=getDatabase, parameterTypes=[java.lang.String], parameterVariableName=[databaseName], parameterDescriptor=(java.lang.String databaseName), apiDescriptor=com.mongodb.MongoClient.getDatabase(java.lang.String databaseName), lineNumber=550, apiId=145, fullName=com.mongodb.MongoClient.getDatabase(java.lang.String databaseName):550, type=0}]
2019-01-17 18:37:36 [DEBUG](c.n.p.p.BaseClassFileTransformer   :47 ) [transform] classLoader:ParallelWebappClassLoader
  3. Please check the log if there isn't any errors.
  4. Check Collector log if collector has received mongodb data from the agent. it should be something like below.(below log means mongodb data has been received from the Pinpoint-Agent 
    2019-01-17 18:52:57 [DEBUG](c.n.p.c.h.t.ThriftSpanHandler      :67 ) Received SPAN=TSpan(agentId:roy-app, applicationName:roy-app, agentStartTime:1547718733242, transactionId:00 01 BA DB E6 D9 85 2D 05, spanId:-8960468173174276706, parentSpanId:-1, startTime:1547718777366, elapsed:328, rpc:/mongodb/filterGEO.pinpoint, serviceType:1010, endPoint:localhost:9090, remoteAddr:0:0:0:0:0:0:0:1, annotations:[TAnnotation(key:46, value:<TAnnotationValue intValue:200>)], flag:0, err:0, spanEventList:[TSpanEvent(sequence:4, startElapsed:175, endElapsed:59, serviceType:2651, endPoint:10.106.157.82:27017, annotations:[TAnnotation(key:151, value:<TAnnotationValue stringValue:customers>), TAnnotation(key:152, value:<TAnnotationValue stringValue:secondary>), TAnnotation(key:150, value:<TAnnotationValue stringStringValue:TStringStringValue(stringValue1:{ "loc" : { "$geoWithin" : { "$polygon" : [["?", "?"], ["?", "?"], ["?", "?"], ["?", "?"]] } } }, stringValue2:28.56402,79.93652,27.27569,26.16394,42.69404,20.02808,48.61541,51.37207)>)], depth:5, destinationId:myMongoDb, apiId:361), TSpanEvent(sequence:3, startElapsed:62, endElapsed:172, serviceType:5071, depth:4, apiId:35), TSpanEvent(sequence:2, startElapsed:10, endElapsed:288, serviceType:5071, depth:3, apiId:263), TSpanEvent(sequence:1, startElapsed:2, endElapsed:326, serviceType:5051, depth:2, apiId:-3), TSpanEvent(sequence:0, startElapsed:1, endElapsed:327, serviceType:1011, depth:1, apiId:-2)], parentApplicationType:0, apiId:2, applicationServiceType:1010, loggingTransactionInfo:0)
  5. Check Hbase if there is any data.
shotgun88 commented 5 years ago

i am using the version of mongodb-java-driver is 3.2.2 image and i've checked my application's log appears pinpoints detect mongodb's information,but seems not as u mention,please see below image image besides,i cant find out any information about mongodb in colletor's log image

RoySRose commented 5 years ago

@shotgun88 if you can't find any info about mongodb in Pinpoint-Collector's log. try to find it in Pinpoint-Agent's log first.(change the log level to DEBUG, before running it)

shotgun88 commented 5 years ago

can you tell me where to set up log's level for Pinpoint-Agent?

RoySRose commented 5 years ago

please check FAQ

shotgun88 commented 5 years ago

here is agent's log,please check as below: image

shotgun88 commented 5 years ago

it seems the agent just init mongo's configuration,but not send mongo's data information to collect,please see agent's sending data as below: image

please see collect's log as below: image image

RoySRose commented 5 years ago

@shotgun88 were there any transaction flow to mongodb? by mongodb java driver? for example. any insertion or update... CRUD?

shotgun88 commented 5 years ago

maybe insert some data to mongodb when i just operated on application. but no mongodb's information to send even comes up in log

RoySRose commented 5 years ago

  1. please check if plugin is disabled in pinpoint.config file

    profiler.mongo=true
profiler.mongo.collectjson=true
profiler.mongo.tracebsonbindvalue=true

  2. can you show me your code? the on where you execute insert.

RoySRose commented 5 years ago

@shotgun88 any updates?

shotgun88 commented 5 years ago

i've checked agent's mongo configuration was open as below: image and i try many times to check everywhere,looks like all fine,but still can not able to detected mongodb.i have no idea where to goes wrong.....

RoySRose commented 5 years ago

@shotgun88

can I have a small look at your application code. where you CRUD. for example 

collection.insertOne()
shotgun88 commented 5 years ago

please see below: image

RoySRose commented 5 years ago

@shotgun88 how did you get your mongoDatabase() in the first place? by mongoClient.getDatabase method?

shotgun88 commented 5 years ago

please see below: image

RoySRose commented 5 years ago

@shotgun88 I've created a simple application wit spring-data-mongo, and checked it work. I'll test few more things and let you know the results

RoySRose commented 5 years ago

Hello, @shotgun88

I've checked with spring-data, spring-boot. All came up fine. I'd like to reproduce what you are experiencing, but have no clue.

If you can send me the full DEBUG log of Agent, Collector, Web. I'll take a look.

Feel free to reopen this issue if you'd like.

shotgun88 commented 5 years ago

Hi RoySRose please check logs as below:

mbox144-pinpoint.log service144-pinpoint.log stat.log catalina.txt

RoySRose commented 5 years ago

Hello, @shotgun88 I've looked at the logs. couldn't find anything special. I've assumed how you're using mongodb-java-driver with piece of your code. and tried to regenerate the problem. But couldn't, I've checked the plugin working with spring-data-mongo (with and without) spring-boot.

fixed plugin has been released, why don't you try with v1.8.2-RC1

shotgun88 commented 5 years ago

i am sorry to let u know that is not aevalible for v1.8.2-RC1 it is still can not work it out for mongodb please see detail as below: image image catalina.txt [Uploading stat.log…]() [Uploading mbox144-pinpoint.log…]() [Uploading service144-pinpoint.log…]()

shotgun88 commented 5 years ago

mbox144-pinpoint.log service144-pinpoint.log stat.log

shotgun88 commented 5 years ago

Hi RoySRose @RoySRose can i provide the application to u guys to see what exactly happen(why can not able to detect mongodb),please send me e-mail address

RoySRose commented 5 years ago

oh~ really? of course~ maybe this thread is getting too long. provide me with the application on here

shotgun88 commented 5 years ago

Hi RoySRose that email address was not able to access which blocked email send me back(i've been try twice). have u got any email address?

RoySRose commented 5 years ago

sungwook0115.kim@gmail.com would do the trick.

shotgun88 commented 5 years ago

please see below with attackment

application.zip

RoySRose commented 5 years ago

@shotgun88

how should I set this application.zip to work? any advice?

can I just start it with start.sh?

shotgun88 commented 5 years ago

unzip the application and fix conf/application.properties of configuration as yours finally add agent's configuration on start.sh as below: image

RoySRose commented 5 years ago

@shotgun88 what next :)

I've run... sh start.sh and no transactions going back and forth

there is properties on dubbo, and... localhost? do I need anything additional in local?

image

shotgun88 commented 5 years ago

@RoySRose yes,i think it sign up mongo and dubbo configuration on that conf/application.properties in your enviroment. and run sh bin/start.sh finally watch logs show up the application was start up with finish detail then it will be all things right as below: image

RoySRose commented 5 years ago

I've set mongodb and dubbo config. also started a zookeeper with docker for dubbo.

and now I get... I can't find where 10.66.96.84 came from since there isn't such address in the properties. maybe it's in the source code.

guessing with the log. it's trying to start up a server. Since I can't see the source code. I can't assume what it can be.

image

shotgun88 commented 5 years ago

counld u please send me some information about configuation?

RoySRose commented 5 years ago

this is it. I've started up a zookeeper on localhost:2181 mongodb also taken care of. that's all I've done

# dubbo config
dubbo.application=poggyio-mbox
dubbo.protocol=dubbo
dubbo.protocol.host=localhost
dubbo.protocol.port=20804
dubbo.protocol.payload=33554432
dubbo.registry=127.0.0.1:2181
dubbo.logger=slf4j
dubbo.qos.enable=false
dubbo.qos.port=22224

# mongodb config
#mongodb.host=10.106.157.82
#mongodb.port=27017
mongodb.dbname=poggyio
mongodb.username=
mongodb.password=
mongodb.write-concern=SAFE
mongodb.authentication.dbname=
mongodb.replica-set=localhost:27017
mongodb.credentials=

mongodb.options.connectionPerHost=64
mongodb.options.connectionTimeout=3000