search

inspectIT / inspectit-ocelot

inspectIT Ocelot - Java agent for collecting application performance, tracing and behavior data
http://www.inspectit.rocks/
Apache License 2.0
202 stars 69 forks source link

InspectIT not able recognize my application ,need help for fetching service graph on grafana #1524

Open sowndarya123 opened 1 year ago

sowndarya123 commented 1 year ago

I have sample application ,application started with agent but was not recognized by UI .I also tried with pet application .Please help with that

danipaniii commented 1 year ago

Hello sowndarya123, Thanks for your inquiry! In order to help you I need a couple more information on what you are trying to do. Are you trying to set up the demo application? And where exactly do you encounter which problems?

sowndarya123 commented 1 year ago

hi Team.. am trying to set up the demo application .I tried the run the scenario 1 on local docker desktop .Some started but others did not . Later i tried to run a sample application with CMR and inspectIT UI ,and agent .I see that CMR got started but when the application is started with agent .I dont see my application on inspectIT UI.

I need your quick help on this. Request for one-one if possible

SaschaBrechmannVHV commented 1 year ago

Hi @sowndarya123 , inspectIT CRM is the very old Version of inspectIT. Do not use this, until you know what you are doing. This version is not mentained any more!

Use the inspectIT Ocelot Version only. About your Question: Could you share the logout put from the agent, after you startet the applikation? do you configured the agent for file based configuration oder http based configuration? What are your commandline to start the application with the agent? Regards, Sascha

sowndarya123 commented 1 year ago

Thank you for the clarity on CMR.

2022-09-27 11:24:31.627 INFO [vets-service,,] 18 --- [nfoReplicator-0] com.netflix.discovery.DiscoveryClient : DiscoveryClient_VETS-SERVICE/vets-service:vets-service:8083 - registration status: 204 2022-09-27 11:24:59.139 INFO [vets-service,,] 18 --- [nio-8083-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring DispatcherServlet 'dispatcherServlet' 2022-09-27 11:24:59.140 INFO [vets-service,,] 18 --- [nio-8083-exec-1] o.s.web.servlet.DispatcherServlet : Initializing Servlet 'dispatcherServlet' 2022-09-27 11:24:59.153 INFO [vets-service,,] 18 --- [nio-8083-exec-1] o.s.web.servlet.DispatcherServlet : Completed initialization in 13 ms 2022-09-27 11:25:10.313 WARN [vets-service,,] 18 --- [/api/v2/spans}}] z.r.AsyncReporter$BoundedAsyncReporter : Spans were dropped due to exceptions. All subsequent errors will be logged at FINE level. 2022-09-27 11:25:10.318 WARN [vets-service,,] 18 --- [/api/v2/spans}}] z.r.AsyncReporter$BoundedAsyncReporter : Dropped 1 spans due to ResourceAccessException(I/O error on POST request for "http://tracing-server:9411/api/v2/spans": tracing-server; nested exception is java.net.UnknownHostException: tracing-server)

org.springframework.web.client.ResourceAccessException: I/O error on POST request for "http://tracing-server:9411/api/v2/spans": tracing-server; nested exception is java.net.UnknownHostException: tracing-server at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:785) at org.springframework.cloud.sleuth.zipkin2.ZipkinRestTemplateWrapper.doExecute(ZipkinRestTemplateWrapper.java:69) at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:660) at org.springframework.cloud.sleuth.zipkin2.RestTemplateSender.post(RestTemplateSender.java:151) at org.springframework.cloud.sleuth.zipkin2.RestTemplateSender$HttpPostCall.doExecute(RestTemplateSender.java:169) at org.springframework.cloud.sleuth.zipkin2.RestTemplateSender$HttpPostCall.doExecute(RestTemplateSender.java:159) at zipkin2.Call$Base.execute(Call.java:391) at zipkin2.reporter.AsyncReporter$BoundedAsyncReporter.flush(AsyncReporter.java:299) at zipkin2.reporter.AsyncReporter$Flusher.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.UnknownHostException: tracing-server at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at java.base/sun.net.NetworkClient.doConnect(Unknown Source) at java.base/sun.net.www.http.HttpClient.openServer(Unknown Source) at java.base/sun.net.www.http.HttpClient.openServer(Unknown Source) at java.base/sun.net.www.http.HttpClient.(Unknown Source) at java.base/sun.net.www.http.HttpClient.New(Unknown Source) at java.base/sun.net.www.http.HttpClient.New(Unknown Source) at java.base/sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown Source) at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(Unknown Source) at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source) at java.base/sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source) at org.springframework.http.client.SimpleBufferingClientHttpRequest.executeInternal(SimpleBufferingClientHttpRequest.java:76) at org.springframework.http.client.AbstractBufferingClientHttpRequest.executeInternal(AbstractBufferingClientHttpRequest.java:48) at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:66) at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:776) ... 9 common frames omitted

am looking for service graph to get generated from my application .Need your help on that .

apart from pet application for any sample application what is the prerequisite for the service graph in grafana for any application

sowndarya123 commented 1 year ago
  --

EMERGENCY! EUREKA MAY BE INCORRECTLY CLAIMING INSTANCES ARE UP WHEN THEY'RE NOT. RENEWALS ARE LESSER THAN THRESHOLD AND HENCE THE INSTANCES ARE NOT BEING EXPIRED JUST TO BE SAFE.

SaschaBrechmannVHV commented 1 year ago

Hey @sowndarya123 , we are NOT YOUR SUPPORT contact. We do not get money from you. So be cool.

Your Logs look wired. Are you exporting to Zipkin ?

sowndarya123 commented 1 year ago

hi Sasha,

Sorry to bother you regarding time.. But am trying this from long time and finally looking out for help. So curious to fix this. I didn't change the code. It had tracing config

tobiangerstein commented 1 year ago

Hi @sowndarya123 as @danipaniii and @SaschaBrechmannVHV already pointed out we need to have more information here :)

sowndarya123 commented 1 year ago

Hi ,

Am using the official demo environment (https://github.com/inspectIT/inspectit-ocelot-demo )

i dont see a specific container for zipkin on my docker desktop but i see jaeger running .

"level":"info","ts":1664277742.1521714,"caller":"healthcheck/handler.go:129","msg":"Health Check state change","status":"ready"} {"level":"info","ts":1664277742.1521912,"caller":"app/server.go:276","msg":"Starting GRPC server","port":16685,"addr":":16685"} {"level":"info","ts":1664277742.1522212,"caller":"app/server.go:257","msg":"Starting HTTP server","port":16686,"addr":":16686"} {"level":"info","ts":1664277743.1522913,"caller":"channelz/logging.go:50","msg":"[core]Subchannel Connectivity change to CONNECTING","system":"grpc","grpc_log":true} {"level":"info","ts":1664277743.1523316,"caller":"channelz/logging.go:50","msg":"[core]Subchannel picks a new address \":16685\" to connect","system":"grpc","grpc_log":true} {"level":"info","ts":1664277743.1524496,"caller":"grpclog/component.go:71","msg":"[core]pickfirstBalancer: UpdateSubConnState: 0xc000621080, {CONNECTING }","system":"grpc","grpc_log":true} {"level":"info","ts":1664277743.152471,"caller":"channelz/logging.go:50","msg":"[core]Channel Connectivity change to CONNECTING","system":"grpc","grpc_log":true} {"level":"info","ts":1664277743.1526525,"caller":"channelz/logging.go:50","msg":"[core]Subchannel Connectivity change to READY","system":"grpc","grpc_log":true} {"level":"info","ts":1664277743.1526895,"caller":"grpclog/component.go:71","msg":"[core]pickfirstBalancer: UpdateSubConnState: 0xc000621080, {READY }","system":"grpc","grpc_log":true} {"level":"info","ts":1664277743.1526976,"caller":"channelz/logging.go:50","msg":"[core]Channel Connectivity change to READY","system":"grpc","grpc_log":true}

sowndarya123 commented 1 year ago

error -logs from API Gateway.

72.22.0.9:8081] The connection observed an error

reactor.netty.http.client.PrematureCloseException: Connection prematurely closed BEFORE response

2022-09-27 13:00:05.402 ERROR [api-gateway,685120f05e6e9dfa,685120f05e6e9dfa] 17 --- [or-http-epoll-4] a.w.r.e.AbstractErrorWebExceptionHandler : [f80eb55e-3] 500 Server Error for HTTP GET "/api/gateway/owners/9"

org.springframework.web.reactive.function.client.WebClientRequestException: Connection prematurely closed BEFORE response; nested exception is reactor.netty.http.client.PrematureCloseException: Connection prematurely closed BEFORE response at org.springframework.web.reactive.function.client.ExchangeFunctions$DefaultExchangeFunction.lambda$wrapException$9(ExchangeFunctions.java:141) Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: Error has been observed at the following site(s): | checkpoint ⇢ Request to GET http://customers-service/owners/9 [DefaultWebClient] | checkpoint ⇢ Handler org.springframework.samples.petclinic.api.boundary.web.ApiGatewayController#getOwnerDetails(int) [DispatcherHandler] | checkpoint ⇢ org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter [DefaultWebFilterChain] | checkpoint ⇢ org.springframework.cloud.sleuth.instrument.web.TraceWebFilter [DefaultWebFilterChain] | checkpoint ⇢ org.springframework.boot.actuate.metrics.web.reactive.server.MetricsWebFilter [DefaultWebFilterChain] | checkpoint ⇢ HTTP GET "/api/gateway/owners/9" [ExceptionHandlingWebHandler] Stack trace: at org.springframework.web.reactive.function.client.ExchangeFunctions$DefaultExchangeFunction.lambda$wrapException$9(ExchangeFunctions.java:141) at reactor.core.publisher.MonoErrorSupplied.subscribe(MonoErrorSupplied.java:70) at reactor.core.publisher.Mono.subscribe(Mono.java:4150) at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:103) at reactor.core.publisher.FluxPeek$PeekSubscriber.onError(FluxPeek.java:221) at reactor.core.publisher.FluxPeek$PeekSubscriber.onError(FluxPeek.java:221) at reactor.core.publisher.FluxPeek$PeekSubscriber.onError(FluxPeek.java:221) at reactor.core.publisher.MonoNext$NextSubscriber.onError(MonoNext.java:93) at reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onError(MonoFlatMapMany.java:204) at reactor.core.publisher.SerializedSubscriber.onError(SerializedSubscriber.java:124) at reactor.core.publisher.FluxRetryWhen$RetryWhenMainSubscriber.whenError(FluxRetryWhen.java:224) at reactor.core.publisher.FluxRetryWhen$RetryWhenOtherSubscriber.onError(FluxRetryWhen.java:273) at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.drain(FluxConcatMap.java:413) at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.onNext(FluxConcatMap.java:250) at reactor.core.publisher.EmitterProcessor.drain(EmitterProcessor.java:491) at reactor.core.publisher.EmitterProcessor.tryEmitNext(EmitterProcessor.java:299) at reactor.core.publisher.SinkManySerialized.tryEmitNext(SinkManySerialized.java:97) at reactor.core.publisher.InternalManySink.emitNext(InternalManySink.java:27) at reactor.core.publisher.FluxRetryWhen$RetryWhenMainSubscriber.onError(FluxRetryWhen.java:189) at reactor.core.publisher.MonoCreate$DefaultMonoSink.error(MonoCreate.java:189) at reactor.netty.http.client.HttpClientConnect$HttpObserver.onUncaughtException(HttpClientConnect.java:385) at reactor.netty.ReactorNetty$CompositeConnectionObserver.onUncaughtException(ReactorNetty.java:647) at reactor.netty.resources.DefaultPooledConnectionProvider$DisposableAcquire.onUncaughtException(DefaultPooledConnectionProvider.java:220) at reactor.netty.resources.DefaultPooledConnectionProvider$PooledConnection.onUncaughtException(DefaultPooledConnectionProvider.java:468) at reactor.netty.http.client.HttpClientOperations.onInboundClose(HttpClientOperations.java:290) at reactor.netty.channel.ChannelOperationsHandler.channelInactive(ChannelOperationsHandler.java:74) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:248) at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:241) at io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:81) at io.netty.handler.codec.http.HttpContentDecoder.channelInactive(HttpContentDecoder.java:235) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:248) at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:241) at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelInactive(CombinedChannelDuplexHandler.java:418) at io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:389) at io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:354) at io.netty.handler.codec.http.HttpClientCodec$Decoder.channelInactive(HttpClientCodec.java:311) at io.netty.channel.CombinedChannelDuplexHandler.channelInactive(CombinedChannelDuplexHandler.java:221) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:248) at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:241) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1405) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:248) at io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:901) at io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:831) at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) Caused by: reactor.netty.http.client.PrematureCloseException: Connection prematurely closed BEFORE response

2022-09-27 13:05:05.248 INFO [api-gateway,,] 17 --- [trap-executor-0] c.n.d.s.r.aws.ConfigClusterResolver : Resolving eureka endpoints via configuration 2022-09-27 13:10:05.249 INFO [api-gateway,,] 17 --- [trap-executor-0] c.n.d.s.r.aws.ConfigClusterResolver : Resolving eureka endpoints via configuration 2022-09-27 13:15:05.250 INFO [api-gateway,,] 17 --- [trap-executor-0] c.n.d.s.r.aws.ConfigClusterResolver : Resolving eureka endpoints via configuration 2022-09-27 13:20:05.251 INFO [api-gateway,,] 17 --- [trap-executor-0] c.n.d.s.r.aws.ConfigClusterResolver : Resolving eureka endpoints via configuration 2022-09-27 13:25:05.251 INFO [api-gateway,,] 17 --- [trap-executor-0] c.n.d.s.r.aws.ConfigClusterResolver : Resolving eureka endpoints via configuration 2022-09-27 13:30:05.252 INFO [api-gateway,,] 17 --- [trap-executor-0] c.n.d.s.r.aws.ConfigClusterResolver : Resolving eureka endpoints via configuration 2022-09-27 13:35:05.253 INFO [api-gateway,,] 17 --- [trap-executor-0] c.n.d.s.r.aws.ConfigClusterResolver : Resolving eureka endpoints via configuration

tobiangerstein commented 1 year ago

Can you share the docker-compose file, which you are using?

sowndarya123 commented 1 year ago

version: '3'

services:

Ocelot

Serves as a collector for traces

agent: image: inspectit/inspectit-ocelot-agent:${INSPECTIT_OCELOT_VERSION} container_name: agent deploy: resources: limits: memory: 128M volumes:

volumes: agent-vol:

heiko-holz commented 1 year ago

Hi @sowndarya123,

can you please edit your previous post and use proper code formatting? It is really hard to read your posts when the error logs / configs are not formatted properly.

Also, can you please attach your docker-compose.yaml as a file?

It seams that none of your services is called tracing-server. I assume you have set Jaeger or Zipkin endpoint to http://tracing-server:1411/api/traces in the inspectit Ocelot configuration.

Either change your configuration or rename the Jaeger service in the docker compose file.

sowndarya123 commented 1 year ago

custom-service-logs.txt visits-service.txt vets-service-logs.txt

These are the log files . I want the service graph to populate for the petapplication.