search

Chaffelson / nipyapi

A convenient Python wrapper for Apache NiFi
Other
243 stars 76 forks source link

get_provenance and get_provenance_event api's giving errors #297

Closed FemiKB closed 2 years ago

FemiKB commented 2 years ago

Description

I want to use nipyapi to fetch data provenance from GetFile processor, but when I'm calling get_provenance or get_provenance_event api , It's returning errors

What I Did

I use the following command to call provenance api's

test = ProvenanceApi() thread = test.get_provenance(id='6853aa90-bd63-3bd7-aff8-e3c2639c1471')

I got error message:

reply: 'HTTP/1.1 500 Internal Server Error\r\n' header: Date: Thu, 18 Nov 2021 06:05:41 GMT header: X-Frame-Options: SAMEORIGIN header: Content-Security-Policy: frame-ancestors 'self' header: X-XSS-Protection: 1; mode=block header: X-Content-Type-Options: nosniff header: Strict-Transport-Security: max-age=31540000 header: Vary: Accept-Encoding header: Content-Type: text/plain header: Content-Length: 79 header: Server: Jetty(9.4.42.v20210604) 2021-11-18 11:35:41,685 DEBUG https://localhost:9443 "GET /nifi-api/provenance/6853aa90-bd63-3bd7-aff8-e3c2639c1471 HTTP/1.1" 500 79 2021-11-18 11:35:41,685 DEBUG https://localhost:9443 "GET /nifi-api/provenance/6853aa90-bd63-3bd7-aff8-e3c2639c1471 HTTP/1.1" 500 79 DEBUG:urllib3.connectionpool:https://localhost:9443 "GET /nifi-api/provenance/6853aa90-bd63-3bd7-aff8-e3c2639c1471 HTTP/1.1" 500 79 DEBUG:nipyapi.nifi.rest:response body: An unexpected error has occurred. Please check the logs for additional details. ApiException()

I have no idea about the error. Does anyone know how to resolve the problem? thanks!

Chaffelson commented 2 years ago

A 500 error generally means that you need to look in the NiFi log for an NPE, did you find anything there?

FemiKB commented 2 years ago

It returns java.lang.NullPointerException ( in the nifi user log)

ERROR [NiFi Web Server-115] o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred: java.lang.NullPointerException. Returning Internal Server Error response. java.lang.NullPointerException: null at org.apache.nifi.provenance.index.lucene.LuceneEventIndex.retrieveQuerySubmission(LuceneEventIndex.java:781) at org.apache.nifi.provenance.WriteAheadProvenanceRepository.retrieveQuerySubmission(WriteAheadProvenanceRepository.java:262) at org.apache.nifi.web.controller.ControllerFacade.getProvenanceQuery(ControllerFacade.java:1092) at org.apache.nifi.web.controller.ControllerFacade$$FastClassBySpringCGLIB$$5a42ba54.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:779) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:692) at org.apache.nifi.web.controller.ControllerFacade$$EnhancerBySpringCGLIB$$507094dd.getProvenanceQuery() at org.apache.nifi.web.StandardNiFiServiceFacade.getProvenance(StandardNiFiServiceFacade.java:3183) at org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:779) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750) at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89) at org.apache.nifi.web.NiFiServiceFacadeLock.proceedWithReadLock(NiFiServiceFacadeLock.java:161) at org.apache.nifi.web.NiFiServiceFacadeLock.getLock(NiFiServiceFacadeLock.java:120) at jdk.internal.reflect.GeneratedMethodAccessor104.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:634) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:624) at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:72) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:692) at org.apache.nifi.web.StandardNiFiServiceFacade$$EnhancerBySpringCGLIB$$85012f01.getProvenance() at org.apache.nifi.web.api.ProvenanceResource.getProvenance(ProvenanceResource.java:311) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244)

Chaffelson commented 2 years ago

Right, your usage is probably incorrect. According to the NiFi docs get_provenance returns the results of a Provenance query via an ID, which is previously requested with submit_provenance_request. If you want to work with the low level APIs I suggest you enable developer mode in Chrome, then capture the request sequence in the UI, then duplicate it in NiPyAPI.