airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
16.18k stars 4.13k forks source link

[api] `/v1/destinations` throws `java.lang.NullPointerException: null` #37092

Open persunde opened 7 months ago

persunde commented 7 months ago

Platform Version

0.57.2

What step the error happened?

Other

Revelant information

I get a nullpointer error when trying to use the "Airbyte API" with the OSS deployment when calling most API endpoints. This only happens for the "Airbyte API" and not when using the "Configuration API".

Tested and same error on versions: 0.57.2 and 0.55.1

Only tested GET /v1/destinations and GET /v1/workspaces, but this seems to affect more endpoints. I was able to successfully call POST /v1/workspaces (creat a workspace).

Example of Curl calls that fails are:

curl --request GET \
     --url 'http://localhost:8006/v1/destinations?includeDeleted=false&limit=20&offset=0' \
     --header 'accept: application/json'

Response: {"type":"about:blank","status":500}

curl --request GET \
    --url 'http://localhost:8006/v1/workspaces?includeDeleted=false&limit=20&offset=0' \
    --header 'accept: application/json'

Response: {"type":"about:blank","status":500}

Logs are from the OSS hosted "airbyte-api-server".

Relevant log output

2024-04-12 05:21:34 INFO i.a.a.s.n.LoggingNettyChannelHandler(channelRead):38 - [GET] /v1/workspaces?includeDeleted=false&limit=20&offset=0
2024-04-12 05:21:34 ERROR i.m.h.s.RouteExecutor(logException):282 - Unexpected error occurred: null
java.lang.NullPointerException: null
    at io.airbyte.api.server.services.WorkspaceServiceImpl.listWorkspaces(WorkspaceService.kt:320) ~[io.airbyte-airbyte-api-server-0.57.2.jar:?]
    at io.airbyte.api.server.services.WorkspaceServiceImpl.controllerListWorkspaces$lambda$4(WorkspaceService.kt:361) ~[io.airbyte-airbyte-api-server-0.57.2.jar:?]
    at io.airbyte.api.server.apiTracking.TrackingHelper.callWithTracker(TrackingHelper.kt:107) ~[io.airbyte-airbyte-api-server-0.57.2.jar:?]
    at io.airbyte.api.server.services.WorkspaceServiceImpl.controllerListWorkspaces(WorkspaceService.kt:359) ~[io.airbyte-airbyte-api-server-0.57.2.jar:?]
    at io.airbyte.api.server.controllers.WorkspacesController.listWorkspaces(WorkspacesController.kt:69) ~[io.airbyte-airbyte-api-server-0.57.2.jar:?]
    at io.airbyte.api.server.controllers.$WorkspacesController$Definition$Exec.dispatch(Unknown Source) ~[io.airbyte-airbyte-api-server-0.57.2.jar:?]
    at io.micronaut.context.AbstractExecutableMethodsDefinition$DispatchedExecutableMethod.invokeUnsafe(AbstractExecutableMethodsDefinition.java:461) ~[micronaut-inject-4.3.12.jar:4.3.12]
    at io.micronaut.context.DefaultBeanContext$BeanContextUnsafeExecutionHandle.invokeUnsafe(DefaultBeanContext.java:4276) ~[micronaut-inject-4.3.12.jar:4.3.12]
    at io.micronaut.web.router.AbstractRouteMatch.execute(AbstractRouteMatch.java:271) ~[micronaut-router-4.3.12.jar:4.3.12]
    at io.micronaut.http.server.RouteExecutor.executeRouteAndConvertBody(RouteExecutor.java:488) ~[micronaut-http-server-4.3.12.jar:4.3.12]
    at io.micronaut.http.server.RouteExecutor.lambda$callRoute$6(RouteExecutor.java:465) ~[micronaut-http-server-4.3.12.jar:4.3.12]
    at io.micronaut.core.execution.ExecutionFlow.lambda$async$1(ExecutionFlow.java:87) ~[micronaut-core-4.3.12.jar:4.3.12]
    at io.micronaut.core.propagation.PropagatedContext.lambda$wrap$3(PropagatedContext.java:211) ~[micronaut-core-4.3.12.jar:4.3.12]
    at io.micronaut.core.propagation.PropagatedContext.lambda$wrap$3(PropagatedContext.java:211) ~[micronaut-core-4.3.12.jar:4.3.12]
    at java.base/java.util.concurrent.ThreadPerTaskExecutor$TaskRunner.run(ThreadPerTaskExecutor.java:314) ~[?:?]
    at java.base/java.lang.VirtualThread.run(VirtualThread.java:309) ~[?:?]
2024-04-12 05:21:34 INFO i.a.a.s.n.LoggingNettyChannelHandler(capture):88 - Request: [b8216a70-fb14-455d-963e-7d81b8fb5863] -- {"headers":{"host":["localhost"],"x-real-ip":["192.168.80.1"],"x-forwarded-for":["192.168.80.1"],"connection":["close"],"user-agent":["curl\/7.81.0"],"accept":["application\/json"],"x-airbyte-request-id":["b8216a70-fb14-455d-963e-7d81b8fb5863"]},"requestURI":"\/v1\/workspaces?includeDeleted=false&limit=20&offset=0","bodyText":"","queryString":"includeDeleted=false&limit=20&offset=0","method":"GET"}
2024-04-12 05:21:34 INFO i.a.a.s.n.LoggingNettyChannelHandler(capture):89 - Response: [b8216a70-fb14-455d-963e-7d81b8fb5863] -- {"headers":{"content-type":["application\/problem+json"],"date":["Fri, 12 Apr 2024 05:21:34 GMT"],"content-length":["35"]},"status":500,"bodyText":"{\"type\":\"about:blank\",\"status\":500}"}
2024-04-12 05:21:34 INFO i.m.h.s.n.h.a.e.AccessLog(log):125 - airbyte-proxy.airbyte_airbyte_internal - - [12/Apr/2024:05:21:34 +0000] "GET /v1/workspaces?includeDeleted=false&limit=20&offset=0 HTTP/1.0" 500 35

2024-04-12 05:20:28 INFO i.a.a.s.n.LoggingNettyChannelHandler(channelRead):38 - [GET] /v1/destinations?includeDeleted=false&limit=20&offset=0
2024-04-12 05:20:28 ERROR i.m.h.s.RouteExecutor(logException):282 - Unexpected error occurred: null
java.lang.NullPointerException: null
    at io.airbyte.api.server.services.DestinationServiceImpl.listDestinationsForWorkspaces(DestinationService.kt:244) ~[io.airbyte-airbyte-api-server-0.57.2.jar:?]
    at io.airbyte.api.server.controllers.DestinationsController.listDestinations$lambda$4(DestinationsController.kt:167) ~[io.airbyte-airbyte-api-server-0.57.2.jar:?]
    at io.airbyte.api.server.apiTracking.TrackingHelper.callWithTracker(TrackingHelper.kt:107) ~[io.airbyte-airbyte-api-server-0.57.2.jar:?]
    at io.airbyte.api.server.controllers.DestinationsController.listDestinations(DestinationsController.kt:166) ~[io.airbyte-airbyte-api-server-0.57.2.jar:?]
    at io.airbyte.api.server.controllers.$DestinationsController$Definition$Exec.dispatch(Unknown Source) ~[io.airbyte-airbyte-api-server-0.57.2.jar:?]
    at io.micronaut.context.AbstractExecutableMethodsDefinition$DispatchedExecutableMethod.invokeUnsafe(AbstractExecutableMethodsDefinition.java:461) ~[micronaut-inject-4.3.12.jar:4.3.12]
    at io.micronaut.context.DefaultBeanContext$BeanContextUnsafeExecutionHandle.invokeUnsafe(DefaultBeanContext.java:4276) ~[micronaut-inject-4.3.12.jar:4.3.12]
    at io.micronaut.web.router.AbstractRouteMatch.execute(AbstractRouteMatch.java:271) ~[micronaut-router-4.3.12.jar:4.3.12]
    at io.micronaut.http.server.RouteExecutor.executeRouteAndConvertBody(RouteExecutor.java:488) ~[micronaut-http-server-4.3.12.jar:4.3.12]
    at io.micronaut.http.server.RouteExecutor.lambda$callRoute$6(RouteExecutor.java:465) ~[micronaut-http-server-4.3.12.jar:4.3.12]
    at io.micronaut.core.execution.ExecutionFlow.lambda$async$1(ExecutionFlow.java:87) ~[micronaut-core-4.3.12.jar:4.3.12]
    at io.micronaut.core.propagation.PropagatedContext.lambda$wrap$3(PropagatedContext.java:211) ~[micronaut-core-4.3.12.jar:4.3.12]
    at io.micronaut.core.propagation.PropagatedContext.lambda$wrap$3(PropagatedContext.java:211) ~[micronaut-core-4.3.12.jar:4.3.12]
    at java.base/java.util.concurrent.ThreadPerTaskExecutor$TaskRunner.run(ThreadPerTaskExecutor.java:314) ~[?:?]
    at java.base/java.lang.VirtualThread.run(VirtualThread.java:309) ~[?:?]
2024-04-12 05:20:28 INFO i.a.a.s.n.LoggingNettyChannelHandler(capture):88 - Request: [43dfa2d4-d55c-4249-92b7-377a58a69912] -- {"headers":{"host":["localhost"],"x-real-ip":["192.168.80.1"],"x-forwarded-for":["192.168.80.1"],"connection":["close"],"user-agent":["curl\/7.81.0"],"accept":["application\/json"],"x-airbyte-request-id":["43dfa2d4-d55c-4249-92b7-377a58a69912"]},"requestURI":"\/v1\/destinations?includeDeleted=false&limit=20&offset=0","bodyText":"","queryString":"includeDeleted=false&limit=20&offset=0","method":"GET"}
2024-04-12 05:20:28 INFO i.a.a.s.n.LoggingNettyChannelHandler(capture):89 - Response: [43dfa2d4-d55c-4249-92b7-377a58a69912] -- {"headers":{"content-type":["application\/problem+json"],"date":["Fri, 12 Apr 2024 05:20:28 GMT"],"content-length":["35"]},"status":500,"bodyText":"{\"type\":\"about:blank\",\"status\":500}"}
2024-04-12 05:20:28 INFO i.m.h.s.n.h.a.e.AccessLog(log):125 - airbyte-proxy.airbyte_airbyte_internal - - [12/Apr/2024:05:20:28 +0000] "GET /v1/destinations?includeDeleted=false&limit=20&offset=0 HTTP/1.0" 500 35
marcosmarxm commented 7 months ago

Thanks for reporting the issue @persunde I added it to the team responsible for the API.

dermasmid commented 6 months ago

im getting the same on a clean gke helm install

dermasmid commented 6 months ago

solution here: https://github.com/airbytehq/airbyte/discussions/35116#discussioncomment-8569228