search

GoogleCloudPlatform / DataflowJavaSDK

Google Cloud Dataflow provides a simple, powerful model for building both batch and streaming parallel data processing pipelines.
http://cloud.google.com/dataflow
855 stars 325 forks source link

Dataflow SpringBoot app fails to run when bundled as a fat uber jar #649

Closed raj-saxena closed 4 years ago

raj-saxena commented 4 years ago

I have been working on a service that programmatically builds Dataflow pipeline using SpringBoot. My service runs perfectly when the application is started from the (IntelliJ) IDE or when I run with ./gradlew run but fails when I create the fat-uber-jar.

The issue seems to be related to https://issues.apache.org/jira/browse/BEAM-1325 and another issue https://github.com/GoogleCloudPlatform/DataflowJavaSDK/issues/538 (although that is about an IDE and this is more broadly about development and deployment).

To verify, I opened the built jar and I see the following entries in the MANIFEST.MF file:

Manifest-Version: 1.0
Main-Class: org.springframework.boot.loader.JarLauncher
Start-Class: com.example.dataExtractor.ApplicationKt
Spring-Boot-Version: 2.2.5.RELEASE
Spring-Boot-Classes: BOOT-INF/classes/
Spring-Boot-Lib: BOOT-INF/lib/

This seems to be a pretty standard way of building SpringBoot apps and would appreciate some guidance in this direction. Meanwhile, I have unblocked myself with a workaround which has its own drawbacks. I wrote an article with more context if it helps.

ajamato commented 4 years ago

Hi, you may wish to ask this question on the Apache Beam github repro, which is now used as the SDK for Dataflow. This repo is no longer being updated and users are advised to use the Apache Beam SDK instead.

https://github.com/apache/beam

raj-saxena commented 4 years ago

Thanks. For someone looking for this in the future, I have created an issue here