search

spring-cloud / spring-cloud-dataflow-samples

Sample starter applications and code for use with the Spring Cloud Data Flow project
http://cloud.spring.io/spring-cloud-dataflow/
220 stars 203 forks source link

Update samples to LAG (latest and greatest) #156

Closed onobc closed 2 years ago

onobc commented 2 years ago

Motivation

The samples are intended to give users a good starting place on how to do particular use cases w/ SCDF. We need to make sure what we are recommending is up-to-speed w/ latest libs and usage of the libs. We should also make sure the samples we recommend are still relevant.

Steps

Step Details

ITEM 1

ITEM 2

ITEM 3

ITEM 4

ITEM 5

onobc commented 2 years ago 
INFO] Reactor Summary for ticktock 1.0.0-SNAPSHOT:
[INFO] 
[INFO] multi-output-time-rabbit ........................... SUCCESS [ 26.027 s]
[INFO] multi-input-log-rabbit ............................. SUCCESS [  4.110 s]
[INFO] multi-output-time-kafka ............................ FAILURE [  1.696 s]
[INFO] multi-input-log-kafka .............................. SKIPPED
[INFO] ticktock ........................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  34.873 s
[INFO] Finished at: 2022-01-16T16:00:32Z
[INFO] ------------------------------------------------------------------------
Error:  Failed to execute goal org.apache.maven.plugins:maven-javadoc-plugin:3.2.0:jar (javadoc) on project multi-output-time-kafka: MavenReportException: Error while generating Javadoc: 
Error:  Exit code: 1 - javadoc: warning - No source files for package org.springframework.cloud.stream.app.time.source.kafka
Error:  javadoc: error - No public or protected classes found to document.
Error:  
Error:  Command line was: /usr/lib/jvm/adoptopenjdk-8-hotspot-amd64/jre/../bin/javadoc @options @packages
Error:  
Error:  Refer to the generated Javadoc files in '/home/runner/work/spring-cloud-dataflow-samples/spring-cloud-dataflow-samples/multi-io-samples/ticktock/multi-output-time-kafka/target/apidocs' dir.
Error:  -> [Help 1]
Error:  
Error:  To see the full stack trace of the errors, re-run Maven with the -e switch.
Error:  Re-run Maven using the -X switch to enable full debug logging.
Error:  
Error:  For more information about the errors and possible solutions, please read the following articles:
Error:  [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
Error:  
Error:  After correcting the problems, you can resume the build with the command
Error:    mvn <args> -rf :multi-output-time-kafka
Error: Process completed with exit code 1.
onobc commented 2 years ago 
[INFO] Downloaded from maven-central: https://repo.maven.apache.org/maven2/org/springframework/spring-beans/5.2.8.RELEASE/spring-beans-5.2.8.RELEASE.jar (689 kB at 3.4 MB/s)
Warning:  Use of META-INF/dataflow-configuration-metadata-whitelist.properties is deprecated. Please use META-INF/dataflow-configuration-metadata.properties
[INFO] Attaching /home/runner/work/spring-cloud-dataflow-samples/spring-cloud-dataflow-samples/multi-io-samples/ticktock/multi-input-log-rabbit/target/multi-input-log-rabbit-1.0.0-SNAPSHOT-metadata.jar to current project
onobc commented 2 years ago 
[INFO] Downloaded: https://repo.maven.apache.org/maven2/org/springframework/session/spring-session-bom/Bean-SR3/spring-session-bom-Bean-SR3.pom (3 KB at 145.0 KB/sec)
Warning:  
Warning:  Some problems were encountered while building the effective model for org.springframework.cloud.dataflow.samples:http-ingest:jar:1.0.0.BUILD-SNAPSHOT
Warning:  'parent.relativePath' of POM org.springframework.cloud.dataflow.samples:http-ingest:1.0.0.BUILD-SNAPSHOT (/home/runner/work/spring-cloud-dataflow-samples/spring-cloud-dataflow-samples/kafka-samples/http-ingest/pom.xml) points at org.springframework.cloud.dataflow.samples:kafka-samples-parent instead of org.springframework.boot:spring-boot-starter-parent, please verify your project structure @ line 5, column 11
Warning:  
Warning:  Some problems were encountered while building the effective model for org.springframework.cloud.dataflow.samples:kstreams-join-user-clicks-and-region:jar:1.0.0.BUILD-SNAPSHOT
Warning:  'parent.relativePath' of POM org.springframework.cloud.dataflow.samples:kstreams-join-user-clicks-and-region:1.0.0.BUILD-SNAPSHOT (/home/runner/work/spring-cloud-dataflow-samples/spring-cloud-dataflow-samples/kafka-samples/kstreams-join-user-clicks-and-region/pom.xml) points at org.springframework.cloud.dataflow.samples:kafka-samples-parent instead of org.springframework.boot:spring-boot-starter-parent, please verify your project structure @ line 6, column 10
Warning:  
Warning:  Some problems were encountered while building the effective model for org.springframework.cloud.dataflow.samples:kstreams-log-user-clicks-per-region:jar:1.0.0.BUILD-SNAPSHOT
Warning:  'parent.relativePath' of POM org.springframework.cloud.dataflow.samples:kstreams-log-user-clicks-per-region:1.0.0.BUILD-SNAPSHOT (/home/runner/work/spring-cloud-dataflow-samples/spring-cloud-dataflow-samples/kafka-samples/kstreams-log-user-clicks-per-region/pom.xml) points at org.springframework.cloud.dataflow.samples:kafka-samples-parent instead of org.springframework.boot:spring-boot-starter-parent, please verify your project structure @ line 5, column 11
Warning:  
Warning:  Some problems were encountered while building the effective model for org.springframework.cloud.dataflow.samples:kstreams-word-count:jar:1.0.0.BUILD-SNAPSHOT
Warning:  'parent.relativePath' of POM org.springframework.cloud.dataflow.samples:kstreams-word-count:1.0.0.BUILD-SNAPSHOT (/home/runner/work/spring-cloud-dataflow-samples/spring-cloud-dataflow-samples/kafka-samples/kstreams-word-count/pom.xml) points at org.springframework.cloud.dataflow.samples:kafka-samples-parent instead of org.springframework.boot:spring-boot-starter-parent, please verify your project structure @ line 6, column 10
Warning:  
Warning:  It is highly recommended to fix these problems because they threaten the stability of your build.
Warning:  
Warning:  For this reason, future Maven versions might no longer support building such malformed projects.
onobc commented 2 years ago

Another issue is the inconsistency w/ the version numbers of the samples. Some use snapshot starting at 0.0.1-snapshot, some w/ snapshot starting at 1.0.0-snapshot and some w/ plain versions such as 1.0.0. We should make this consistent.

onobc commented 2 years ago

Another issue is the multiple ways we use to build docker containers:

  1. jib
  2. maven docker plugin
  3. spring-boot build image
ilayaperumalg commented 2 years ago

Another issue is the inconsistency w/ the version numbers of the samples. Some use snapshot starting at 0.0.1-snapshot, some w/ snapshot starting at 1.0.0-snapshot and some w/ plain versions such as 1.0.0. We should make this consistent.

I agree. These samples were initially moved from different places and versioning was named from there. Given that these samples are already in the artifact repositories, we can have them consistent versioning starting with some higher version numbers, say 2.0.0-SNAPSHOT for the entire samples to avoid confusion to the already existing ones.

ilayaperumalg commented 2 years ago

Another issue is the multiple ways we use to build docker containers:

yeah, we recently discussed about this for the main images for SCDF/Skipper/OOTB apps etc., we can be consistent here as well.

cppwfs commented 2 years ago
onobc commented 2 years ago

I updated the task samples to use spring-boot build image. Unless there is a reason to use the other image builders.

I am in favor of SBBI as well.

I like the use of 2.0.0-SNAPSHOT version for all samples except the task. We may want to set those to 2.0.0 because that is what we will use for our task samples in SCDF. Thoughts?

Make sense to me.

cppwfs commented 2 years ago

Keep:

✅ transition-sample ✅ timestamp-task ✅ timestamp-batch ✅ scenario-task ✅ monitoring-samples ✅ kafka-samples ✅ security-ldap-uaa-example ✅ java-dsl ✅ function-samples ✅ dataflow-website ✅ dataflow-template-example ✅ file-ingest

Discuss

? spring-cloud-stream-function-bindings (refer them to the spring cloud stream apps?) ? dataflow-template-load-generator ? keystore ? multi-io-samples ? keystore ? dataflow-zuul ? custom-dataflow-builds

Discard:

❌ dataflow-migrate-schedules