search

spring-attic / spring-cloud-dataflow-server-mesos

Spring Cloud Data Flow Implementation for Apache Mesos
http://cloud.spring.io/spring-cloud-dataflow-server-mesos/
Apache License 2.0
10 stars 25 forks source link

Cannot Register Non-Docker Apps #98

Closed jejo closed 7 years ago

jejo commented 7 years ago

We've tried and succesfully registered starter apps based from the docs via:

app import --uri http://bit.ly/stream-applications-rabbit-docker

Now we are trying to register some jar apps via http uri but we encounter error from SDCF logs in DC/OS:

Failed to launch container: Failed to run 'docker -H unix:///var/run/docker.sock pull /tmp/deployer-resource-cache4992892517587757949/http-06349fbc8ae348e3e77cd78fadd1b369310a3cbf-our-app-1.0.0.BUILD-SNAPSHOT.jar:latest': exited with status 1; stderr='Error parsing reference: "/tmp/deployer-resource-cache4992892517587757949/http-06349fbc8ae348e3e77cd78fadd1b369310a3cbf-our-app-1.0.0.BUILD-SNAPSHOT.jar:latest" is not a valid repository/tag '

sabbyanandan commented 7 years ago

Hi, @jejo: Thanks for trying Spring Cloud Data Flow! The bit.ly link you're using is outdated - we will update the reference guide shortly. In the meantime, please try out the latest release with the links provided in the project site.

sabbyanandan commented 7 years ago

Also, as an FYI, Spring Cloud Data Flow's Mesos implementation currently only supports docker application artifacts. You will not be able to use Maven application artifacts.

jejo commented 7 years ago

Thanks @sabbyanandan. We'll try docker for our apps. We are currently building up a data processing engine via tasks and streams in SCDF and going to put it in DC/OS. It will be processing around 4+ million records at a time. Is SCDF in DC/OS currently stable to do these critical processes? Thanks!

sabbyanandan commented 7 years ago

The 1.0.GA release of Spring Cloud Data Flow's Mesos implementation is a "generally available" release and is production ready.

As for the 4+ million messages/second, it is not really the Spring Cloud Data Flow that's doing the data processing, but it is the "apps" themselves and the underlying messaging middleware that you intend to use. Our customers/community have generally observed high success with Apache Kafka as the messaging middleware for high throughput requirements - it'd be as simple as registering the kafka-docker apps [http://bit.ly/Avogadro-RC1-stream-applications-kafka-docker] to get started.

For further questions, please use StackOverFlow forum and GitHub issues for bugs/feature requests.