search

spring-cloud / spring-cloud-dataflow

A microservices-based Streaming and Batch data processing in Cloud Foundry and Kubernetes
https://dataflow.spring.io
Apache License 2.0
1.11k stars 579 forks source link

Register local modules #435

Closed Mwea closed 8 years ago

Mwea commented 8 years ago

I have a jar with parent spring-cloud-dataflow-parent, which I register using my local repository --coordinates. But once I execute module info on it, I receive this error: java.lang.RuntimeException: org.eclipse.aether.resolution.ArtifactResolutionException: Could not find artifact ... in repository 1 (https://repo.spring.io/libs-snapshot) Is there any solution to this ? Like module upload for Spring XD ?

ericbottard commented 8 years ago

Are you 100% sure about the way you registered the module? Coordinates syntax can be a bit tricky. Could you show the exact command you used for registration?

Mwea commented 8 years ago

Actually I'm not sure about the syntax and didn't find any documentation, here is my line

module register --coordinates com.rd:rabbit-source-custom:jar:/home/titouan/Softwares/poc/rabbit-source-custom/target/ --name rabbit-custom --type source

is it right ?

ericbottard commented 8 years ago

Sadly not. Thanks for reporting this issue, we need to update the documentation.

The syntax is 

module register --name rabbit-custom --type source --coordinates <groupId>:<artifactId>[:<extension>[:<classifier>]]:<version>

so 

module register --name rabbit-custom --type source --coordinates com.rd:rabbit-source-custom:jar:exec:1.0.0.BUILD-SNAPSHOT

is likely what you want (your mileage may vary regarding the version). Also, pay attention to the qualifier (exec above), assuming you developed your module the same way out-of-the-box modules are done.

Mwea commented 8 years ago

Thanks ! Even without link to the path of my jar application, spring will locate my file ?

ericbottard commented 8 years ago

You need to have mvn install'd your app first. The app is then fetched from your local maven repository

Mwea commented 8 years ago

Just tried to install then launch it, and got this error : 

45:58.912  INFO 14638 --- [           main] o.s.c.d.a.launcher.ModuleLauncherRunner  : Launching
        com.rd:rabbit-source-custom:jar:exec:1.0.SNAPSHOT with arguments {server.port=59891, spring.cloud.stream.bindings.output.destination=test.rabbit-source-custom, dataflow.group-deployment-id=test-1457106351397, spring.jmx.default-domain=test.rabbit-source-custom}

2016-03-04 16:46:19.922 ERROR 14638 --- [           main] o.s.boot.SpringApplication               : Application startup failed

java.lang.IllegalStateException: Failed to execute CommandLineRunner

rabbit-source-custom is actually just a copy of the rabbit-source module..

Full stack is here : 


  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v1.3.2.RELEASE)

2016-03-04 16:45:53.072  INFO 14638 --- [           main] o.s.c.d.a.l.ModuleLauncherApplication    : Starting ModuleLauncherApplication v1.0.0.BUILD-SNAPSHOT on L83834 with PID 14638 (/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar started by titouan in /tmp/spring-cloud-data-flow-4003264049568105520/test-1457106351397/test.rabbit-source-custom)
2016-03-04 16:45:53.079  INFO 14638 --- [           main] o.s.c.d.a.l.ModuleLauncherApplication    : No active profile set, falling back to default profiles: default
2016-03-04 16:45:53.220  INFO 14638 --- [           main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@17756e62: startup date [Fri Mar 04 16:45:53 CET 2016]; root of context hierarchy
2016-03-04 16:45:55.548  INFO 14638 --- [           main] f.a.AutowiredAnnotationBeanPostProcessor : JSR-330 'javax.inject.Inject' annotation found and supported for autowiring
2016-03-04 16:45:58.542  INFO 14638 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup
2016-03-04 16:45:58.556  INFO 14638 --- [           main] o.s.c.support.DefaultLifecycleProcessor  : Starting beans in phase 0
2016-03-04 16:45:58.912  INFO 14638 --- [           main] o.s.c.d.a.launcher.ModuleLauncherRunner  : Launching
    com.rd:rabbit-source-custom:jar:exec:1.0.SNAPSHOT with arguments {server.port=59891, spring.cloud.stream.bindings.output.destination=test.rabbit-source-custom, dataflow.group-deployment-id=test-1457106351397, spring.jmx.default-domain=test.rabbit-source-custom}

2016-03-04 16:46:19.922 ERROR 14638 --- [           main] o.s.boot.SpringApplication               : Application startup failed

java.lang.IllegalStateException: Failed to execute CommandLineRunner
    at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:809) [spring-boot-1.3.2.RELEASE.jar!/:1.3.2.RELEASE]
    at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:790) [spring-boot-1.3.2.RELEASE.jar!/:1.3.2.RELEASE]
    at org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:777) [spring-boot-1.3.2.RELEASE.jar!/:1.3.2.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:308) [spring-boot-1.3.2.RELEASE.jar!/:1.3.2.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1191) [spring-boot-1.3.2.RELEASE.jar!/:1.3.2.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1180) [spring-boot-1.3.2.RELEASE.jar!/:1.3.2.RELEASE]
    at org.springframework.cloud.dataflow.app.launcher.ModuleLauncherApplication.main(ModuleLauncherApplication.java:45) [spring-cloud-dataflow-app-launcher365761935810078470.jar!/:1.0.0.BUILD-SNAPSHOT]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_66-internal]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_66-internal]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_66-internal]
    at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_66-internal]
    at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:54) [spring-cloud-dataflow-app-launcher365761935810078470.jar!/:1.0.0.BUILD-SNAPSHOT]
    at java.lang.Thread.run(Thread.java:745) [na:1.8.0_66-internal]
Caused by: java.lang.RuntimeException: org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact com.rd:rabbit-source-custom:jar:exec:1.0.SNAPSHOT from/to repository 1 (https://repo.spring.io/libs-snapshot): connect timed out
    at org.springframework.cloud.dataflow.app.resolver.AetherModuleResolver.resolve(AetherModuleResolver.java:257) ~[spring-cloud-dataflow-app-launcher365761935810078470.jar!/:1.0.0.BUILD-SNAPSHOT]
    at org.springframework.cloud.dataflow.app.resolver.AetherModuleResolver.resolve(AetherModuleResolver.java:184) ~[spring-cloud-dataflow-app-launcher365761935810078470.jar!/:1.0.0.BUILD-SNAPSHOT]
    at org.springframework.cloud.dataflow.app.launcher.ModuleLauncher.resolveModule(ModuleLauncher.java:263) ~[spring-cloud-dataflow-app-launcher365761935810078470.jar!/:1.0.0.BUILD-SNAPSHOT]
    at org.springframework.cloud.dataflow.app.launcher.ModuleLauncher.launchModule(ModuleLauncher.java:235) ~[spring-cloud-dataflow-app-launcher365761935810078470.jar!/:1.0.0.BUILD-SNAPSHOT]
    at org.springframework.cloud.dataflow.app.launcher.ModuleLauncher.launchIndividualModules(ModuleLauncher.java:228) ~[spring-cloud-dataflow-app-launcher365761935810078470.jar!/:1.0.0.BUILD-SNAPSHOT]
    at org.springframework.cloud.dataflow.app.launcher.ModuleLauncher.launch(ModuleLauncher.java:100) ~[spring-cloud-dataflow-app-launcher365761935810078470.jar!/:1.0.0.BUILD-SNAPSHOT]
    at org.springframework.cloud.dataflow.app.launcher.ModuleLauncherRunner.run(ModuleLauncherRunner.java:67) ~[spring-cloud-dataflow-app-launcher365761935810078470.jar!/:1.0.0.BUILD-SNAPSHOT]
    at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:806) [spring-boot-1.3.2.RELEASE.jar!/:1.3.2.RELEASE]
    ... 12 common frames omitted
Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact com.rd:rabbit-source-custom:jar:exec:1.0.SNAPSHOT from/to repository 1 (https://repo.spring.io/libs-snapshot): connect timed out
    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:444) ~[aether-impl-1.0.2.v20150114.jar!/:na]
    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:246) ~[aether-impl-1.0.2.v20150114.jar!/:na]
    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifact(DefaultArtifactResolver.java:223) ~[aether-impl-1.0.2.v20150114.jar!/:na]
    at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveArtifact(DefaultRepositorySystem.java:294) ~[aether-impl-1.0.2.v20150114.jar!/:na]
    at org.springframework.cloud.dataflow.app.resolver.AetherModuleResolver.resolve(AetherModuleResolver.java:253) ~[spring-cloud-dataflow-app-launcher365761935810078470.jar!/:1.0.0.BUILD-SNAPSHOT]
    ... 19 common frames omitted
Caused by: org.eclipse.aether.transfer.ArtifactTransferException: Could not transfer artifact com.rd:rabbit-source-custom:jar:exec:1.0.SNAPSHOT from/to repository 1 (https://repo.spring.io/libs-snapshot): connect timed out
    at org.eclipse.aether.connector.basic.ArtifactTransportListener.transferFailed(ArtifactTransportListener.java:43) ~[aether-connector-basic-1.0.2.v20150114.jar!/:na]
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run(BasicRepositoryConnector.java:355) ~[aether-connector-basic-1.0.2.v20150114.jar!/:na]
    at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run(RunnableErrorForwarder.java:67) ~[aether-util-1.0.2.v20150114.jar!/:na]
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector$DirectExecutor.execute(BasicRepositoryConnector.java:581) ~[aether-connector-basic-1.0.2.v20150114.jar!/:na]
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector.get(BasicRepositoryConnector.java:249) ~[aether-connector-basic-1.0.2.v20150114.jar!/:na]
    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.performDownloads(DefaultArtifactResolver.java:520) ~[aether-impl-1.0.2.v20150114.jar!/:na]
    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:421) ~[aether-impl-1.0.2.v20150114.jar!/:na]
    ... 23 common frames omitted
Caused by: java.net.SocketTimeoutException: connect timed out
    at java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:1.8.0_66-internal]
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[na:1.8.0_66-internal]
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~[na:1.8.0_66-internal]
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[na:1.8.0_66-internal]
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[na:1.8.0_66-internal]
    at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_66-internal]
    at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:668) ~[na:1.8.0_66-internal]
    at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:532) ~[httpclient-4.5.1.jar!/:4.5.1]
    at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:409) ~[httpclient-4.5.1.jar!/:4.5.1]
    at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177) ~[httpclient-4.5.1.jar!/:4.5.1]
    at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304) ~[httpclient-4.5.1.jar!/:4.5.1]
    at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611) ~[httpclient-4.5.1.jar!/:4.5.1]
    at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446) ~[httpclient-4.5.1.jar!/:4.5.1]
    at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882) ~[httpclient-4.5.1.jar!/:4.5.1]
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:71) ~[httpclient-4.5.1.jar!/:4.5.1]
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55) ~[httpclient-4.5.1.jar!/:4.5.1]
    at org.apache.http.impl.client.DecompressingHttpClient.execute(DecompressingHttpClient.java:158) ~[httpclient-4.5.1.jar!/:4.5.1]
    at org.eclipse.aether.transport.http.HttpTransporter.execute(HttpTransporter.java:287) ~[aether-transport-http-1.0.2.v20150114.jar!/:na]
    at org.eclipse.aether.transport.http.HttpTransporter.implGet(HttpTransporter.java:243) ~[aether-transport-http-1.0.2.v20150114.jar!/:na]
    at org.eclipse.aether.spi.connector.transport.AbstractTransporter.get(AbstractTransporter.java:59) ~[aether-spi-1.0.2.v20150114.jar!/:na]
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector$GetTaskRunner.runTask(BasicRepositoryConnector.java:447) ~[aether-connector-basic-1.0.2.v20150114.jar!/:na]
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run(BasicRepositoryConnector.java:350) ~[aether-connector-basic-1.0.2.v20150114.jar!/:na]
    ... 28 common frames omitted

2016-03-04 16:46:19.927  INFO 14638 --- [           main] .b.l.ClasspathLoggingApplicationListener : Application failed to start with classpath: [jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/spring-boot-starter-actuator-1.3.2.RELEASE.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/spring-boot-starter-1.3.2.RELEASE.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/spring-boot-1.3.2.RELEASE.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/spring-boot-autoconfigure-1.3.2.RELEASE.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/spring-boot-starter-logging-1.3.2.RELEASE.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/logback-classic-1.1.3.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/logback-core-1.1.3.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/jul-to-slf4j-1.7.13.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/log4j-over-slf4j-1.7.13.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/snakeyaml-1.16.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/spring-boot-actuator-1.3.2.RELEASE.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/jackson-databind-2.6.5.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/jackson-annotations-2.6.5.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/jackson-core-2.6.5.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/spring-context-4.2.4.RELEASE.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/spring-beans-4.2.4.RELEASE.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/spring-expression-4.2.4.RELEASE.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/spring-boot-starter-validation-1.3.2.RELEASE.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/tomcat-embed-el-8.0.30.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/hibernate-validator-5.2.2.Final.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/validation-api-1.1.0.Final.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/jboss-logging-3.3.0.Final.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/classmate-1.1.0.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/spring-boot-configuration-processor-1.3.2.RELEASE.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/json-20140107.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/spring-core-4.2.4.RELEASE.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/spring-boot-loader-1.3.2.RELEASE.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/aether-impl-1.0.2.v20150114.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/aether-api-1.0.2.v20150114.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/aether-spi-1.0.2.v20150114.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/aether-util-1.0.2.v20150114.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/aether-connector-basic-1.0.2.v20150114.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/aether-transport-file-1.0.2.v20150114.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/aether-transport-http-1.0.2.v20150114.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/httpclient-4.5.1.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/httpcore-4.4.4.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/commons-codec-1.9.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/jcl-over-slf4j-1.7.13.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/maven-aether-provider-3.1.0.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/maven-model-3.1.0.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/maven-model-builder-3.1.0.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/plexus-interpolation-1.16.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/maven-repository-metadata-3.1.0.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/org.eclipse.sisu.plexus-0.0.0.M2a.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/cdi-api-1.0.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/jsr250-api-1.0.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/javax.inject-1.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/guava-10.0.1.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/jsr305-1.3.9.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/sisu-guice-3.1.0-no_aop.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/aopalliance-1.0.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/org.eclipse.sisu.inject-0.0.0.M2a.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/asm-3.3.1.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/plexus-classworlds-2.4.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/plexus-component-annotations-1.5.5.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/plexus-utils-3.0.10.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/spring-aop-4.2.4.RELEASE.jar!/, jar:file:/tmp/spring-cloud-dataflow-app-launcher365761935810078470.jar!/lib/slf4j-api-1.7.13.jar!/]
2016-03-04 16:46:19.927  INFO 14638 --- [           main] utoConfigurationReportLoggingInitializer : 

Error starting ApplicationContext. To display the auto-configuration report enable debug logging (start with --debug)

2016-03-04 16:46:19.928  INFO 14638 --- [           main] s.c.a.AnnotationConfigApplicationContext : Closing org.springframework.context.annotation.AnnotationConfigApplicationContext@17756e62: startup date [Fri Mar 04 16:45:53 CET 2016]; root of context hierarchy
2016-03-04 16:46:19.930  INFO 14638 --- [           main] o.s.c.support.DefaultLifecycleProcessor  : Stopping beans in phase 0
2016-03-04 16:46:19.935  INFO 14638 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Unregistering JMX-exposed beans on shutdown
ericbottard commented 8 years ago

Can you provide the output of the following command?

find ~/.m2/repository/com/rd/rabbit-source-custom

This is assuming you're running some *nix, right?

Mwea commented 8 years ago

Yup, I'm on ubuntu 15.04

find /home/titouan/.m2/repository/com/rd/rabbit-source-custom
/home/titouan/.m2/repository/com/rd/rabbit-source-custom
/home/titouan/.m2/repository/com/rd/rabbit-source-custom/maven-metadata-local.xml
/home/titouan/.m2/repository/com/rd/rabbit-source-custom/1.0.SNAPSHOT
/home/titouan/.m2/repository/com/rd/rabbit-source-custom/1.0.SNAPSHOT/resolver-status.properties
/home/titouan/.m2/repository/com/rd/rabbit-source-custom/home
/home/titouan/.m2/repository/com/rd/rabbit-source-custom/home/titouan
/home/titouan/.m2/repository/com/rd/rabbit-source-custom/home/titouan/Softwares
/home/titouan/.m2/repository/com/rd/rabbit-source-custom/home/titouan/Softwares/project-poc
/home/titouan/.m2/repository/com/rd/rabbit-source-custom/home/titouan/Softwares/project-poc/rabbit-source-custom
/home/titouan/.m2/repository/com/rd/rabbit-source-custom/home/titouan/Softwares/project-poc/rabbit-source-custom/target
/home/titouan/.m2/repository/com/rd/rabbit-source-custom/home/titouan/Softwares/project-poc/rabbit-source-custom/target/rabbit-source-custom-
/home/titouan/.m2/repository/com/rd/rabbit-source-custom/home/titouan/Softwares/project-poc/rabbit-source-custom/target/rabbit-source-custom-/home
/home/titouan/.m2/repository/com/rd/rabbit-source-custom/home/titouan/Softwares/project-poc/rabbit-source-custom/target/rabbit-source-custom-/home/titouan
/home/titouan/.m2/repository/com/rd/rabbit-source-custom/home/titouan/Softwares/project-poc/rabbit-source-custom/target/rabbit-source-custom-/home/titouan/Softwares
/home/titouan/.m2/repository/com/rd/rabbit-source-custom/home/titouan/Softwares/project-poc/rabbit-source-custom/target/rabbit-source-custom-/home/titouan/Softwares/project-poc
/home/titouan/.m2/repository/com/rd/rabbit-source-custom/home/titouan/Softwares/project-poc/rabbit-source-custom/target/rabbit-source-custom-/home/titouan/Softwares/project-poc/rabbit-source-custom
/home/titouan/.m2/repository/com/rd/rabbit-source-custom/home/titouan/Softwares/project-poc/rabbit-source-custom/target/rabbit-source-custom-/home/titouan/Softwares/project-poc/rabbit-source-custom/target
/home/titouan/.m2/repository/com/rd/rabbit-source-custom/1.0.0.BUILD-SNAPSHOT
/home/titouan/.m2/repository/com/rd/rabbit-source-custom/1.0.0.BUILD-SNAPSHOT/resolver-status.properties
/home/titouan/.m2/repository/com/rd/rabbit-source-custom/1.0-SNAPSHOT
/home/titouan/.m2/repository/com/rd/rabbit-source-custom/1.0-SNAPSHOT/maven-metadata-local.xml
/home/titouan/.m2/repository/com/rd/rabbit-source-custom/1.0-SNAPSHOT/_maven.repositories
/home/titouan/.m2/repository/com/rd/rabbit-source-custom/1.0-SNAPSHOT/rabbit-source-custom-1.0-SNAPSHOT.jar
/home/titouan/.m2/repository/com/rd/rabbit-source-custom/1.0-SNAPSHOT/rabbit-source-custom-1.0-SNAPSHOT.pom
ericbottard commented 8 years ago

Is the source for this app publicly available? That would help. Your problem is with the exec qualifier. In the out of the box modules we use this as the qualifier (for the boot "uberjar"). Not sure how you built your module. The "simple" jar that is in your repository may, or may not, be what you want. Without seeing your pom, I have no way of knowing.

See our configuration at https://github.com/spring-cloud/spring-cloud-stream-modules/blob/26ba4dcd9d5b5da42fffcff2893f38502bfa8e87/pom.xml#L76-L86

channing-jackson commented 8 years ago

I'm actually experiencing this same issue. Running a snapshot build of spring-dataflow-admin (or server) and the same version of the shell, I've maven install'd my artifact in my local repository. My module register command was as follows: 

register --name custom-jms-1.0.24-SNAPSHOT --type source --coordinates com.blah:custom-jms-source:jar:exec:1.0.24-SNAPSHOT

It was successfully registered, but when I attempt module info, I get this response:

Command failed org.springframework.cloud.dataflow.rest.client.DataFlowClientException: org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact com.blah:custom-jms-source:jar:exec:1.0.24-SNAPSHOT from/to repository 1 (https://repo.spring.io/libs-snapshot): Connection refused

org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact com.blah:custom-jms-source:jar:exec:1.0.24-SNAPSHOT from/to repository 1 (https://repo.spring.io/libs-snapshot): Connection refused

org.springframework.cloud.dataflow.rest.client.DataFlowClientException: org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact com.blah:custom-jms-source:jar:exec:1.0.24-SNAPSHOT from/to repository 1 (https://repo.spring.io/libs-snapshot): Connection refused

    at org.springframework.cloud.dataflow.rest.client.VndErrorResponseErrorHandler.handleError(VndErrorResponseErrorHandler.java:52)
    at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:641)
    at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:597)
    at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:557)
    at org.springframework.web.client.RestTemplate.getForObject(RestTemplate.java:264)
    at org.springframework.cloud.dataflow.rest.client.ModuleTemplate.info(ModuleTemplate.java:82)
    at org.springframework.cloud.dataflow.shell.command.ModuleCommands.info(ModuleCommands.java:84)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:216)
    at org.springframework.shell.core.SimpleExecutionStrategy.invoke(SimpleExecutionStrategy.java:64)
    at org.springframework.shell.core.SimpleExecutionStrategy.execute(SimpleExecutionStrategy.java:57)
    at org.springframework.shell.core.AbstractShell.executeCommand(AbstractShell.java:130)
    at org.springframework.shell.core.JLineShell.promptLoop(JLineShell.java:533)
    at org.springframework.shell.core.JLineShell.run(JLineShell.java:179)
    at java.lang.Thread.run(Unknown Source)

Seeing your latest post on this thread, are you suggesting that my spring-boot-maven-plugin setup needs to set a classifier? Here is what my POM entry looks like: 

<plugin>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-maven-plugin</artifactId>
    <version>1.3.2.RELEASE</version>
    <configuration>
        <layout>MODULE</layout>
    </configuration>
    <executions>
        <execution>
            <goals>
                <goal>repackage</goal>
            </goals>
        </execution>
    </executions>
</plugin>
channing-jackson commented 8 years ago

@Mwea What does the 'parent' section of your POM look like? I can't seem to find documentation on how to set that up, and I'm thinking that using the Spring-XD parent should be replaced: 

    <parent>
        <groupId>org.springframework.xd</groupId>
        <artifactId>spring-xd-module-parent</artifactId>
        <version>1.3.0.RELEASE</version>
    </parent>
ericbottard commented 8 years ago

@channing-jackson you don't need to, but that is what we do for OOTB modules (so that they can be composed later). What you do need however, is to be consistent between how you build your module (your pom does not use a qualifier) and how you register the module (your command above specifies one as exec (4th part of the coordinates). Leave that blank if you don't have a qualifier (you can also omit the jar extension then), see 

--coordinates <groupId>:<artifactId>[:<extension>[:<classifier>]]:<version>

Hope that makes sense.

Mwea commented 8 years ago

My source code isn't available but I can give you my pom xml for rabbit-custom : 

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <artifactId>rabbit-source-custom</artifactId>
    <packaging>jar</packaging>
    <name>rabbit-source-custom</name>
    <description>Rabbit Source stream module</description>
    <parent>
        <artifactId>poc-fraud</artifactId>
        <groupId>com.rd</groupId>
        <version>1.0-SNAPSHOT</version>
    </parent>

    <organization>
        <name>myproject</name>
        <url>http://www.myproject.com</url>
    </organization>

    <properties>
        <start-class>org.springframework.cloud.stream.module.rabbit.source.RabbitSourceApplication</start-class>
    </properties>

    <dependencies>
        <!--<dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-stream-test-support-internal</artifactId>
        </dependency>-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-stream</artifactId>
        </dependency>
        <dependency>
            <groupId>com.rd</groupId>
            <artifactId>aida-domain</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.integration</groupId>
            <artifactId>spring-integration-amqp</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.integration</groupId>
            <artifactId>spring-integration-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

</project>

And here is the pom xml of my parent : 


<?xml version="1.0" encoding="UTF-8"?>
<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xmlns="http://maven.apache.org/POM/4.0.0"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.rd</groupId>
    <artifactId>poc-fraud</artifactId>
    <version>1.0-SNAPSHOT</version>

    <organization>
        <name>myproject</name>
        <url>http://www.myproject.com</url>
    </organization>

    <modules>
        <module>myproject-backoffice</module>
        <module>myproject-batch</module>
        <module>myproject-domain</module>
        <module>myproject-service-discovery-server</module>
        <module>myproject-dataflow-server</module>
        <module>myproject-config-server</module>
        <module>gpfdist-sink-custom</module>
        <module>rabbit-source-custom</module>
    </modules>

    <packaging>pom</packaging>

    <properties>
        <apt-maven-plugin.version>1.1.3</apt-maven-plugin.version>
        <assertj-core.version>1.6.1</assertj-core.version>
        <awaitility.version>1.4.0</awaitility.version>
        <commons-io.version>2.4</commons-io.version>
        <commons-lang.version>2.6</commons-lang.version>
        <gatling.version>2.1.5</gatling.version>
        <gatling-maven-plugin.version>2.1.5</gatling-maven-plugin.version>
        <hibernate.version>4.3.6.Final</hibernate.version>
        <java.version>1.8</java.version>
        <javax.inject.version>1</javax.inject.version>
        <joda-time.version>2.8.2</joda-time.version>
        <joda-time-hibernate.version>1.3</joda-time-hibernate.version>
        <json-path.version>0.9.1</json-path.version>
        <liquibase.version>3.3.2</liquibase.version>
        <liquibase-slf4j.version>1.2.1</liquibase-slf4j.version>
        <liquibase-hibernate4.version>3.5</liquibase-hibernate4.version>
        <maven.build.timestamp.format>yyyyMMddHHmmss</maven.build.timestamp.format>
        <maven.compiler.source>${java.version}</maven.compiler.source>
        <maven.compiler.target>${java.version}</maven.compiler.target>
        <maven-enforcer-plugin.version>1.3.1</maven-enforcer-plugin.version>
        <maven-sortpom-plugin.version>2.3.0</maven-sortpom-plugin.version>
        <maven-war-plugin.version>2.6</maven-war-plugin.version>
        <metrics-spark-reporter.version>1.2</metrics-spark-reporter.version>
        <metrics-spring.version>3.0.4</metrics-spring.version>
        <querydsl.version>3.6.3</querydsl.version>
        <run.addResources>false</run.addResources>
        <sonar-maven-plugin.version>2.3</sonar-maven-plugin.version>
        <spring.version>4.2.5.RELEASE</spring.version>
        <!--<spring-batch.version>3.0.5.RELEASE</spring-batch.version>-->
        <spring-batch.version>3.0.6.RELEASE</spring-batch.version>
        <spring-boot.version>1.3.3.RELEASE</spring-boot.version>
        <spring-boot-starter.version>1.3.3.RELEASE</spring-boot-starter.version>
        <spring-cloud-dataflow.version>1.0.0.BUILD-SNAPSHOT</spring-cloud-dataflow.version>
        <spring-cloud-netflix.version>1.0.4.RELEASE</spring-cloud-netflix.version>
        <spring-context.version>4.2.5.RELEASE</spring-context.version>
        <spring-integration.version>4.2.2.RELEASE</spring-integration.version>
        <spring-integration-amqp.version>${spring-integration.version}</spring-integration-amqp.version>
        <spring-integration-java-dsl.version>1.0.2.RELEASE</spring-integration-java-dsl.version>
        <!--<spring-security.version>4.0.0.RELEASE</spring-security.version>-->
        <!--<spring-security-ldap.version>4.0.0.RELEASE</spring-security-ldap.version>-->
        <!--<spring-security-oauth2.version>2.0.1.RELEASE</spring-security-oauth2.version>-->
        <rabbitmq-client.version>3.5.6</rabbitmq-client.version>
        <swagger-springmvc.version>0.9.5</swagger-springmvc.version>
        <usertype-core.version>3.2.0.GA</usertype-core.version>
        <tomcat.version>7.0.62</tomcat.version>
        <project.version>1.0-SNAPSHOT</project.version>
        <hsqldb.version>2.3.3</hsqldb.version>
    </properties>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>com.mattbertolini</groupId>
                <artifactId>liquibase-slf4j</artifactId>
                <version>${liquibase-slf4j.version}</version>
            </dependency>
            <dependency>
                <groupId>com.mysema.querydsl</groupId>
                <artifactId>querydsl-jpa</artifactId>
                <version>${querydsl.version}</version>
            </dependency>
            <dependency>
                <groupId>com.rabbitmq</groupId>
                <artifactId>amqp-client</artifactId>
                <version>${rabbitmq-client.version}</version>
            </dependency>
            <dependency>
                <groupId>javax.inject</groupId>
                <artifactId>javax.inject</artifactId>
                <version>${javax.inject.version}</version>
            </dependency>
            <dependency>
                <groupId>joda-time</groupId>
                <artifactId>joda-time</artifactId>
                <version>${joda-time.version}</version>
            </dependency>
            <dependency>
                <groupId>joda-time</groupId>
                <artifactId>joda-time-hibernate</artifactId>
                <version>${joda-time-hibernate.version}</version>
            </dependency>
            <dependency>
                <groupId>org.hsqldb</groupId>
                <artifactId>hsqldb</artifactId>
                <version>${hsqldb.version}</version>
            </dependency>
            <dependency>
                <groupId>org.liquibase</groupId>
                <artifactId>liquibase-core</artifactId>
                <version>${liquibase.version}</version>
                <exclusions>
                    <exclusion>
                        <artifactId>jetty-servlet</artifactId>
                        <groupId>org.eclipse.jetty</groupId>
                    </exclusion>
                </exclusions>
            </dependency>
            <dependency>
                <groupId>org.springframework</groupId>
                <artifactId>spring-aop</artifactId>
                <version>${spring.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework</groupId>
                <artifactId>spring-beans</artifactId>
                <version>${spring.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework</groupId>
                <artifactId>spring-context</artifactId>
                <version>${spring-context.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework</groupId>
                <artifactId>spring-context-support</artifactId>
                <version>${spring-context.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework</groupId>
                <artifactId>spring-core</artifactId>
                <version>${spring.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework</groupId>
                <artifactId>spring-expression</artifactId>
                <version>${spring.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework</groupId>
                <artifactId>spring-messaging</artifactId>
                <version>${spring.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework</groupId>
                <artifactId>spring-test</artifactId>
                <version>${spring.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework</groupId>
                <artifactId>spring-tx</artifactId>
                <version>${spring.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework</groupId>
                <artifactId>spring-web</artifactId>
                <version>${spring.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework</groupId>
                <artifactId>spring-webmvc</artifactId>
                <version>${spring.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework</groupId>
                <artifactId>spring-jdbc</artifactId>
                <version>${spring.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot</artifactId>
                <version>${spring-boot.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-actuator</artifactId>
                <version>${spring-boot.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-autoconfigure</artifactId>
                <version>${spring-boot.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-configuration-processor</artifactId>
                <version>${spring-boot.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-loader</artifactId>
                <version>${spring-boot.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter</artifactId>
                <version>${spring-boot-starter.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-actuator</artifactId>
                <version>${spring-boot-starter.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-aop</artifactId>
                <version>${spring-boot-starter.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-freemarker</artifactId>
                <version>${spring-boot-starter.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-jdbc</artifactId>
                <version>${spring-boot-starter.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-logging</artifactId>
                <version>${spring-boot-starter.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-redis</artifactId>
                <version>${spring-boot-starter.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-tomcat</artifactId>
                <version>${spring-boot-starter.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-test</artifactId>
                <version>${spring-boot-starter.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-web</artifactId>
                <version>${spring-boot-starter.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-batch</artifactId>
                <version>${spring-boot-starter.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.batch</groupId>
                <artifactId>spring-batch-core</artifactId>
                <version>${spring-batch.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.batch</groupId>
                <artifactId>spring-batch-infrastructure</artifactId>
                <version>${spring-batch.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-integration</artifactId>
                <version>${spring-boot-starter.version}</version>
                <exclusions>
                    <exclusion>
                        <groupId>org.springframework.integration</groupId>
                        <artifactId>spring-integration-file</artifactId>
                    </exclusion>
                </exclusions>
            </dependency>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-starter-dataflow-server-local</artifactId>
                <version>1.0.0.BUILD-SNAPSHOT</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-stream</artifactId>
                <version>1.0.0.BUILD-SNAPSHOT</version>
            </dependency>
            <!-- dependency to connect to detected cloud services -->
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-spring-service-connector</artifactId>
                <version>1.2.1.RELEASE</version>
            </dependency>
            <!--TODO: Move transport related dependencies here pending https://jira.spring.io/browse/XD-3337  -->
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-stream-binder-redis</artifactId>
                <version>1.0.0.BUILD-SNAPSHOT</version>
            </dependency>
            <dependency>
                <groupId>org.cloudfoundry</groupId>
                <artifactId>cloudfoundry-client-lib</artifactId>
                <version>${cloudfoundry-client-lib.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.cloud.stream.module</groupId>
                <artifactId>spring-cloud-stream-modules-common-configuration</artifactId>
                <version>1.0.0.BUILD-SNAPSHOT</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.cloud.stream.module</groupId>
                <artifactId>spring-cloud-stream-modules-analytics</artifactId>
                <version>1.0.0.BUILD-SNAPSHOT</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.integration</groupId>
                <artifactId>spring-integration-gemfire</artifactId>
                <version>${spring-integration.version}</version>
                <exclusions>
                    <exclusion>
                        <groupId>org.springframework.data</groupId>
                        <artifactId>spring-data-gemfire</artifactId>
                    </exclusion>
                </exclusions>
            </dependency>
            <dependency>
                <groupId>org.springframework.data</groupId>
                <artifactId>spring-data-gemfire</artifactId>
                <version>1.6.2.RELEASE</version>
                <exclusions>
                    <exclusion>
                        <groupId>com.gemstone.gemfire</groupId>
                        <artifactId>gemfire</artifactId>
                    </exclusion>
                </exclusions>
            </dependency>
            <dependency>
                <groupId>com.gemstone.gemfire</groupId>
                <artifactId>gemfire</artifactId>
                <version>8.0.0</version>
                <exclusions>
                    <exclusion>
                        <groupId>net.sourceforge.jline</groupId>
                        <artifactId>jline</artifactId>
                    </exclusion>
                    <exclusion>
                        <groupId>org.fusesource.jansi</groupId>
                        <artifactId>jansi</artifactId>
                    </exclusion>
                </exclusions>
            </dependency>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-stream-test-support</artifactId>
                <version>${spring-cloud-stream.version}</version>
                <scope>test</scope>
            </dependency>
            <dependency>
                <groupId>org.springframework.cloud.stream.module</groupId>
                <artifactId>spring-cloud-stream-modules-test-support</artifactId>
                <version>1.0.0.BUILD-SNAPSHOT</version>
                <scope>test</scope>
            </dependency>
            <dependency>
                <groupId>org.apache.ftpserver</groupId>
                <artifactId>ftpserver-core</artifactId>
                <version>1.0.6</version>
                <scope>test</scope>
            </dependency>
            <dependency>
                <groupId>org.apache.sshd</groupId>
                <artifactId>sshd-core</artifactId>
                <version>0.10.1</version>
                <scope>test</scope>
            </dependency>
            <dependency>
                <groupId>org.springframework.cloud.stream.module</groupId>
                <artifactId>script-variable-generator</artifactId>
                <version>1.0.0.BUILD-SNAPSHOT</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.integration</groupId>
                <artifactId>spring-integration-file</artifactId>
                <version>${spring-integration.version}</version>
            </dependency>

            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-undertow</artifactId>
                <version>${spring-boot-starter.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-netflix</artifactId>
                <version>${spring-cloud-netflix.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-config</artifactId>
                <version>1.0.4.RELEASE</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-config-server</artifactId>
                <version>1.0.4.RELEASE</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-config-monitor</artifactId>
                <version>1.0.4.RELEASE</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-starter-stream-redis</artifactId>
                <version>1.0.4.RELEASE</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.integration</groupId>
                <artifactId>spring-integration-amqp</artifactId>
                <version>${spring-integration-amqp.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.integration</groupId>
                <artifactId>spring-integration-core</artifactId>
                <version>${spring-integration.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.integration</groupId>
                <artifactId>spring-integration-annotation</artifactId>
                <version>${spring-integration.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.integration</groupId>
                <artifactId>spring-integration-java-dsl</artifactId>
                <version>${spring-integration-java-dsl.version}</version>
            </dependency>
            <dependency>
                <groupId>com.rd</groupId>
                <artifactId>myproject-domain</artifactId>
                <version>${project.version}</version>
            </dependency>
        </dependencies>
    </dependencyManagement>

</project>
ericbottard commented 8 years ago

@Mwea I don't even see where you configure the boot plugin. Is your produced artifact a boot uberjar in anyway?

Mwea commented 8 years ago

Where should I configure the boot plugin ? Actually I'm pretty lost..

channing-jackson commented 8 years ago

@ericbottard Is there documentation for building custom modules available yet? I had previously used an options class when building for Spring-XD, and I'm not seeing that same convention available in the current build of cloud dataflow modules.

Also, should we be looking at spring-cloud-stream as a framework with which to develop modules for spring-cloud-dataflow, or is this a separate project, allowing you to build microservices and tie them together using a redis store for connecting channels?

sabbyanandan commented 8 years ago

Hi @Mwea and @channing-jackson: We are working on the developer guide for custom modules via: spring-cloud/spring-cloud-stream#301 and it is targeted to go with the GA release of Spring Cloud Stream project.

While this is in progress, perhaps you could use any of the OOTB applications as reference to build your custom application. They're the same modules from Spring XD, but now refactored to adapt to Spring Cloud Stream programming model.

channing-jackson commented 8 years ago

Thanks @sabbyanandan. I will take a look and see what I can figure out.

channing-jackson commented 8 years ago

@sabbyanandan @ericbottard

One thing that is sort of related...

When I try to get module info for one of the OOTB modules, source:time for example, I get the following error: 

Command failed org.springframework.cloud.dataflow.rest.client.DataFlowClientException: org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact org.springframework.cloud.stream.module:time-source:jar:exec:1.0.0.BUILD-SNAPSHOT from/to repository 1 (https://repo.spring.io/libs-snapshot): Connection refused

org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact org.springframework.cloud.stream.module:time-source:jar:exec:1.0.0.BUILD-SNAPSHOT from/to repository 1 (https://repo.spring.io/libs-snapshot): Connection refused

org.springframework.cloud.dataflow.rest.client.DataFlowClientException: org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact org.springframework.cloud.stream.module:time-source:jar:exec:1.0.0.BUILD-SNAPSHOT from/to repository 1 (https://repo.spring.io/libs-snapshot): Connection refused

    at org.springframework.cloud.dataflow.rest.client.VndErrorResponseErrorHandler.handleError(VndErrorResponseErrorHandler.java:52)
    at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:641)
    at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:597)
    at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:557)
    at org.springframework.web.client.RestTemplate.getForObject(RestTemplate.java:264)
    at org.springframework.cloud.dataflow.rest.client.ModuleTemplate.info(ModuleTemplate.java:82)
    at org.springframework.cloud.dataflow.shell.command.ModuleCommands.info(ModuleCommands.java:84)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:216)
    at org.springframework.shell.core.SimpleExecutionStrategy.invoke(SimpleExecutionStrategy.java:64)
    at org.springframework.shell.core.SimpleExecutionStrategy.execute(SimpleExecutionStrategy.java:57)
    at org.springframework.shell.core.AbstractShell.executeCommand(AbstractShell.java:130)
    at org.springframework.shell.core.JLineShell.promptLoop(JLineShell.java:533)
    at org.springframework.shell.core.JLineShell.run(JLineShell.java:179)
    at java.lang.Thread.run(Unknown Source)
channing-jackson commented 8 years ago

@ericbottard @sabbyanandan Hi there. Returning for more information. I wrote a custom module based on the samples you pointed me to. It compiled and built properly, and I installed it in my local repository.

The following module register command worked fine...

module register --name mqseries-source-1.0.0.BUILD-SNAPSHOT --type source --coordinates com.blah:mqseries-source:jar:exec:1.0.0.BUILD-SNAPSHOT

Then tried to get the module info, and get this error: 

Command failed org.springframework.cloud.dataflow.rest.client.DataFlowClientException: org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact com.blah:mqseries-source:jar:exec:1.0.0.BUILD-SNAPSHOT from/to repository 1 (https://repo.spring.io/libs-snapshot): Connection refused

org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact com.blah:mqseries-source:jar:exec:1.0.0.BUILD-SNAPSHOT from/to repository 1 (https://repo.spring.io/libs-snapshot): Connection refused

org.springframework.cloud.dataflow.rest.client.DataFlowClientException: org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact com.blah:mqseries-source:jar:exec:1.0.0.BUILD-SNAPSHOT from/to repository 1 (https://repo.spring.io/libs-snapshot): Connection refused

    at org.springframework.cloud.dataflow.rest.client.VndErrorResponseErrorHandler.handleError(VndErrorResponseErrorHandler.java:52)
    at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:641)
    at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:597)
    at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:557)
    at org.springframework.web.client.RestTemplate.getForObject(RestTemplate.java:264)
    at org.springframework.cloud.dataflow.rest.client.ModuleTemplate.info(ModuleTemplate.java:82)
    at org.springframework.cloud.dataflow.shell.command.ModuleCommands.info(ModuleCommands.java:84)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:216)
    at org.springframework.shell.core.SimpleExecutionStrategy.invoke(SimpleExecutionStrategy.java:64)
    at org.springframework.shell.core.SimpleExecutionStrategy.execute(SimpleExecutionStrategy.java:57)
    at org.springframework.shell.core.AbstractShell.executeCommand(AbstractShell.java:130)
    at org.springframework.shell.core.JLineShell.promptLoop(JLineShell.java:533)
    at org.springframework.shell.core.JLineShell.run(JLineShell.java:179)
    at java.lang.Thread.run(Unknown Source)
****

According to your statements above, it should have found my artifact in my local repository and unpacked it there, correct? It does not appear to be doing that.

Mwea commented 8 years ago

@sabbyanandan Hi ! Actually I'm trying to use this applications , my rabbit-source-custom is just a copy of yours but apparently there is a problem somewhere if I can't use it..

ericbottard commented 8 years ago

@channing-jackson Almost there, I guess. It should have found the application in your local maven repository, but it doesn't unpack it. Let's make sure it's where it's supposed to be first. Could you please paste the output of the following command: find ~/.m2/repository/com/blah/mqseries-source ?

PS: in your module register command, you may want to use a simpler --name, dropping the version from it for example (but you don't have to, it's entirely up to you. Just understand that that name is what you'll then use in your stream definitions, e.g. stream create foo --definition "mqseries-source-1.0.0.BUILD-SNAPSHOT | log")

Mwea commented 8 years ago

@ericbottard Could you give me more information about this configuration ?

@mwea I don't even see where you configure the boot plugin. Is your produced artifact a boot uberjar in anyway?

ericbottard commented 8 years ago

@Mwea The boot plugin is configured in your pom, either directly or e.g. in your parent pom if you use one. See our configuration at https://github.com/spring-cloud/spring-cloud-stream-modules/blob/26ba4dcd9d5b5da42fffcff2893f38502bfa8e87/pom.xml#L76-L86

If you don't know what spring boot is, or what a boot "uberjar" is, I suggest you quickly read http://docs.spring.io/spring-boot/docs/current-SNAPSHOT/reference/htmlsingle/#getting-started-first-application-executable-jar. This is an important pre-requisite to working with spring cloud dataflow

Mwea commented 8 years ago

@ericbottard Thanks ! Apparently it was coming from my dependencies, actually I face another problem.. Is it possible to specify a config server for Rabbit Source Modules ? I added a file bootstrap.yml to the ressources of the rabbit-source modules

spring:
  application:
    name: rabbit
  cloud:
    config:
      uri: http://localhost:8888/

But apparently my config server does not serve file even if rabbit.yml exist on my directory ( the config server works for my other Spring applications ) , do you have any idea ?

channing-jackson commented 8 years ago

@ericbottard 

$ find ./.m2/repository/com/blah/mqseries-source
./.m2/repository/com/blah/mqseries-source
./.m2/repository/com/blah/mqseries-source/1.0.0.BUILD-SNAPSHOT
./.m2/repository/com/blah/mqseries-source/1.0.0.BUILD-SNAPSHOT/maven-metadata-local.xml
./.m2/repository/com/blah/mqseries-source/1.0.0.BUILD-SNAPSHOT/mqseries-source-1.0.0.BUILD-SNAPSHOT-exec.jar
./.m2/repository/com/blah/mqseries-source/1.0.0.BUILD-SNAPSHOT/mqseries-source-1.0.0.BUILD-SNAPSHOT.jar
./.m2/repository/com/blah/mqseries-source/1.0.0.BUILD-SNAPSHOT/mqseries-source-1.0.0.BUILD-SNAPSHOT.pom
./.m2/repository/com/blah/mqseries-source/1.0.0.BUILD-SNAPSHOT/_remote.repositories
./.m2/repository/com/blah/mqseries-source/maven-metadata-local.xml
sabbyanandan commented 8 years ago

I cloned the existing time source and verified it using the Local SPI implementation. 

2016-03-07 10:45:20.361  INFO 95911 --- [hannel-adapter1] log.sink                                 : Mon 7 Mar 16 10:45:20
2016-03-07 10:45:21.367  INFO 95911 --- [hannel-adapter1] log.sink                                 : Mon 7 Mar 16 10:45:21
2016-03-07 10:45:22.372  INFO 95911 --- [hannel-adapter1] log.sink                                 : Mon 7 Mar 16 10:45:22

The source code is here for your reference.

@channing-jackson: Looking at your error, it is possibly due to the corrupted "application registry" in redis. Perhaps you could flush the cache and retry.

→ redis-cli
127.0.0.1:6379> FLUSHALL
OK
127.0.0.1:6379>
markfisher commented 8 years ago

Or if you only want to delete the collections used by the registry:

$ redis-cli keys spring.cloud.artifact.* | xargs redis-cli del
channing-jackson commented 8 years ago

@ericbottard @sabbyanandan Hi there. I cleared out my entire space by deleting all the apps and services, just to "start fresh" in the hopes that would reset the potential corruption you mentioned above. I then got a new instance of Redis bound to a new instance of scdf in my space, and got them running successfully.

Registered my module using the following command: 

module register --name mqseries-jms-1.0.2 --type source --coordinates com.blah:mqseries-source:jar:exec:1.0.2-SNAPSHOT

Received a message that the module was successfully registered. Then asked for module info using this command: 

module info --name source:mqseries-jms-1.0.2

Got this response: 

Command failed org.springframework.cloud.dataflow.rest.client.DataFlowClientException: org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact com.blah:mqseries-source:jar:exec:1.0.2-SNAPSHOT from/to repository 1 (https://repo.spring.io/libs-snapshot): Connection refused

org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact com.blah:mqseries-source:jar:exec:1.0.2-SNAPSHOT from/to repository 1 (https://repo.spring.io/libs-snapshot): Connection refused

org.springframework.cloud.dataflow.rest.client.DataFlowClientException: org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact com.blah:mqseries-source:jar:exec:1.0.2-SNAPSHOT from/to repository 1 (https://repo.spring.io/libs-snapshot): Connection refused

    at org.springframework.cloud.dataflow.rest.client.VndErrorResponseErrorHandler.handleError(VndErrorResponseErrorHandler.java:52)
    at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:641)
    at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:597)
    at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:557)
    at org.springframework.web.client.RestTemplate.getForObject(RestTemplate.java:264)
    at org.springframework.cloud.dataflow.rest.client.ModuleTemplate.info(ModuleTemplate.java:82)
    at org.springframework.cloud.dataflow.shell.command.ModuleCommands.info(ModuleCommands.java:84)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:216)
    at org.springframework.shell.core.SimpleExecutionStrategy.invoke(SimpleExecutionStrategy.java:64)

Based on my understanding, it should not be attempting to get any information from repo.spring.io at all. Am I missing something?

Mwea commented 8 years ago

I finally got something to work, thanks @ericbottard ! I think more documentation should be available about dependencies and building package or maybe more reference between these documentations.

ericbottard commented 8 years ago

@channing-jackson I could successfully test that with no network access, one can use a module that resides in the local maven repo, even when it's a -SNAPSHOT. So, it seems that for some reason, the deployer thinks that you don't have the needed module locally..

But something intrigues me... What do you mean

new instance of scdf in my space

Are you deploying on CF? Because if you are, then obviously you need to tell the deployer where your custom maven repository is. Sorry I missed this, having 2 issues reported at once in this thread.

A little bit more context: In the current implementation of things, the app gets downloaded by the ModuleLauncher, which in your case executes on CF. I also assume your app is not deployed in a public maven repository. So you need to instruct the admin server of your custom maven repository location, via deployer.remoteRepositories (see this class)

channing-jackson commented 8 years ago

@ericbottard Thanks for the added information. I will look at the links you provided above and see what I can figure out.

In answer to your question, yes, I am deploying to an instance of CF on our internal network.

ericbottard commented 8 years ago

@channing-jackson Can you confirm then, that everytime you gave the output of some find ~/.m2/repository command, it was NOT on the machine the app eventually executes (but rather, I guess, on your own laptop)? If so, then that's your problem.

channing-jackson commented 8 years ago

@ericbottard
Yes sir, I can confirm that :-)

Can I assume that if I use an ENV variable for "deployer.remoteRepositories" the application will read that into the deployer configuration after restaging?

ericbottard commented 8 years ago

Yes sir, I can confirm that :-)

OK, so I think we've finally sorted this out. Sorry for the misunderstanding

Can I assume that if I use an ENV variable for "deployer.remoteRepositories" the application will read that into the deployer configuration after restaging?

Correct. Notice though, that this should be set on the deployer admin (that will automatically propagate it to every app it deploys)

channing-jackson commented 8 years ago

@ericbottard

So close! I'm not clear on where the "deployer admin" is. All I see in my CF space is the dataflow-server app which came from the spring-cloud-dataflow-server 1.0.0.BUILD-SNAPSHOT. Do I need to push the deployer admin as a separate application? If so, is this the spring-cloud-dataflow-deployer-cloudfoundry 1.0.0.BUILD-SNAPSHOT?
ericbottard commented 8 years ago

hmmm, wait you should be using spring-cloud-dataflow-deployer-cloudfoundry indeed, if targeting cloudfoundry.

Else, you'll have an app that spawns processes "locally" (which is really, on CF) but has no failover, elasticity, or anything

ericbottard commented 8 years ago

see http://docs.spring.io/spring-cloud-dataflow-admin-cloudfoundry/docs/current-SNAPSHOT/reference/htmlsingle/#getting-started

channing-jackson commented 8 years ago

Ok, yeah, that doc looks very familiar. I've been following those instructions. I guess I'm not clear on the nomenclature between what you said in your earlier post about "deployer admin."

Clarification: In my earlier post, I should have said my deployment came from spring-cloud-dataflow-server-cloudfoundry. 

spring-cloud-dataflow-server-cloudfoundry-1.0.0.BUILD-20160304.164242-4.jar
sabbyanandan commented 8 years ago

@channing-jackson: We recently moved away from the term admin to server at the core level, so that's part of the confusion.

As @ericbottard pointed out, If you're pushing spring-cloud-dataflow-server-cloudfoundry-1.0.0.BUILD-SNAPSHOT.jar with deployer.remoteRepositories override poitning to your organization's remote maven repository, you should be good.

channing-jackson commented 8 years ago

@sabbyanandan @ericbottard After downloading the latest snapshot build, and reconfiguring using the newly provided MAVEN_REMOTE_REPOSITORIES placeholder environment variable, I have resolved the issues I was having previously with the dataflow server searching repo.spring.io for my custom artifacts.

I've successfully registered a custom artifact and created a simple stream with it, and also created the simple 'ticktock' stream. The only problem with all this was, the MAVEN_REMOTE_REPOSITORIES environment variable did not propagate automatically to the modules when the streams were deployed. I had to go into the web interface of PCF and set the environment variables on each module manually and restart them to get it to stick.

Thanks for the ongoing support!

ericbottard commented 8 years ago

@channing-jackson To help diagnose the issue, could you please give us:

Thanks!

channing-jackson commented 8 years ago

@ericbottard

I'm assuming you mean cf env <module> for a module immediately after I create a stream, and haven't yet done any manual environment settings?

SPRING_APPLICATION_JSON: {"offline":"false","args.0.timeUnit":"MINUTES","args.0.spring.cloud.stream.bindings.output.destination":"ticktock.time","args.0.dataflow.group-deployment-id":"ticktock-1457713012726","args.0.spring.jmx.default-domain":"ticktock.time","remoteRepositories":"https://mavenrepo.blahblah.com/nexus/content/repositories/snapshots/,https://mavenrepo.blahblah.com/nexus/content/repositories/releases/,https://mavenrepo.blahblah.com/nexus/content/repositories/spring-milestones/,https://mavenrepo.blahblah.com/nexus/content/repositories/spring-snapshots/,https://mavenrepo.blahblah.com/nexus/content/repositories/spring-release/","modules":"org.springframework.cloud.stream.module:time-source:jar:exec:1.0.0.BUILD-SNAPSHOT"}

I do see that a "remoteRepositories" value is properly set here, but the values do not appear to be taking. Here is an excerpt from the error at startup: 

2016-03-11T09:18:14.03-0700 [APP/0]      ERR Caused by: org.eclipse.aether.transfer.ArtifactTransferException: Could not transfer artifact org.springframework.cloud.stream.module:time-source:jar:exec:1.0.0.BUILD-SNAPSHOT from/to repository 1 (https://repo.spring.io/libs-snapshot): connect timed out
2016-03-11T09:18:14.03-0700 [APP/0]      ERR    at org.eclipse.aether.connector.basic.ArtifactTransportListener.transferFailed(ArtifactTransportListener.java:43)
2016-03-11T09:18:14.03-0700 [APP/0]      ERR    at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run(BasicRepositoryConnector.java:355)
2016-03-11T09:18:14.03-0700 [APP/0]      ERR    at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run(RunnableErrorForwarder.java:67)
2016-03-11T09:18:14.03-0700 [APP/0]      ERR    at org.eclipse.aether.connector.basic.BasicRepositoryConnector$DirectExecutor.execute(BasicRepositoryConnector.java:581)
2016-03-11T09:18:14.03-0700 [APP/0]      ERR    at org.eclipse.aether.connector.basic.BasicRepositoryConnector.get(BasicRepositoryConnector.java:249)
2016-03-11T09:18:14.03-0700 [APP/0]      ERR    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.performDownloads(DefaultArtifactResolver.java:520)
2016-03-11T09:18:14.03-0700 [APP/0]      ERR    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:421)
2016-03-11T09:18:14.03-0700 [APP/0]      ERR    ... 23 more
2016-03-11T09:18:14.03-0700 [APP/0]      ERR Caused by: java.net.SocketTimeoutException: connect timed out
2016-03-11T09:18:14.03-0700 [APP/0]      ERR    at java.net.PlainSocketImpl.socketConnect(Native Method)

The only application I pushed was spring-cloud-dataflow-server-cloudfoundry

http://repo.spring.io/snapshot/org/springframework/cloud/spring-cloud-dataflow-server-cloudfoundry/1.0.0.BUILD-SNAPSHOT/spring-cloud-dataflow-server-cloudfoundry-1.0.0.BUILD-20160310.002112-6.jar

http://repo.spring.io/snapshot/org/springframework/cloud/spring-cloud-dataflow-server-cloudfoundry/1.0.0.BUILD-SNAPSHOT/spring-cloud-dataflow-server-cloudfoundry-1.0.0.BUILD-20160310.002112-6.jar.sha1

Here is a link to the snapshot version of the shell I used:

http://repo.spring.io/snapshot/org/springframework/cloud/spring-cloud-dataflow-shell/1.0.0.BUILD-SNAPSHOT/spring-cloud-dataflow-shell-1.0.0.BUILD-20160310.162112-15.jar

http://repo.spring.io/snapshot/org/springframework/cloud/spring-cloud-dataflow-shell/1.0.0.BUILD-SNAPSHOT/spring-cloud-dataflow-shell-1.0.0.BUILD-20160310.162112-15.jar.sha1

ericbottard commented 8 years ago

I'm assuming you mean cf env for a module immediately after I create a stream, and haven't yet done any manual environment settings?

correct

sabbyanandan commented 8 years ago

@channing-jackson: For some reason the maven prefix is missing for few keys in your SPRING_APPLICATION_JSON document.

I pulled the latest SNAPSHOT [1.0.0.BUILD-20160311.171914-7.jar] and with that, I'm able to get the env-var's set at the dataflow-server propagated to all the downstream applications automatically.

SPRING_APPLICATION_JSON: {"args.0.spring.cloud.stream.bindings.output.destination":"foooooooobar.time","args.0.dataflow.group-deployment-id":"foooooooobar-1457840433582","args.0.spring.jmx.default-domain":"foooooooobar.time","maven.remoteRepositories":"https://repo.spring.io/libs-snapshot","maven.offline":"false","modules":"org.springframework.cloud.stream.module:time-source:jar:exec:1.0.0.BUILD-SNAPSHOT"}
SPRING_CLOUD_DATAFLOW_MODULE: foooooooobar:time

SPRING_APPLICATION_JSON: {"args.0.spring.cloud.stream.bindings.input.destination":"foooooooobar.time","args.0.spring.cloud.stream.bindings.input.group":"default","args.0.dataflow.group-deployment-id":"foooooooobar-1457840433582","args.0.spring.jmx.default-domain":"foooooooobar.log","maven.remoteRepositories":"https://repo.spring.io/libs-snapshot","maven.offline":"false","modules":"org.springframework.cloud.stream.module:log-sink:jar:exec:1.0.0.BUILD-SNAPSHOT","args.0.spring.cloud.stream.bindings.input.durableSubscription":"true"}
SPRING_CLOUD_DATAFLOW_MODULE: foooooooobar:log

Notice maven.remoteRepositories and maven.offline keys and the maven prefix.

styjdt commented 8 years ago

Thanks to this discussion I was able to register some custom modules using a remote repository specified via environment variable in cloud-foundry dataflow server. Unfortunately the m2 repo where the module artifacts are located is that slow, that the default timeout of 60 seconds is too short to load the artifact from the repo. I tried with environment variable CF_STARTUP_TIMEOUT for dataflow server, but as expected, this env is meant to be used by the cf cli and is not recognized by the dataflow server. (https://docs.cloudfoundry.org/devguide/deploy-apps/large-app-deploy.html)

Is there any way to customize startup time for the modules? Any hidden environment variable for cloudfoundry dataflow server? Would it help to place a manifest.yml in the module jar somewhere? I don't think so, because the app deployed by the dataflow server is not aware of the module in the beginning...

Any hint how to proceed? Alexander

channing-jackson commented 8 years ago

@sabbyanandan Thank you for the update Sabby. I downloaded and installed the snapshot build here: http://repo.spring.io/snapshot/org/springframework/cloud/spring-cloud-dataflow-server-cloudfoundry/1.0.0.BUILD-SNAPSHOT/spring-cloud-dataflow-server-cloudfoundry-1.0.0.BUILD-20160311.171914-7.jar [edited to provide correct link]

And the remoteRepositories environment variable is now maven.remoteRepositories.

As far as I'm concerned, this issue is resolved.

sabbyanandan commented 8 years ago

Thanks, @channing-jackson.

Hi, @styjdt: Thanks for reporting the problem, we are aware of it.

We are currently in the process of adapting dataflow-server-cloudfoundry deployer with 2.0.0.M1 release of cf-java-client. The env-var's overrides in order to deploy large applications needs supported at this deployer level and it will be addressed along with the current refactoring work in progress.

Given this is not directly related to core dataflow-server project, I'll close this and we can track the progress through: spring-cloud/spring-cloud-dataflow-admin-cloudfoundry#70. Please share any other findings in this issue directly.