search

conductor-oss / conductor

Conductor is an event driven orchestration platform
https://conductor-oss.org
Apache License 2.0
16.01k stars 421 forks source link

Unit Test Error #67

Closed antz-H closed 7 months ago

antz-H commented 7 months ago

I just wanted to document the Unit-Test problems I encountered through this, so that other people might also encounter them.

I run the source code on my window-system , the execute the test case , it reported errors.

source code version : v3.16.0

this is full of error message

Multiple Failures (2 failures)
    org.flywaydb.core.api.FlywayException: Unable to obtain inputstream for resource: db/migration_external_postgres/R__initial_schema.sql
    java.lang.NullPointerException: Cannot invoke "com.netflix.conductor.postgres.storage.PostgresPayloadTestUtil.getDataSource()" because "this.testPostgres" is null
org.gradle.internal.exceptions.DefaultMultiCauseException: Multiple Failures (2 failures)
    org.flywaydb.core.api.FlywayException: Unable to obtain inputstream for resource: db/migration_external_postgres/R__initial_schema.sql
    java.lang.NullPointerException: Cannot invoke "com.netflix.conductor.postgres.storage.PostgresPayloadTestUtil.getDataSource()" because "this.testPostgres" is null
    at app//org.junit.vintage.engine.execution.TestRun.getStoredResultOrSuccessful(TestRun.java:200)
    at app//org.junit.vintage.engine.execution.RunListenerAdapter.fireExecutionFinished(RunListenerAdapter.java:248)
    at app//org.junit.vintage.engine.execution.RunListenerAdapter.testFinished(RunListenerAdapter.java:214)
    at app//org.junit.vintage.engine.execution.RunListenerAdapter.testFinished(RunListenerAdapter.java:88)
    at app//org.junit.runner.notification.SynchronizedRunListener.testFinished(SynchronizedRunListener.java:87)
    at app//org.junit.runner.notification.RunNotifier$9.notifyListener(RunNotifier.java:225)
    at app//org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72)
    at app//org.junit.runner.notification.RunNotifier.fireTestFinished(RunNotifier.java:222)
    at app//org.junit.internal.runners.model.EachTestNotifier.fireTestFinished(EachTestNotifier.java:38)
    at app//org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:372)
    at app//org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:252)
    at app//org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:97)
    at app//org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
    at app//org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
    at app//org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
    at app//org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
    at app//org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
    at app//org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
    at app//org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70)
    at app//org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
    at app//org.junit.runners.ParentRunner.run(ParentRunner.java:413)
    at app//org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:191)
    at app//org.junit.runner.JUnitCore.run(JUnitCore.java:137)
    at app//org.junit.runner.JUnitCore.run(JUnitCore.java:115)
    at app//org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42)
    at app//org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80)
    at app//org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72)
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:107)
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
    at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
    at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
    at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
    at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:53)
    at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:99)
    at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:79)
    at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:75)
    at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:61)
    at java.base@17.0.10/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base@17.0.10/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base@17.0.10/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base@17.0.10/java.lang.reflect.Method.invoke(Method.java:568)
    at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
    at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
    at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
    at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
    at jdk.proxy1/jdk.proxy1.$Proxy2.stop(Unknown Source)
    at org.gradle.api.internal.tasks.testing.worker.TestWorker$3.run(TestWorker.java:193)
    at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
    at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
    at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
    at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
    at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:133)
    at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:71)
    at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
    at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)
    Suppressed: org.flywaydb.core.api.FlywayException: Unable to obtain inputstream for resource: db/migration_external_postgres/R__initial_schema.sql
        at app//org.flywaydb.core.internal.resource.classpath.ClassPathResource.read(ClassPathResource.java:118)
        at app//org.flywaydb.core.internal.resolver.sql.SqlMigrationResolver$1.read(SqlMigrationResolver.java:85)
        at app//org.flywaydb.core.internal.resolver.ChecksumCalculator.calculateChecksumForResource(ChecksumCalculator.java:64)
        at app//org.flywaydb.core.internal.resolver.ChecksumCalculator.calculate(ChecksumCalculator.java:43)
        at app//org.flywaydb.core.internal.resolver.sql.SqlMigrationResolver.getChecksumForLoadableResource(SqlMigrationResolver.java:110)
        at app//org.flywaydb.core.internal.resolver.sql.SqlMigrationResolver.addMigrations(SqlMigrationResolver.java:150)
        at app//org.flywaydb.core.internal.resolver.sql.SqlMigrationResolver.resolveMigrations(SqlMigrationResolver.java:72)
        at app//org.flywaydb.core.internal.resolver.sql.SqlMigrationResolver.resolveMigrations(SqlMigrationResolver.java:49)
        at app//org.flywaydb.core.internal.resolver.CompositeMigrationResolver.lambda$collectMigrations$0(CompositeMigrationResolver.java:127)
        at java.base@17.0.10/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:273)
        at java.base@17.0.10/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
        at java.base@17.0.10/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
        at java.base@17.0.10/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
        at java.base@17.0.10/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
        at java.base@17.0.10/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base@17.0.10/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
        at app//org.flywaydb.core.internal.resolver.CompositeMigrationResolver.collectMigrations(CompositeMigrationResolver.java:128)
        at app//org.flywaydb.core.internal.resolver.CompositeMigrationResolver.doFindAvailableMigrations(CompositeMigrationResolver.java:117)
        at app//org.flywaydb.core.internal.resolver.CompositeMigrationResolver.resolveMigrations(CompositeMigrationResolver.java:107)
        at app//org.flywaydb.core.internal.resolver.CompositeMigrationResolver.resolveMigrations(CompositeMigrationResolver.java:113)
        at app//org.flywaydb.core.internal.command.DbValidate.validate(DbValidate.java:86)
        at app//org.flywaydb.core.Flyway.doValidate(Flyway.java:402)
        at app//org.flywaydb.core.Flyway.lambda$migrate$0(Flyway.java:144)
        at app//org.flywaydb.core.FlywayExecutor.execute(FlywayExecutor.java:196)
        at app//org.flywaydb.core.Flyway.migrate(Flyway.java:140)
        at app//com.netflix.conductor.postgres.storage.PostgresPayloadTestUtil.flywayMigrate(PostgresPayloadTestUtil.java:64)
        at app//com.netflix.conductor.postgres.storage.PostgresPayloadTestUtil.<init>(PostgresPayloadTestUtil.java:40)
        at app//com.netflix.conductor.postgres.storage.PostgresPayloadStorageTest.setup(PostgresPayloadStorageTest.java:73)
        at java.base@17.0.10/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base@17.0.10/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base@17.0.10/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base@17.0.10/java.lang.reflect.Method.invoke(Method.java:568)
        at app//org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
        at app//org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        at app//org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
        at app//org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33)
        at app//org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
        at app//org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:75)
        at app//org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
        at app//org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:86)
        at app//org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:84)
        at app//org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
        ... 48 more
    Suppressed: java.lang.NullPointerException: Cannot invoke "com.netflix.conductor.postgres.storage.PostgresPayloadTestUtil.getDataSource()" because "this.testPostgres" is null
        at com.netflix.conductor.postgres.storage.PostgresPayloadStorageTest.teardown(PostgresPayloadStorageTest.java:298)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
        at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46)
        at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
        at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:86)
        at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:84)
        at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
        ... 48 more

image

antz-H commented 7 months ago

The second test case problem

15:47:16.351 [docker-java-stream--333906299] ERROR com.github.dockerjava.api.async.ResultCallbackTemplate - Error during callback
com.github.dockerjava.api.exception.InternalServerErrorException: Status 500: {"message":"Head \"https://registry-1.docker.io/v2/testcontainers/ryuk/manifests/0.3.1\": unauthorized: incorrect username or password"}

    at org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder.execute(DefaultInvocationBuilder.java:247) ~[testcontainers-1.15.3.jar:?]
    at org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder.lambda$executeAndStream$1(DefaultInvocationBuilder.java:269) ~[testcontainers-1.15.3.jar:?]
    at java.lang.Thread.run(Thread.java:840) [?:?]

Status 500: {"message":"Head \"https://registry-1.docker.io/v2/testcontainers/ryuk/manifests/0.3.1\": unauthorized: incorrect username or password"}

com.github.dockerjava.api.exception.InternalServerErrorException: Status 500: {"message":"Head \"https://registry-1.docker.io/v2/testcontainers/ryuk/manifests/0.3.1\": unauthorized: incorrect username or password"}

    at app//org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder.execute(DefaultInvocationBuilder.java:247)
    at app//org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder.lambda$executeAndStream$1(DefaultInvocationBuilder.java:269)
    at java.base@17.0.10/java.lang.Thread.run(Thread.java:840)

image

antz-H commented 7 months ago

I review the source code（3.16.0） of condutor for some time. And this for beginner' guide.

The first mistake above is because of the Flyway's version issue. The second mistake above is beacause of the docker-hub environment issue when you want to run test .

antz-H commented 7 months ago

Others.

The conductor support different deployment architecture.

  1. postgresql
  2. redis + postgresql (this is default and docker-image-3.15.0 only supports this )
  3. redis + mysql + es

If you want to change (such as use the es), you should build from source code .

So review the source code is more important , because the official docs are too few.

First shoud to review the code , i think is the module-- docker， there you can find many what you want, such as how to build , how to start(run) .

Second if you want to change the deployment architecture , you should build your own image.

Welcome to correct my mistakes.

antz-H commented 7 months ago

If you want to use the conductoross/conductor:3.15.0 in prod , you can run with below command line like this

docker run -v C://home//tmp//conductor//config-redis_postgres.properties:/app/config/config.properties --init -p 8080:8080 -p 1234:5000   conductoross/conductor:3.15.0
antz-H commented 7 months ago

congfig.properties

# Database persistence type.
# Below are the properties for redis
conductor.db.type=redis_standalone
conductor.queue.type=redis_standalone

conductor.redis.hosts=127.0.0.1:6379:us-east-1c
conductor.redis-lock.serverAddress=redis://127.0.0.1:6379
conductor.redis.taskDefCacheRefreshInterval=1
conductor.redis.workflowNamespacePrefix=conductor
conductor.redis.queueNamespacePrefix=conductor_queues

spring.datasource.url=jdbc:postgresql://127.0.0.1:5432/conductor
spring.datasource.username=postgres
spring.datasource.password=root

# Additional modules for metrics collection exposed to Prometheus (optional)
conductor.metrics-prometheus.enabled=true
management.endpoints.web.exposure.include=prometheus

# Load sample kitchen sink workflow
loadSample=true