Switch to docker-compose

[sebplorenz]

The first commit in this PR adds docker-compose, the second removes the source directories of the components.

The updates of this PR are a major change and influence the other PRs. We need merge this PR before the others and then updated the other PRs. Or the other way around.

resolves #57

[sebplorenz]

@Arian-IDSA @aitorce @SebastianOpriel please check.

[jfernandezsqs]

Hi @sebplorenz! Thanks for the pull requests, I have tried it in my local machine but it is giving some errors.

sqs@sqs-VirtualBox:~/IDS-testbed-feature-57_using_docker-compose$ docker-compose up
ERROR: The Compose file './docker-compose.yml' is invalid because:
Unsupported config option for volumes: 'broker-fuseki'
Unsupported config option for networks: 'local'
Unsupported config option for services: 'omejdn-ui'

I have reviewed the changes in this pull request and I have found the following: Inside the folder DataspaceConnectorA/conf there are two files (config.json and connectora-daps.p12). In the config.json file line 60 it is pointing to

"ids:keyStore" : {
    "@id" : "file:///conf/connector-daps.p12"
  }

This file does not exist. Also, the file connectora.p12 is not included inside DataspaceConnectorA/conf folder.

Inside the folder DataspaceConnectorB/conf there is in line 60 of the config.json file

"ids:keyStore" : {
    "@id" : "file:///conf/connector-daps.p12"
  }

This file does not exist. It is connectorb-daps.p12

I guess that in your environment this configuration has worked, can you please tell me the reason why it appears these errors at executing docker-compose up

[jfernandezsqs]

I have included version: '3.5' at the beginning of the docker-compose.yml and it is giving these error.

ERROR: no such image: ghcr.io/fraunhofer-aisec/omejdn-server:"1.6.0": invalid reference format

[sebplorenz]

@jfernandezsqs I tried in a fresh environment and its working. What version of docker compose are you using? I tried with docker-compose and docker compose. Both works. Following versions are installed on my machine:

s@ubuntu:~/IdeaProjects/IDS-testbed_sebplorenz$ docker compose version
Docker Compose version v2.5.0
s@ubuntu:~/IdeaProjects/IDS-testbed_sebplorenz$ docker-compose --version
docker-compose version 1.27.4, build unknown
s@ubuntu:~/IdeaProjects/IDS-testbed_sebplorenz$ docker --version
Docker version 20.10.14, build a224086

[jfernandezsqs]

I have this versions installed.

sqs@sqs-VirtualBox:~/Descargas/IDS-testbed-feature-57_using_docker-compose$ docker --version
Docker version 20.10.12, build 20.10.12-0ubuntu2~20.04.1
sqs@sqs-VirtualBox:~/Descargas/IDS-testbed-feature-57_using_docker-compose$ docker-compose --version
docker-compose version 1.25.0, build unknown

[jfernandezsqs]

I changed to docker-compose version 1.27.4 and now it is working. However, I tried to register Connector A at MDB and it is giving an internal recipient error. These are the terminal logs.

connectora_1           | 2022-05-17T12:39:59,331 [https-jsse-nio-8080-exec-1] INFO - Cached DAPS DAT expired or no expiration set. [expiration=(null)]
connectora_1           | 2022-05-17T12:39:59,331 [https-jsse-nio-8080-exec-1] WARN - TEST_DEPLOYMENT: IDS-Message is sent without a valid DAT, will not be sent in PRODUCTIVE_DEPLOYMENT. [code=(IMSCOW0041), reason=(Mandatory required information of the connector certificate is missing (AKI/SKI)! Needed to determine the fingerprint of the connector for the DAPS! Possible reason: Are you using a valid connector certificate issued by the DAPS?)]
connectora_1           | 2022-05-17T12:39:59,405 [https-jsse-nio-8080-exec-1] INFO - Successfully loaded Keystore.
connectora_1           | 2022-05-17T12:39:59,459 [https-jsse-nio-8080-exec-1] INFO - Successfully loaded Truststore.
connectora_1           | 2022-05-17T12:39:59,497 [https-jsse-nio-8080-exec-1] WARN - Trustmanager is trusting all Certificates in TEST_DEPLOYMENT mode, you should not use this in production! [code=(IMSCOW0032)]
connectora_1           | 2022-05-17T12:39:59,498 [https-jsse-nio-8080-exec-1] INFO - Cached DAPS DAT expired or no expiration set. [expiration=(null)]
connectora_1           | 2022-05-17T12:39:59,498 [https-jsse-nio-8080-exec-1] WARN - TEST_DEPLOYMENT: IDS-Message is sent without a valid DAT, will not be sent in PRODUCTIVE_DEPLOYMENT. [code=(IMSCOW0041), reason=(Mandatory required information of the connector certificate is missing (AKI/SKI)! Needed to determine the fingerprint of the connector for the DAPS! Possible reason: Are you using a valid connector certificate issued by the DAPS?)]
connectora_1           | 2022-05-17T12:39:59,516 [https-jsse-nio-8080-exec-1] INFO - Sending request to https://broker-reverseproxy/infrastructure ...
broker-core_1          | 12:39:59.605 [http-nio-8080-exec-10] INFO  de.fraunhofer.iais.eis.ids.component.ecosystemintegration.daps.DapsSecurityTokenProvider - Token needs to be fetched anew from DAPS
omejdn-server_1        | "Unable to load key ``keys/clients/Mjg6NkE6MzI6MDY6QjM6MUU6NEU6MDc6NDI6QUI6RjM6OUQ6QTE6ODU6MUQ6Q0U6NEY6QUM6RTM6NUM6a2V5aWQ6Mjg6NkE6MzI6MDY6QjM6MUU6NEU6MDc6NDI6QUI6RjM6OUQ6QTE6ODU6MUQ6Q0U6NEY6QUM6RTM6NUM=.cert'': No such file or directory @ rb_sysopen - keys/clients/Mjg6NkE6MzI6MDY6QjM6MUU6NEU6MDc6NDI6QUI6RjM6OUQ6QTE6ODU6MUQ6Q0U6NEY6QUM6RTM6NUM6a2V5aWQ6Mjg6NkE6MzI6MDY6QjM6MUU6NEU6MDc6NDI6QUI6RjM6OUQ6QTE6ODU6MUQ6Q0U6NEY6QUM6RTM6NUM=.cert"
omejdn-server_1        | Error decoding JWT eyJhbGciOiJSUzI1NiJ9.eyJpc3MiOiIyODo2QTozMjowNjpCMzoxRTo0RTowNzo0MjpBQjpGMzo5RDpBMTo4NToxRDpDRTo0RjpBQzpFMzo1QzprZXlpZDoyODo2QTozMjowNjpCMzoxRTo0RTowNzo0MjpBQjpGMzo5RDpBMTo4NToxRDpDRTo0RjpBQzpFMzo1QyIsInN1YiI6IjI4OjZBOjMyOjA2OkIzOjFFOjRFOjA3OjQyOkFCOkYzOjlEOkExOjg1OjFEOkNFOjRGOkFDOkUzOjVDOmtleWlkOjI4OjZBOjMyOjA2OkIzOjFFOjRFOjA3OjQyOkFCOkYzOjlEOkExOjg1OjFEOkNFOjRGOkFDOkUzOjVDIiwiQGNvbnRleHQiOiJodHRwczovL3czaWQub3JnL2lkc2EvY29udGV4dHMvY29udGV4dC5qc29ubGQiLCJAdHlwZSI6ImlkczpEYXRSZXF1ZXN0VG9rZW4iLCJleHAiOjE2NTI4Nzc1OTksImlhdCI6MTY1Mjc5MTE5OSwiYXVkIjoiaWRzYzpJRFNfQ09OTkVDVE9SU19BTEwiLCJuYmYiOjE2NTI3OTExOTl9.Gc5Mr1cfzuc84cQ5lm9hUTMh6FyfcaHwDb0wIPa0PMTVtaPVxXBMUDnYMgVQ5cNazy1IZIY1YaCQB5OUXWpRUskucTukiPvn-WeovyMFhJnZSsCjclH86bfVGAHVY26ytORlYbLxDhP6Km_PMPCOvk5YjrZcOE423JslhWeypF7N3H-PsL2JfVnszSy3e6kpTcK8CbtcfAseaD8X1jlruhJs2YSTYuW6Vv_6Dua2RKK2T2v4bB7tfSCTThc6qRFsCiWAz52HIymU9q4JqOngKrX4DHaMygMZ6Spzj69yd8FE9Q0H6rgB9fb7LgIsnw9FdYDUiio-zawIEtG0aXMd_g: No verification key available
omejdn-server_1        | 172.22.0.2 - - [17/May/2022:12:39:59 +0000] "POST /token HTTP/1.1" 400 98 0.0023
omejdn_1               | 172.22.0.3 - - [17/May/2022:12:39:59 +0000] "POST /auth/token HTTP/1.1" 400 98 "-" "okhttp/3.12.1" "-"
broker-core_1          | de.fraunhofer.iais.eis.ids.component.core.TokenRetrievalException: Unable to retrieve DAPS token.
broker-core_1          |    at de.fraunhofer.iais.eis.ids.component.ecosystemintegration.daps.DapsSecurityTokenProvider.getSecurityToken(DapsSecurityTokenProvider.java:96)
broker-core_1          |    at de.fraunhofer.iais.eis.ids.component.core.SecurityTokenProvider.getSecurityTokenAsDAT(SecurityTokenProvider.java:12)
broker-core_1          |    at de.fraunhofer.iais.eis.ids.component.core.MessageDispatcher.handle(MessageDispatcher.java:69)
broker-core_1          |    at de.fraunhofer.iais.eis.ids.component.core.MessageDispatcher.lambda$dispatch$2(MessageDispatcher.java:51)
broker-core_1          |    at java.base/java.util.Optional.map(Optional.java:265)
broker-core_1          |    at de.fraunhofer.iais.eis.ids.component.core.MessageDispatcher.dispatch(MessageDispatcher.java:51)
broker-core_1          |    at de.fraunhofer.iais.eis.ids.component.core.DefaultComponent.process(DefaultComponent.java:45)
broker-core_1          |    at de.fraunhofer.iais.eis.ids.component.interaction.multipart.MultipartComponentInteractor.process(MultipartComponentInteractor.java:87)
broker-core_1          |    at de.fraunhofer.iais.eis.ids.component.protocol.http.server.ComponentController.processMessage(ComponentController.java:237)
broker-core_1          |    at de.fraunhofer.iais.eis.ids.component.protocol.http.server.ComponentController.infrastructureLevelCommunication(ComponentController.java:205)
broker-core_1          |    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
broker-core_1          |    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
broker-core_1          |    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
broker-core_1          |    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
broker-core_1          |    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:189)
broker-core_1          |    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
broker-core_1          |    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:102)
broker-core_1          |    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895)
broker-core_1          |    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:800)
broker-core_1          |    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
broker-core_1          |    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1038)
broker-core_1          |    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:942)
broker-core_1          |    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1005)
broker-core_1          |    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:908)
broker-core_1          |    at javax.servlet.http.HttpServlet.service(HttpServlet.java:665)
broker-core_1          |    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:882)
broker-core_1          |    at javax.servlet.http.HttpServlet.service(HttpServlet.java:750)
broker-core_1          |    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
broker-core_1          |    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
broker-core_1          |    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
broker-core_1          |    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
broker-core_1          |    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
broker-core_1          |    at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
broker-core_1          |    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
broker-core_1          |    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
broker-core_1          |    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
broker-core_1          |    at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:92)
broker-core_1          |    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
broker-core_1          |    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
broker-core_1          |    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
broker-core_1          |    at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:93)
broker-core_1          |    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
broker-core_1          |    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
broker-core_1          |    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
broker-core_1          |    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200)
broker-core_1          |    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
broker-core_1          |    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
broker-core_1          |    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
broker-core_1          |    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
broker-core_1          |    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
broker-core_1          |    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
broker-core_1          |    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
broker-core_1          |    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
broker-core_1          |    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
broker-core_1          |    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
broker-core_1          |    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:373)
broker-core_1          |    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
broker-core_1          |    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
broker-core_1          |    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1589)
broker-core_1          |    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
broker-core_1          |    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
broker-core_1          |    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
broker-core_1          |    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
broker-core_1          |    at java.base/java.lang.Thread.run(Thread.java:829)
broker-core_1          | Caused by: java.io.IOException: Unable to retrieve DAPS token (response code: 400) 
broker-core_1          |  Response: {"error":"invalid_client","error_description":"Error decoding JWT: No verification key available"}
broker-core_1          |    at de.fraunhofer.iais.eis.ids.component.ecosystemintegration.daps.DapsTokenRetriever.acquireAccessTokenFromDaps(DapsTokenRetriever.java:190)
broker-core_1          |    at de.fraunhofer.iais.eis.ids.component.ecosystemintegration.daps.DapsTokenRetriever.retrieveToken(DapsTokenRetriever.java:101)
broker-core_1          |    at de.fraunhofer.iais.eis.ids.component.ecosystemintegration.daps.DapsSecurityTokenProvider.getSecurityToken(DapsSecurityTokenProvider.java:94)
broker-core_1          |    ... 63 more
broker-reverseproxy_1  | 172.22.0.7 - - [17/May/2022:12:39:59 +0000] "POST /infrastructure HTTP/1.1" 200 1460 "-" "okhttp/4.9.3"
connectora_1           | 2022-05-17T12:39:59,639 [https-jsse-nio-8080-exec-1] INFO - Successfully received response to request.
connectora_1           | 2022-05-17T12:39:59,645 [https-jsse-nio-8080-exec-1] INFO - Successfully passed SHACL-Validation.
connectora_1           | 2022-05-17T12:39:59,681 [https-jsse-nio-8080-exec-1] DEBUG - Received unexpected response message. [response=({reason=https://w3id.org/idsa/code/INTERNAL_RECIPIENT_ERROR, payload=Failed to retrieve own DAPS token, preventing a valid response., type=class de.fraunhofer.iais.eis.RejectionMessageImpl})]

[sebplorenz]

Hi, I had two commits locally that I forgot to push. Please update your local branch and try again. This should fix the error from your message above.

[jfernandezsqs]

Thanks @sebplorenz! Your solution is working correctly. However, this PR #62 does not use testbed CA certificates. I have created PR #71 that includes your proposed solution using certificates obtained from IDS-testbed CA. Therefore, I proceed to close this pull request.