Build failed

[spetrac]

Describe the bug Building the trusted connector from a freshly cloned repository failed at synchronizing artifacts back to /core-platform.

To Reproduce I work with windows 11 and docker on WSL2. The build failed at the last step previously when using the docker-compose, but to make sure its not just windows, I launched a Ubuntu subsystem with docker. The build started and continued normal, but at last it failed with the same error message:

BUILD FAILED in 4m 30s
20 actionable tasks: 20 executed
Synchronize built artifacts back to /core-platform...
sending incremental file list
rsync: change_dir "/build/ids-connector/build/libs" failed: No such file or directory (2)

sent 20 bytes  received 12 bytes  64.00 bytes/sec
total size is 0  speedup is 0.00
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1207) [sender=3.1.3]

[spetrac]

I missed the 2 build failures that were above the BUILD FAILED message:

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':ids-infomodel-manager:spotlessKotlinCheck'.
> The following files had format violations:
      src/main/kotlin/de/fhg/aisec/ids/informationmodelmanager/InfoModelService.kt
          @@ -1,197 +1,197 @@
          -/*-\r\n
          - * ========================LICENSE_START=================================\r\n
          - * ids-infomodel-manager\r\n
          - * %%\r\n
          - * Copyright (C) 2021 Fraunhofer AISEC\r\n
          - * %%\r\n
          - * Licensed under the Apache License, Version 2.0 (the "License");\r\n
          - * you may not use this file except in compliance with the License.\r\n
          - * You may obtain a copy of the License at\r\n
          - *\r\n
          - *      http://www.apache.org/licenses/LICENSE-2.0\r\n
          - *\r\n
          - * Unless required by applicable law or agreed to in writing, software\r\n
          - * distributed under the License is distributed on an "AS IS" BASIS,\r\n
          - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\r\n
          - * See the License for the specific language governing permissions and\r\n
          - * limitations under the License.\r\n
          - * =========================LICENSE_END==================================\r\n
          - */\r\n
          -package de.fhg.aisec.ids.informationmodelmanager\r\n
          -\r\n
          -import de.fhg.aisec.ids.api.conm.ConnectionManager\r\n
          -import de.fhg.aisec.ids.api.infomodel.ConnectorProfile\r\n
          -import de.fhg.aisec.ids.api.infomodel.InfoModel\r\n
          -import de.fhg.aisec.ids.api.settings.Settings\r\n
          -import de.fraunhofer.iais.eis.Connector\r\n
          -import de.fraunhofer.iais.eis.ConnectorEndpointBuilder\r\n
          -import de.fraunhofer.iais.eis.Resource\r\n
          -import de.fraunhofer.iais.eis.ResourceBuilder\r\n
          -import de.fraunhofer.iais.eis.ResourceCatalog\r\n
          -import de.fraunhofer.iais.eis.ResourceCatalogBuilder\r\n
          -import de.fraunhofer.iais.eis.SecurityProfile\r\n
          -import de.fraunhofer.iais.eis.TrustedConnector\r\n
          -import de.fraunhofer.iais.eis.TrustedConnectorBuilder\r\n
          -import de.fraunhofer.iais.eis.ids.jsonld.Serializer\r\n
          -import de.fraunhofer.iais.eis.util.ConstraintViolationException\r\n
          -import de.fraunhofer.iais.eis.util.TypedLiteral\r\n
          -import org.slf4j.LoggerFactory\r\n
          -import org.springframework.beans.factory.annotation.Autowired\r\n
          -import org.springframework.stereotype.Component\r\n
          -import java.net.URI\r\n
          -import java.net.URISyntaxException\r\n
          -\r\n
          -/** IDS Info Model Manager. */\r\n
          -@Component("ids-infomodel-manager")\r\n
          -class InfoModelService : InfoModel {\r\n
          -    @Autowired\r\n
          -    private lateinit var settings: Settings\r\n
      ... (346 more lines that didn't fit)
  Violations also present in:
      src/main/kotlin/de/fhg/aisec/ids/informationmodelmanager/deserializer/CustomModule.kt
      src/main/kotlin/de/fhg/aisec/ids/informationmodelmanager/deserializer/CustomObjectMapper.kt
  Run './gradlew :ids-infomodel-manager:spotlessApply' to fix these violations.

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
> Get more help at https://help.gradle.org.
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':ids-connector:spotlessKotlinCheck'.
> The following files had format violations:
      src/main/kotlin/de/fhg/aisec/ids/ConnectorConfiguration.kt
          @@ -1,146 +1,146 @@
          -/*-\r\n
          - * ========================LICENSE_START=================================\r\n
          - * ids-connector\r\n
          - * %%\r\n
          - * Copyright (C) 2021 Fraunhofer AISEC\r\n
          - * %%\r\n
          - * Licensed under the Apache License, Version 2.0 (the "License");\r\n
          - * you may not use this file except in compliance with the License.\r\n
          - * You may obtain a copy of the License at\r\n
          - *\r\n
          - *      http://www.apache.org/licenses/LICENSE-2.0\r\n
          - *\r\n
          - * Unless required by applicable law or agreed to in writing, software\r\n
          - * distributed under the License is distributed on an "AS IS" BASIS,\r\n
          - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\r\n
          - * See the License for the specific language governing permissions and\r\n
          - * limitations under the License.\r\n
          - * =========================LICENSE_END==================================\r\n
          - */\r\n
          -package de.fhg.aisec.ids\r\n
          -\r\n
          -import de.fhg.aisec.ids.api.LazyProducer\r\n
          -import de.fhg.aisec.ids.api.cm.ContainerManager\r\n
          -import de.fhg.aisec.ids.api.infomodel.InfoModel\r\n
          -import de.fhg.aisec.ids.api.settings.Settings\r\n
          -import de.fhg.aisec.ids.camel.idscp2.ListenerManager\r\n
          -import de.fhg.aisec.ids.camel.idscp2.Utils\r\n
          -import de.fhg.aisec.ids.camel.processors.UsageControlMaps\r\n
          -import de.fhg.aisec.ids.rm.RouteManagerService\r\n
          -import org.springframework.beans.factory.annotation.Autowired\r\n
          -import org.springframework.beans.factory.annotation.Value\r\n
          -import org.springframework.boot.CommandLineRunner\r\n
          -import org.springframework.context.ApplicationContext\r\n
          -import org.springframework.context.annotation.Bean\r\n
          -import org.springframework.context.annotation.Configuration\r\n
          -import java.net.URI\r\n
          -\r\n
          -@Configuration\r\n
          -class ConnectorConfiguration {\r\n
          -    @Autowired(required = false)\r\n
          -    private var cml: ContainerManager? = null\r\n
          -\r\n
          -    @Autowired\r\n
          -    private lateinit var settings: Settings\r\n
          -\r\n
          -    @Autowired\r\n
          -    private lateinit var im: InfoModel\r\n
          -\r\n
      ... (244 more lines that didn't fit)
  Violations also present in:
      src/main/kotlin/de/fhg/aisec/ids/Idscp2UsageControlComponent.kt
      src/main/kotlin/de/fhg/aisec/ids/TrustedConnector.kt
  Run './gradlew :ids-connector:spotlessApply' to fix these violations.

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
> Get more help at https://help.gradle.org.
==============================================================================

I suppose, the error could be related to windows-style line-breaks.

[spetrac]

I cloned the repository again with the Ubuntu subsystem and the build did not fail this time.

[milux]

This error was maybe related to outdated source code. Errors thrown came from the linter "spotless" we run on our code. Will check, maybe I've overlooked it during the last release.

[milux]

Nope, checked once again. No linter issues here. :wink:

[spetrac]

I already closed this issue, because cloning the repository with linux resolved the failure.

This means that the issue was caused by the windows-style line-breaks, because git on windows converts the line-endings accordingly. The problem is the build container, which accesses the files from the mounted volume and cannot handle the \r\n line-breaks. I think the system should be more robust and ignore the cariage-return and I believe that there is a solution for this, but currently windows is not even listed as supported platform so this might not have priority.

[milux]

I wouldn't exactly consider checking out the repo on a Windows computer and then building that sources within a linux VM (WSL2) a standard scenario. Apart from that, there are multiple options to resolve this:

• You can configure your git in general or one repo specifically to always use Linux-style line endings. No advanced IDE should have any problem with that.
• You can turn off spotless tasks, look at the Gradle docs or our build script. Removing "check" target from the gradlew command may also suffice.
• You could checkout under WSL2, and even develop there. WSL2, AFAIK, does support GUI-applications by now. There's even support for IntelliJ stuff there or on the way: https://youtrack.jetbrains.com/issue/IJPL-94601/Running-IntelliJ-from-WSL2

Native Windows is not supported for the simple reason that everything is build based on linux containers, which are not really supported there, and especially UDS (Unix Domain Sockets), which, as the name suggests, do not even exist in non-Unix OSes.

[spetrac]

Yes, I already did exactly the third options. That is what I meant by cloning the repository with linux. I cloned the repository with Ubuntu on WSL2 and run the container from their with docker enabled on WSL2. This worked totally fine.

Initially I tried running the container with windows completely, not within WSL2 runtime. The problem was the linux from the build-container image which loaded the cloned folder as volume and acted on it.

[milux]

Yes, I already did exactly the third options. That is what I meant by cloning the repository with linux. I cloned the repository with Ubuntu on WSL2 and run the container from their with docker enabled on WSL2. This worked totally fine.

Initially I tried running the container with windows completely, not within WSL2 runtime. The problem was the linux from the build-container image which loaded the cloned folder as volume and acted on it.

Yeah, that's what I understood. Just assumed that the reason you started off with Windows was that your IDE(s) are there, and you want to make modifications to the TC source. If not, what would be the reason for building the TC yourself in the first place? Docker images are available for download with every version released.

[spetrac]

I actually could not find an official aisec docker image, neither as github package in this repo nor on docker hub, so I just followed the instructions from the "How to build & run" section and the linked "Building the Core Platform" page.

Edit: Found the link for docker-hub in the faq. Previously, I have just search for "trusted-connector" and only got the result from farmstack.

[milux]

I have just search for "trusted-connector" and only got the result from farmstack.

You may want to try "docker hub trusted connector". First 10 Google results for me are Docker image versions of the TC core container, i.e. "the connector". :sweat_smile:

[milux]

But good point, we should include this way more prominent into the README. :+1: