search

Green-Software-Foundation / carbon-aware-sdk

Carbon-Aware SDK
https://carbon-aware-sdk.greensoftware.foundation/
MIT License
463 stars 98 forks source link

[URGENT] WebAPI container has not built due to segmentation fault #496

Closed YaSuenag closed 5 months ago

YaSuenag commented 5 months ago

Contact Details

@YaSuenag

What happened?

493 has been merged, then 2-pre-release.yaml was triggered,but it failed due to segmentation fault. It seems to occur on Arm64 only. (x64 seems to be done without any error)

I did not see this segfault on my environment (Podman on Fedora 39 AArch64), so it is not a bug in CASDK, but we need to investigate.

@danuw @vaughanknight Can you run again 2-pre-release.yaml ? And do you know any known issue about this?

client

WebAPI (Default)

Relevant log output

https://github.com/Green-Software-Foundation/carbon-aware-sdk/actions/runs/8536833345/job/23386182495#step:6:643

 > [linux/arm64 build-env 4/6] RUN dotnet publish CarbonAware.WebApi/src/CarbonAware.WebApi.csproj -c Release -o publish:
7.209 MSBuild version 17.9.6+a4ecab324 for .NET
24.69   Determining projects to restore...
41.42 Segmentation fault (core dumped)

Code of Conduct

YaSuenag commented 5 months ago

I found this article: https://devblogs.microsoft.com/dotnet/improving-multiplatform-container-support/

This article says .NET does not support QEMU, and alternatives to create container image for other platform. Generating WebAPI container image depends docker buildx from docker/build-push-action, and it uses QEMU (maybe), hence I think it (use QEMU) is root cause of this error.

So I changed with following patch:

diff --git a/src/CarbonAware.WebApi/src/Dockerfile b/src/CarbonAware.WebApi/src/Dockerfile
index f01e277..906230a 100644
--- a/src/CarbonAware.WebApi/src/Dockerfile
+++ b/src/CarbonAware.WebApi/src/Dockerfile
@@ -1,11 +1,12 @@
-FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build-env
+FROM --platform=$BUILDPLATFORM mcr.microsoft.com/dotnet/sdk:8.0 AS build-env
+ARG TARGETARCH
 WORKDIR /app

 ENV DOTNET_ROLL_FORWARD LatestMajor
 # Copy everything from source
 COPY . ./
 # Use implicit restore to build and publish
-RUN dotnet publish CarbonAware.WebApi/src/CarbonAware.WebApi.csproj -c Release -o publish
+RUN dotnet publish CarbonAware.WebApi/src/CarbonAware.WebApi.csproj -a $TARGETARCH -o publish
 # Generate OpenAPI spec
 WORKDIR /app/CarbonAware.WebApi/src
 RUN dotnet tool restore && \

However I saw following error when I attempt to create WebAPI container image.

[1/2] STEP 6/8: RUN dotnet publish CarbonAware.WebApi/src/CarbonAware.WebApi.csproj -a $TARGETARCH -c Release -o publish
MSBuild version 17.9.6+a4ecab324 for .NET
  Determining projects to restore...
  Restored /app/CarbonAware/src/CarbonAware.csproj (in 12.49 sec).
  Restored /app/CarbonAware.DataSources/CarbonAware.DataSources.Registration/CarbonAware.DataSources.Registration.csproj (in 12.49 sec).
  Restored /app/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMapsFree/src/CarbonAware.DataSources.ElectricityMapsFree.csproj (in 39 ms).
  Restored /app/CarbonAware.DataSources/CarbonAware.DataSources.Json/src/CarbonAware.DataSources.Json.csproj (in 38 ms).
  Restored /app/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/CarbonAware.DataSources.ElectricityMaps.csproj (in 39 ms).
/app/GSF.CarbonAware/src/GSF.CarbonAware.csproj : error NU1605: Warning As Error: Detected package downgrade: System.IO.FileSystem.Primitives from 4.3.0 to 4.0.1. Reference the package directly from the project to select a different version.  [/app/CarbonAware.WebApi/src/CarbonAware.WebApi.csproj]
/app/GSF.CarbonAware/src/GSF.CarbonAware.csproj : error NU1605:  GSF.CarbonAware -> CarbonAware -> Microsoft.AspNetCore.Mvc.Core 2.2.5 -> Microsoft.Extensions.DependencyModel 2.1.0 -> Microsoft.DotNet.PlatformAbstractions 2.1.0 -> System.IO.FileSystem 4.0.1 -> runtime.unix.System.IO.FileSystem 4.3.0 -> System.IO.FileSystem.Primitives (>= 4.3.0)  [/app/CarbonAware.WebApi/src/CarbonAware.WebApi.csproj]
/app/GSF.CarbonAware/src/GSF.CarbonAware.csproj : error NU1605:  GSF.CarbonAware -> CarbonAware -> Microsoft.AspNetCore.Mvc.Core 2.2.5 -> Microsoft.Extensions.DependencyModel 2.1.0 -> Microsoft.DotNet.PlatformAbstractions 2.1.0 -> System.IO.FileSystem 4.0.1 -> System.IO.FileSystem.Primitives (>= 4.0.1) [/app/CarbonAware.WebApi/src/CarbonAware.WebApi.csproj]
  Failed to restore /app/GSF.CarbonAware/src/GSF.CarbonAware.csproj (in 10.38 sec).
  Restored /app/CarbonAware.WebApi/src/CarbonAware.WebApi.csproj (in 10.33 sec).
  Restored /app/CarbonAware.LocationSources/src/CarbonAware.LocationSources.csproj (in 43 ms).
  Restored /app/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/CarbonAware.DataSources.WattTime.csproj (in 42 ms).
Error: building at STEP "RUN dotnet publish CarbonAware.WebApi/src/CarbonAware.WebApi.csproj -a $TARGETARCH -c Release -o publish": while running runtime: exit status 1

Maybe they are resolved if we update dependency versions, but I don't know why they are not seen without the change. Someone can resove them?

YaSuenag commented 5 months ago

I don't know why they are not seen without the change.

Maybe the cause is "Native dependencies": https://learn.microsoft.com/en-us/dotnet/core/deploying/deploy-with-cli#native-dependencies

So can we update dependencies? I think it is allowed because we've already migrated to .NET 8.

YaSuenag commented 5 months ago

@vaughanknight @danuw I fixed this issue in #498 . I hope it is merged ASAP.