Long Execution Time for dotnet restore in OpenShift Build Pipeline

[linhvuquach]

During our build pipeline execution on OpenShift, we've encountered a significant delay specifically with the dotnet restore command in the Dockerfile. This delay is causing notable slowdowns in our overall build process.

My Dockerfile

#See https://aka.ms/containerfastmode to understand how Visual Studio uses this Dockerfile to build your images for faster debugging.

FROM mcr.microsoft.com/dotnet/aspnet:8.0 AS base
WORKDIR /app
EXPOSE 8080
EXPOSE 8081

FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build
ARG BUILD_CONFIGURATION=Release
ARG HTTP_PROXY="<my-url-local-proxy>"
ENV DOTNET_NUGET_SIGNATURE_VERIFICATION=false
WORKDIR /src
COPY ["My.API/My.API.csproj", "My.API/"]
COPY ["My.Application/My.Application.csproj", "My.Application/"]
COPY ["My.Infrastructure/My.Infrastructure.csproj", "My.Infrastructure/"]
COPY "My.API/nuget.config" .

RUN dotnet restore -v diag "./My.API/./My.API.csproj"
COPY . .
WORKDIR "/src/My.API"
RUN dotnet build "./My.API.csproj" -c $BUILD_CONFIGURATION -o /app/build

FROM build AS publish
RUN dotnet publish "./My.API.csproj" -c $BUILD_CONFIGURATION -o /app/publish /p:UseAppHost=false

FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .

ENTRYPOINT ["dotnet", "My.API.dll"]

My nugget.config

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <clear />
    <add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />
  </packageSources>
</configuration>

Version

I'm using openshift version 4.

Current Result

• Local result: I can work fine with my Dockerfile in my local.
• OpenShift result: The dotnet restore operation consistently takes 1-hour duration and seems to time out during the MSBuild within our OpenShift build pipeline.

Additional Information

• I saw a make sense information about NuGet signed-package verification https://github.com/NuGet/Home/issues/13062 and additional ENV DOTNET_NUGET_SIGNATURE_VERIFICATION=false in Dockerfile. But it didn't resolve yet.
  • You also need it for more information: https://learn.microsoft.com/en-us/dotnet/core/tools/nuget-signed-package-verification
• In my Dockerfile I also add a Proxy variable and use nuget.config file. You can look at it above.
• I tried to create another Node simple application image. But it also stuck on the npm install step. I think the same concept with the dotnet restore command.
• I'm always looking at my resource quota. I don't see CPU, memory,... reach limit:
• I also tried to change aspnet:8.0 to aspnet:7.0. It still didn't success

Any insights or guidance will be expected. Thank you for your attention to this matter.

[linhvuquach]

Update

I'm confused about the requests % in the Observe function

CPU quota

Memory quota

[kopcot]

I had a same issue and my workaround is like that : added enviroment values (even for some of them are already used in the sdk):

ENV DOTNET_USE_POLLING_FILE_WATCHER=false 
ENV NUGET_XMLDOC_MODE=skip 
ENV NUGET_CERT_REVOCATION_MODE=offline 

use mount=type=cache in each restore / build / publish I have multiple services in one docker-compose and with that, I was able to decrease restore/build/publish time (bcs. of using same cache volume) time from like 1hour to 5min -- I'm using low personal NAS-server for it so time are higher as on normal Win-PC

Dockerfile for single service:

FROM mcr.microsoft.com/dotnet/aspnet:8.0.2-alpine3.19-composite-arm64v8 AS base
WORKDIR /app
RUN apk update && apk add curl
RUN rm -rf /var/cache/apk/*
HEALTHCHECK --interval=5s --timeout=10s --retries=5 --start-period=30s \
  CMD curl -f http://localhost:8080/api/health || exit 1
EXPOSE 8080

FROM mcr.microsoft.com/dotnet/sdk:8.0-alpine AS restorer
ARG BUILD_CONFIGURATION=Release
ARG BUILD_PLATFORM=linux-arm64
ENV DOTNET_NUGET_SIGNATURE_VERIFICATION=false
ENV DOTNET_USE_POLLING_FILE_WATCHER=false
ENV NUGET_XMLDOC_MODE=skip
ENV NUGET_CERT_REVOCATION_MODE=offline
RUN mkdir /src
WORKDIR /src
COPY ["Services/Catalog/Catalog.API/Catalog.API.csproj", "./Services/Catalog/Catalog.API/"]
COPY ["Services/Catalog/Catalog.Application/Catalog.Application.csproj", "./Services/Catalog/Catalog.Application/"]
COPY ["Services/Catalog/Catalog.Infrastructure/Catalog.Infrastructure.csproj", "./Services/Catalog/Catalog.Infrastructure/"]
COPY ["Services/Catalog/Catalog.Core/Catalog.Core.csproj", "./Services/Catalog/Catalog.Core/"]
COPY ["Services/Shared/Shared.Api/Shared.Api.csproj", "./Services/Shared/Shared.Api/"]
COPY ["Services/Shared/Shared.Application/Shared.Application.csproj", "./Services/Shared/Shared.Application/"]
COPY ["Services/Shared/Shared.Infrastructure/Shared.Infrastructure.csproj", "./Services/Shared/Shared.Infrastructure/"]
COPY ["Services/Shared/Shared.Core/Shared.Core.csproj", "./Services/Shared/Shared.Core/"]
WORKDIR "/src/Services/Catalog/Catalog.API"
RUN \
    --mount=type=cache,id=nugethttpcache,sharing=locked,target=/root/.local/share/NuGet/http-cache \
    --mount=type=cache,id=nugetglobalpackages,sharing=locked,target=/root/.nuget/packages/ \
    --mount=type=cache,id=nugettemp,sharing=locked,target=/Temp/NuGet \
    --mount=type=cache,id=nugetplugins-cache,sharing=locked,target=/root/.local/share/NuGet/plugin-cache \
#    dotnet restore "./Catalog.API.csproj" -r $BUILD_PLATFORM --disable-parallel
    dotnet restore "./Catalog.API.csproj" -r $BUILD_PLATFORM

FROM restorer AS build
WORKDIR /src
COPY . .
WORKDIR "/src/Services/Catalog/Catalog.API"
RUN \
    --mount=type=cache,id=nugethttpcache,sharing=locked,target=/root/.local/share/NuGet/http-cache \
    --mount=type=cache,id=nugetglobalpackages,sharing=locked,target=/root/.nuget/packages/ \
    --mount=type=cache,id=nugettemp,sharing=locked,target=/Temp/NuGet \
    --mount=type=cache,id=nugetplugins-cache,sharing=locked,target=/root/.local/share/NuGet/plugin-cache \
    dotnet build "./Catalog.API.csproj" -c $BUILD_CONFIGURATION -r $BUILD_PLATFORM --self-contained false -o /app/build 

FROM build AS publish
RUN \
    --mount=type=cache,id=nugethttpcache,sharing=locked,target=/root/.local/share/NuGet/http-cache \
    --mount=type=cache,id=nugetglobalpackages,sharing=locked,target=/root/.nuget/packages/ \
    --mount=type=cache,id=nugettemp,sharing=locked,target=/Temp/NuGet \
    --mount=type=cache,id=nugetplugins-cache,sharing=locked,target=/root/.local/share/NuGet/plugin-cache \
    dotnet publish "./Catalog.API.csproj" -c $BUILD_CONFIGURATION -r $BUILD_PLATFORM --self-contained false -o /app/publish /p:UseAppHost=false

FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "/app/Catalog.API.dll"]

[dungphamdev]

This issue may occur when the build pod reaches its limit. Please try adding more resources to the Tekton builder pod, as shown below:

        - pipelineTaskName: build
          computeResources:
            requests:
              memory: "512Mi"
              cpu: "250m"
            limits:
              memory: "1024Mi"
              cpu: "500m

Read more at: https://github.com/tektoncd/pipeline/blob/main/docs/pipelineruns.md#specifying-taskrunspecs

[linhvuquach]

It works for me if I add taskRunSpecs in the trigger-template.yaml . Thanks @dungphamdev

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close. Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale