DirectoryNotFoundException in Docker Build

anurmatov commented 2 months ago

Environment

Metalama.Framework Version: 2024.2.19 Runtime: .NET 8.0.1 OS: Debian GNU/Linux 12 (bookworm) Docker Base Image: mcr.microsoft.com/dotnet/sdk:8.0-bookworm-slim

Issue Description

During our Docker build process, Metalama is encountering a DirectoryNotFoundException. The build (same with dotnet test) fails when trying to access a directory that doesn't exist.

Error message

System.IO.DirectoryNotFoundException: Could not find a part of the path '/tmp/mtlm/Metalama/AssemblyLocator/2024.2.19/default/bin/Debug/net6.0'.

Stack Trace

from metalama crash log file:

16.66 Build failed. Outputting Metalama crash report:
16.67 Metalama Version: 2024.2.19
16.67 Runtime: .NET 8.0.1
16.67 Processor Architecture: Arm64
16.67 OS Description: Debian GNU/Linux 12 (bookworm)
16.67 OS Architecture: Arm64
16.67 Exception type: System.IO.DirectoryNotFoundException
16.67 Exception message: Could not find a part of the path '/tmp/mtlm/Metalama/AssemblyLocator/2024.2.19/default/bin/Debug/net6.0'.
16.67 ===== Exception =====
16.67 System.IO.DirectoryNotFoundException: Could not find a part of the path '/tmp/mtlm/Metalama/AssemblyLocator/2024.2.19/default/bin/Debug/net6.0'.
16.67    at System.IO.Enumeration.FileSystemEnumerator`1.CreateDirectoryHandle(String path, Boolean ignoreNotFound)
16.67    at System.IO.Enumeration.FileSystemEnumerator`1.Init()
16.67    at System.IO.Enumeration.FileSystemEnumerable`1..ctor(String directory, FindTransform transform, EnumerationOptions options, Boolean isNormalized)
16.67    at System.IO.Enumeration.FileSystemEnumerableFactory.UserFiles(String directory, String expression, EnumerationOptions options)
16.67    at System.IO.Directory.InternalEnumeratePaths(String path, String searchPattern, SearchTarget searchTarget, EnumerationOptions options)
16.67    at Metalama.Framework.Engine.CompileTime.ReferenceAssemblyLocator..ctor(ProjectServiceProvider& serviceProvider, String additionalPackageReferences)
16.67    at Metalama.Framework.Engine.CompileTime.ReferenceAssemblyLocatorProvider.GetInstance(ProjectServiceProvider& serviceProvider)
16.67    at Metalama.Framework.Engine.Services.ServiceProviderExtensions.GetReferenceAssemblyLocator(ProjectServiceProvider serviceProvider)
16.67    at Metalama.Framework.Engine.CompileTime.SystemTypeResolver..ctor(ProjectServiceProvider& serviceProvider)
16.67    at Metalama.Framework.Engine.Services.ServiceProviderFactory.<>c.<WithProjectScopedServices>b__9_4(ServiceProvider`1 sp)
16.67    at Metalama.Framework.Engine.Services.ServiceProvider`1.WithServiceConditional[T](Func`2 func)
16.67    at Metalama.Framework.Engine.Services.ServiceProviderFactory.WithProjectScopedServices(IServiceProvider`1 serviceProvider, IProjectOptions projectOptions, IEnumerable`1 metadataReferences)
16.67    at Metalama.Framework.Engine.Pipeline.SourceTransformer.Execute(TransformerContext context)
16.67 Metalama Version: 2024.2.19
16.67 Runtime: .NET 8.0.1
16.67 Processor Architecture: Arm64
16.67 OS Description: Debian GNU/Linux 12 (bookworm)
16.67 OS Architecture: Arm64
16.67 Exception type: System.IO.DirectoryNotFoundException
16.67 Exception message: Could not find a part of the path '/tmp/mtlm/Metalama/AssemblyLocator/2024.2.19/default/bin/Debug/net6.0'.
16.67 ===== Exception =====
16.67 System.IO.DirectoryNotFoundException: Could not find a part of the path '/tmp/mtlm/Metalama/AssemblyLocator/2024.2.19/default/bin/Debug/net6.0'.
16.67    at System.IO.Enumeration.FileSystemEnumerator`1.CreateDirectoryHandle(String path, Boolean ignoreNotFound)
16.67    at System.IO.Enumeration.FileSystemEnumerator`1.Init()
16.67    at System.IO.Enumeration.FileSystemEnumerable`1..ctor(String directory, FindTransform transform, EnumerationOptions options, Boolean isNormalized)
16.67    at System.IO.Enumeration.FileSystemEnumerableFactory.UserFiles(String directory, String expression, EnumerationOptions options)
16.67    at System.IO.Directory.InternalEnumeratePaths(String path, String searchPattern, SearchTarget searchTarget, EnumerationOptions options)
16.67    at Metalama.Framework.Engine.CompileTime.ReferenceAssemblyLocator..ctor(ProjectServiceProvider& serviceProvider, String additionalPackageReferences)
16.67    at Metalama.Framework.Engine.CompileTime.ReferenceAssemblyLocatorProvider.GetInstance(ProjectServiceProvider& serviceProvider)
16.67    at Metalama.Framework.Engine.Services.ServiceProviderExtensions.GetReferenceAssemblyLocator(ProjectServiceProvider serviceProvider)
16.67    at Metalama.Framework.Engine.CompileTime.SystemTypeResolver..ctor(ProjectServiceProvider& serviceProvider)
16.67    at Metalama.Framework.Engine.Services.ServiceProviderFactory.<>c.<WithProjectScopedServices>b__9_4(ServiceProvider`1 sp)
16.67    at Metalama.Framework.Engine.Services.ServiceProvider`1.WithServiceConditional[T](Func`2 func)
16.67    at Metalama.Framework.Engine.Services.ServiceProviderFactory.WithProjectScopedServices(IServiceProvider`1 serviceProvider, IProjectOptions projectOptions, IEnumerable`1 metadataReferences)
16.67    at Metalama.Framework.Engine.Pipeline.SourceTransformer.Execute(TransformerContext context)

Dockerfile

FROM mcr.microsoft.com/dotnet/sdk:8.0-bookworm-slim AS build
ENV METALAMA_TEMP=/tmp/mtlm
ARG CONFIGURATION=Release
ARG VERSION=1.0.0.0
WORKDIR /src

# ... [Copy commands for project files omitted for brevity]

RUN dotnet restore "Solution.sln" --configfile "NuGet.config"
# RUN dotnet test "Tests/Tests.csproj" -c ${CONFIGURATION} --no-restore
RUN dotnet build "Project/Project.csproj" -c ${CONFIGURATION} --no-restore -p:Version=${VERSION} -o /app/build || \
    (echo "Build failed. Outputting Metalama crash report:" && \
     cat /tmp/mtlm/Metalama/CrashReports/*.txt && \
     exit 1)

# ... [Rest of the Dockerfile omitted for brevity]

Additonal context

We're using a multi-stage Dockerfile for our build process
The error occurs during the dotnet build step
We've set the METALAMA_TEMP environment variable to /tmp/mtlm

anurmatov commented 2 months ago

worth mentioning if you create a container from last succeeded layer (that dotnet restore step) and run same test/build command it works just fine

gfraiteur commented 2 months ago

I've seen that before but it seemed random.

Does it seem deterministic to you?
Do you have several projects using Metalama, or just one? Part of the randomness could be the parallelism.
Does the problem disappear if you disable parallel build? dotnet build -m:1 --disable-build-servers

anurmatov commented 2 months ago

thanks for the comment, it's deterministic yes, let me try to test w/ turned off parallelism

anurmatov commented 2 months ago

still same issue w/ dotnet build -m:1 --disable-build-servers

project structure:

multiple projects in our solution that use Metalama
there is a private NuGet package containing custom aspects
our main project references this private package with custom aspects

anurmatov commented 2 months ago

our attempts with different metalama versions (2024.0.16, 2024.1.23, 2024.2.19) aren't working out

tests are passed when running in a container from last succeeded docker layer:

@gfraiteur if needed we can come up with a demo for repro

prochan2 commented 2 months ago

Hello @anurmatov . May I ask you to collect logs of the failing build? Collecting Metalama logs is described at https://doc.postsharp.net/metalama/conceptual/configuration/creating-logs .

Before we figure this out, a workaround may be to build an empty project with Metalama installed before building the rest of your projects. The version of Metalama in the project must match the version used in your project.

anurmatov commented 2 months ago

hello @prochan2, attached logs to the ticket, please find them here support.postsharp.net

trying to get a workaround to work - no luck yet

addabis commented 1 month ago

@anurmatov This was fixed in 2024.2.21, the configuration (any casing) env var should be now removed for the helper project build.

postsharp / Metalama