Segmentation fault on Docker

[faibistes]

This minimal program:

using LLama;

string modelPath = @"/model/model.gguf"; // change it to your own model path.

var parameters = new ModelParams(modelPath)
{
    ContextSize = 2048
};
using var model = LLamaWeights.LoadFromFile(parameters);

with these dependencies:

<ItemGroup>
    <PackageReference Include="LLamaSharp" Version="0.10.0" />
    <PackageReference Include="LLamaSharp.Backend.Cpu" Version="0.10.0" />
  </ItemGroup>

Crashes with segmentation fault inside a docker container. The machine has 32 GB of RAM. The program doesn't crash on my windows dev machine, with the same model (codellama)

lscpu:

Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                4
On-line CPU(s) list:   0-3
Thread(s) per core:    2
Core(s) per socket:    2
Socket(s):             1
NUMA node(s):          1
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 85
Model name:            Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
Stepping:              7
CPU MHz:               2593.907
BogoMIPS:              5187.81
Virtualization:        VT-x
Hypervisor vendor:     Microsoft
Virtualization type:   full
L1d cache:             32K
L1i cache:             32K
L2 cache:              1024K
L3 cache:              36608K
NUMA node0 CPU(s):     0-3
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology eagerfpu pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single tpr_shadow vnmi ept vpid fsgsbase bmi1 hle avx2 smep bmi2 erms invpcid rtm avx512f avx512dq rdseed adx smap clflushopt avx512cd avx512bw avx512vl xsaveopt xsavec md_clear

[AsakusaRinne]

Hi, it's an unexpected BUG because I know there're people using LLamaSharp in dockers successfully. Which docker image are you using, or it just crashes in any kinds of docker image?

[faibistes]

This is the Dockerfile:

#See https://aka.ms/containerfastmode to understand how Visual Studio uses this Dockerfile to build your images for faster debugging.

FROM mcr.microsoft.com/dotnet/aspnet:8.0.2-alpine3.19 AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443

FROM mcr.microsoft.com/dotnet/sdk:8.0.201-alpine3.19 AS build
WORKDIR /src
#COPY ["nuget.config", "."]
COPY ["DocGenerator.Pruebas/DocGenerator.Pruebas.csproj", "DocGenerator.Pruebas/"]
COPY ["DocGenerator.Llama/DocGenerator.Llama.csproj", "DocGenerator.Llama/"]
RUN dotnet restore "DocGenerator.Pruebas/DocGenerator.Pruebas.csproj"
COPY . .
WORKDIR "/src/DocGenerator.Pruebas"
RUN dotnet build "DocGenerator.Pruebas.csproj" -c Release -o /app/build

FROM build AS publish
RUN dotnet publish "DocGenerator.Pruebas.csproj" -c Release -o /app/publish /p:UseAppHost=false

FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENV ASPNETCORE_ENVIRONMENT Production
RUN ln -snf /usr/share/zoneinfo/Europe/Madrid /etc/localtime && echo Europe/Madrid > /etc/timezone
ENTRYPOINT ["tail", "-f", "/dev/null"]

Docker version 19.03.13, build 4484c46d9d