Genbox / SimpleS3

A .NET Core implementation of Amazon's S3 API with focus on simplicity, security and performance
MIT License
50 stars 8 forks source link

GetObject not working with NativeAOT #58

Open ivanjx opened 1 year ago

ivanjx commented 1 year ago

Description of the bug PutObject works fine under NativeAOT but when i tried to get the file back with GetObject it gives me an empty stream instead. I manually verified that the PutObject works fine on both normal and AOT mode by downloading and checking the file myself.

How to reproduce?

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net8.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <PublishAot>true</PublishAot>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Genbox.SimpleS3.AmazonS3" Version="3.0.0" />
  </ItemGroup>

</Project>
using System.Text;
using Genbox.SimpleS3.AmazonS3;
using Genbox.SimpleS3.Core.Abstracts.Enums;
using Genbox.SimpleS3.Core.Common.Authentication;
using Genbox.SimpleS3.Core.Extensions;
using Genbox.SimpleS3.Core.Network.Responses.Objects;
using Genbox.SimpleS3.Extensions.AmazonS3;

AmazonS3Config config = new AmazonS3Config();
config.EndpointTemplate = "{Scheme}://play.min.io/{Bucket}";
config.Region = AmazonS3Region.UsEast1;
config.Credentials = new StringAccessKey("1nM2dy8VWatJ7EJ4nEBg", "CPeDvsHZA12kvcMXc6GJzdb6bIKo5FScCsYKKdgL");
config.NamingMode = NamingMode.PathStyle;

using AmazonS3Client client = new AmazonS3Client(config);
using MemoryStream dataStream = new MemoryStream(
    Encoding.ASCII.GetBytes("Hello World!! + " + DateTime.UtcNow.ToString("o")));
PutObjectResponse resp = await client.PutObjectAsync(
    "testbucket",
    "objectname22",
    dataStream);

GetObjectResponse resp2 = await client.GetObjectAsync("testbucket", "objectname22");
using StreamReader reader = new StreamReader(resp2.Content);
Console.WriteLine("Content: [{0}]", await reader.ReadToEndAsync());

Console.WriteLine("Success: " + resp.IsSuccess);

compile with: dotnet publish -c Release

normal output:

Content: [Hello World!! + 2023-08-05T04:00:19.6423971Z]
Success: True

NativeAOT output:

Content: []
Success: True

Expected behavior GetObject's Content has data in it.

ivanjx commented 1 year ago

there is a workaround though by using the SignObject method:

...
string url = client.SignGetObject("testbucket", "objectname22", TimeSpan.FromMinutes(1));
Console.WriteLine("Url: {0}", url);
using HttpClient http = new HttpClient();
using Stream resp2Data = await http.GetStreamAsync(url);
using StreamReader reader2 = new StreamReader(resp2Data);
Console.WriteLine("Content Stream: [{0}]", await reader2.ReadToEndAsync());
...
Content: []
Url: https://play.min.io/testbucket/objectname22?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=1nM2dy8VWatJ7EJ4nEBg%2F20230805%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20230805T041311Z&X-Amz-Expires=60&X-Amz-SignedHeaders=host&X-Amz-Signature=e1f575228ce0bcb0aae4d713bc620ad2ecc2f926f8221ed7955dc7382cd94298
Content Stream: [Hello World!! + 2023-08-05T04:13:10.2145266Z]
Success: True
Genbox commented 1 year ago

Interesting. I'll try and debug it once I get the time for it.

Genbox commented 1 year ago

I've taken a look at it now. There are several issues:

  1. RequestMarshal, ResponseMarshal, and PostMappers are matched via reflection. AOT removes all the unused types, so the marshallers are not registered.
  2. Profiles are serialized via JSON. AOT removes unused public constructors from types, so serialization does not work.
  3. Dependency injection uses dynamic code emitting. Currently, I'm using DI internally to ease the setup of clients.

Other potential problems:

I've tried to root all assemblies, which should remove the trimming issues, but the application still crash - likely due to RequestMarshal being setup via DI.

So for the moment, NativeAOT is not supported by SimpleS3.