Support AOT compilation

[mcdis]

Elastic.Clients.Elasticsearch version: 8.12.0

.NET runtime version: v8.0.101

Operating system version: Windows 11

Description of the problem including expected versus actual behavior: When NativeAOT mode is enabled, data cannot be sent and an exception is thrown. Setting the TypeInfoResolver in JsonSerializerOptions does not help since the execution does not reach the serializer.

Steps to reproduce:

1. Create csproj and enable NativeAot ( true )
2. Create document type with Native Aot serialization (System Json Source Generation) compatible:

   
   [JsonSourceGenerationOptions(UseStringEnumConverter = true, 
   PropertyNamingPolicy = JsonKnownNamingPolicy.CamelCase)]
   [JsonSerializable(typeof(ElasticSearchLogEntry))]
   public partial class ElasticSearchLogEntryContext : JsonSerializerContext;

public class ElasticSearchLogEntry { public static ElasticSearchLogEntryContext Context => ElasticSearchLogEntryContext.Default;

public int Level { get; init; } public int Priority { get; init; } public string Scope { get; init; } public long Timestamp { get; init; } public string Description { get; init; } }

 4. Try IndexAsync(new ElasticSearchLogEntry());
 5. Exception:

InvalidOperationException: Reflection-based serialization has been disabled for this application. Either use the source generator APIs or explicitly configure the 'JsonSerializerOptions.TypeInfoResolver' property.

StackTrace:

at Elastic.Transport.DefaultHttpTransport1.ThrowUnexpectedTransportException[TResponse](Exception killerException, List1 seenExceptions, RequestData requestData, TResponse response, RequestPipeline pipeline) at Elastic.Transport.DefaultHttpTransport1.<RequestCoreAsync>d__191.MoveNext() at Elastic.Clients.Elasticsearch.ElasticsearchClient.<>cDisplayClass28_0`3.<gSendRequest|0>d.MoveNext() at App.Logging.ElasticSearch.ElasticSearchLogListener.<>cDisplayClass3_1.<<-ctor>b1>d.MoveNext()

[flobernd]

Hi @mcdis, AOT compilation is not supported at the moment. There must be a JsonSerializerContext for both, the source- and the request/response serializer. The later one is internal.

Besides that, I'm not even sure if we can support AOT compilation at all. It will at least require some tricks and workarounds as we are creating instances of generic classes with dynamic generic arguments (e.g. for certain JsonConverter classes) in some places. The trimmer might not be able to determine the potential types for the type argument and remove them from the resulting binary. To be able to use things like MakeGenericType, we must ensure that spezialized code exists for a given type. This is not always possible, if types are supplied by the user.

I will leave this issue open as a reminder to do some research in that direction.

[mcdis]

We are using new static abstract methods and generics constraints in interface to do workaround and continue using generics methods with serialization.

Add new interface:

interface IUserPayload
{
  public static abstract IJsonTypeInfoResolver TypeInfoResolver {get;} 
}

Let declare some ordinary Channel with Send generic method:

class Channel
{
  void Send<T>(T _request) where T:IUserPayload
}

So, we can workarond trimmer and link make type links transparency. Usage:

// Code generation context serialization
[JsonSourceGenerationOptions(UseStringEnumConverter = true,
  PropertyNamingPolicy = JsonKnownNamingPolicy.CamelCase)]
[JsonSerializable(typeof(MyJsonSerializablePayload))]
public partial class MyJsonSerializablePayloadContext : JsonSerializerContext;

// User payload
class MyJsonSerializablePayload : IUserPayload
{
  public static IJsonTypeInfoResolver Context => MyJsonSerializablePayloadContext.Default; // Link serializer to type

 // Message body
  public int X {get;init;}
  public int Y {get;init;}
  public int Z {get;init;}
}

// Execution
var channel = new Channel();
var request = new MyJsonSerializablePayload
{
   X = 1,Y = 2, Z = 3
};
channel.Send(request);

Channel inside generic method Send can get IJsonTypeInfoResolver thru typeof(IUserPayload).Context :

void Send<T>(T _request) where T:IUserPayload
{
  var  serializationContext = typeof(T).Context;
  // ...
}

[mcdis]

to summarize, you can serialize your request according to your own rules and regulations, and serialize user data using the attached context, as I showed, or you can try to ask for this context. Then insert user data as a json document or token into the request body.

[mcdis]

The system serializer also allows you to set chains of type resolvers and on the edge, we could simply include generic types for request/response in our user context or TypeInfoResolverChain chain like:

[JsonSerializable(typeof(MyJsonSerializablePayload))]
[JsonSerializable(typeof(IndexRequestDescriptor<MyJsonSerializablePayload>))]
public partial class SerializationContext : JsonSerializerContext;

[mcdis]

I can also suggest limiting ourselves to partial native aot support, for example only for adding data to the index

[flobernd]

Hi @mcdis, thanks for the code examples! Besides the (de-)serialization part, we as well use reflection in other parts of the library. I would have to check all these places, if they are safe to work with the trimmer / AOT compilation. Besides that, the static code generation feature in System.Text.Json is rather new and I have to confirm this works with older C# lang / framework combinations.

I sadly don't expect to have time to look into that soon, but I'll definitely leave this issue open as a reminder. AOT compliation is an interesting feature and especially the System.Text.Json static code generation provides a nice performance boost even for managed / JITed code.

[xiaoyuvax]

I've recently refactored my entire project to nativeAOT, the main work is to replace json.net with System.Text.Json, seems not quite difficult since no UI involved, however NEST is the last part which i can't incorporate, expecting a new nativeAot compatible version to come out asap.

on porting to NativeAOT, several of my cents: some reflection related trimming or nativeAot compatibility warnings at compile-time (mainly on parameters) can be easily solved by adding [DynamicallyAccessedMembersAttribute] or can just be ignored, if they r properly referenced by the source generator, according to this dicussion: https://github.com/dotnet/runtime/discussions/96532#discussioncomment-8155733

Finally, any 3rd party dependency which is not AOT compatible is the final headache. :`(

[flobernd]

on porting to NativeAOT, several of my cents

Thanks to you as well! I know how to get around most of the issues, but at the moment it's just a matter of time (or more precisely, the lack of it).

NEST is the last part which i can't incorporate, expecting a new nativeAot compatible version to come out asap.

Happy to accept PRs, if you need that "asap". NEST in general won't receive feature updates anymore which means all work on AOT will be done on base of the new v8.* client.

Finally, any 3rd party dependency which is not AOT compatible is the final headache. :`(

This is another point. As a predecessor, the Elastic.Transport library must be ready to support AOT.

[xiaoyuvax]

thanks for the information, i'll wait and see :) after all, we have the managed version for now.