Lambda invocation fails with "System.MissingMethodException" in net6.0

nobowned commented 2 years ago

Describe the bug

Invoking a lambda function using the net6.0 Lambda Test Tool can result in a System.MissingMethodException.

I'm not entirely sure why yet, but the following combination results in the defect:

Lambda Test Tool net6.0
Lambda function and project targeting framework net6.0
Lambda project file referencing System.Text.Json Version 6.0.5: \<PackageReference Include="System.Text.Json" Version="6.0.5" />
Calling the method "System.Net.Http.Json.JsonContent.Create" from anywhere within your lambda function's c# code.

Expected Behavior

The invocation should succeed, and the JsonContent should be created.

Current Behavior

System.MissingMethodException: Method not found: 'System.Net.Http.Json.JsonContent System.Net.Http.Json.JsonContent.Create(!!0, System.Net.Http.Headers.MediaTypeHeaderValue, System.Text.Json.JsonSerializerOptions)'. at S3EventFunction.Function.FunctionHandler(S3Event evnt, ILambdaContext context)

Reproduction Steps

Download S3EventFunction.zip
Unzip it
Open a terminal window
Navigate into the unzipped directory
Run "dotnet build" command
Run "dotnet lambda-test-tool-6.0" command
Wait for the Lambda Test Tool GUI to launch
Open the "Example Requests" drop down
Choose "S3 PUT" from the list of requests
Click "Execute Function" button

Wait for it to fail with the following response:

System.MissingMethodException: Method not found: 'System.Net.Http.Json.JsonContent System.Net.Http.Json.JsonContent.Create(!!0, System.Net.Http.Headers.MediaTypeHeaderValue, System.Text.Json.JsonSerializerOptions)'. at S3EventFunction.Function.FunctionHandler(S3Event evnt, ILambdaContext context)

If you don't trust the zip file I uploaded, then here is the code for the lambda function:

using Amazon.Lambda.Core;
using Amazon.Lambda.S3Events;

[assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.Json.JsonSerializer))]

namespace S3EventFunction;

public class Function
{
    public Function()
    {
    }

    public void FunctionHandler(S3Event evnt, ILambdaContext context)
    {
        System.Net.Http.Json.JsonContent.Create("test", options: new System.Text.Json.JsonSerializerOptions());
    }
}

and it's accompanying .csproj:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net6.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <AWSProjectType>Lambda</AWSProjectType>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Amazon.Lambda" Version="0.12.0" />
    <PackageReference Include="Amazon.Lambda.S3Events" Version="2.0.1" />
    <PackageReference Include="Amazon.Lambda.Serialization.Json" Version="2.1.0" />
    <PackageReference Include="System.Text.Json" Version="6.0.5" />
  </ItemGroup>

</Project>

Possible Solution

No response

Additional Information/Context

Workaround: Setting the System.Text.Json version to 6.0.0.

AWS .NET SDK and/or Package version used

AWS .NET Core 6.0 Mock Lambda Test Tool (0.12.3)

Targeted .NET Platform

.NET 6

Operating System and version

Windows 10, OSX Monterey

ashishdhingra commented 2 years ago

@nobowned Good afternoon. Looks like you are using a 3rd party library Amazon.Lambda and having issues while using System.Text.Json version 6.0.5. Unfortunately, we cannot support troubleshooting issues while using 3rd party libraries.

Just curious on why you are not using the AWS Visual Studio Tooklit project templates for generating the skeleton project for Lambda which uses the following dependencies (for example):

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net6.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <GenerateRuntimeConfigurationFiles>true</GenerateRuntimeConfigurationFiles>
    <AWSProjectType>Lambda</AWSProjectType>
    <!-- This property makes the build directory similar to a publish directory and helps the AWS .NET Lambda Mock Test Tool find project dependencies. -->
    <CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies>
    <!-- Generate ready to run images during publishing to improve cold start time. -->
    <PublishReadyToRun>true</PublishReadyToRun>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Amazon.Lambda.Core" Version="2.1.0" />
    <PackageReference Include="Amazon.Lambda.Serialization.SystemTextJson" Version="2.3.0" />
    <PackageReference Include="Amazon.Lambda.S3Events" Version="2.0.1" />
  </ItemGroup>
</Project>

Using the below code for Lambda function works fine:

using Amazon.Lambda.Core;
using Amazon.Lambda.S3Events;

// Assembly attribute to enable the Lambda function's JSON input to be converted into a .NET class.
[assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.SystemTextJson.DefaultLambdaJsonSerializer))]

namespace TestLambdaNet6_Issue1264;

public class Function
{

    /// <summary>
    /// A simple function that takes a string and does a aToUpper
    /// </summary>
    /// <param name="input"></param>
    /// <param name="context"></param>
    /// <returns></returns>
    public void FunctionHandler(S3Event input, ILambdaContext context)
    {
        System.Net.Http.Json.JsonContent.Create("test", options: new System.Text.Json.JsonSerializerOptions());
    }
}

nobowned commented 2 years ago

@ashishdhingra Your example also references Amazon.Lambda. Just add \<PackageReference Include="System.Text.Json" Version="6.0.5" /> to your project file and you will have the same problem as me.

Are you saying you've identified that the problem is with Amazon.Lambda and not with the test tool? How? I don't think it's possible to have a .net lambda without referencing Amazon.Lambda package.

ashishdhingra commented 2 years ago

@ashishdhingra Your example also references Amazon.Lambda. Just add \<PackageReference Include="System.Text.Json" Version="6.0.5" /> to your project file and you will have the same problem as me.

Are you saying you've identified that the problem is with Amazon.Lambda and not with the test tool? How? I don't think it's possible to have a .net lambda without referencing Amazon.Lambda package.

@nobowned If you check the owner of https://www.nuget.org/packages/Amazon.Lambda/0.12.0/ it is maintained by https://www.nuget.org/profiles/iamcarbon, not official https://www.nuget.org/profiles/awsdotnet.

nobowned commented 2 years ago

@ashishdhingra Okay, I changed it to this and it still happens. Please add \<PackageReference Include="System.Text.Json" Version="6.0.5" /> to the example you posted and it will fail.

using Amazon.Lambda.Core;
using Amazon.Lambda.S3Events;

// Assembly attribute to enable the Lambda function's JSON input to be converted into a .NET class.
[assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.SystemTextJson.DefaultLambdaJsonSerializer))]

namespace TestLambdaNet6_Issue1264;

public class Function
{

    /// <summary>
    /// A simple function that takes a string and does a aToUpper
    /// </summary>
    /// <param name="input"></param>
    /// <param name="context"></param>
    /// <returns></returns>
    public void FunctionHandler(S3Event input, ILambdaContext context)
    {
        System.Net.Http.Json.JsonContent.Create("test", options: new System.Text.Json.JsonSerializerOptions());
    }
}

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net6.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <GenerateRuntimeConfigurationFiles>true</GenerateRuntimeConfigurationFiles>
    <AWSProjectType>Lambda</AWSProjectType>
    <!-- This property makes the build directory similar to a publish directory and helps the AWS .NET Lambda Mock Test Tool find project dependencies. -->
    <CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies>
    <!-- Generate ready to run images during publishing to improve cold start time. -->
    <PublishReadyToRun>true</PublishReadyToRun>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Amazon.Lambda.Core" Version="2.1.0" />
    <PackageReference Include="Amazon.Lambda.Serialization.SystemTextJson" Version="2.3.0" />
    <PackageReference Include="Amazon.Lambda.S3Events" Version="2.0.1" />
    <PackageReference Include="System.Text.Json" Version="6.0.5" />
  </ItemGroup>
</Project>

normj commented 2 years ago

Given that System.Text.Json is automatically available as part of .NET 6 is there a reason you are explicitly adding System.Text.Json? I'm not sure yet why the MissingMethodException is happening but it doesn't happen if you don't have the PackageReference to System.Text.Json and you still have System.Text.Json available.

nobowned commented 2 years ago

Given that System.Text.Json is automatically available as part of .NET 6 is there a reason you are explicitly adding System.Text.Json? I'm not sure yet why the MissingMethodException is happening but it doesn't happen if you don't have the PackageReference to System.Text.Json and you still have System.Text.Json available.

We have a netstandard2.0 dependency that uses System.Text.Json features (this example is contrived):

using System.Text.Json;

namespace ClassLibrary1
{
    public class Class1
    {
        public void SerializeTest()
        {
            JsonSerializer.Serialize("test");
        }
    }
}

And within netstandard2.0 libraries you are forced to specify the PackageReference for System.Text.Json:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>netstandard2.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="System.Text.Json" Version="6.0.5" />
  </ItemGroup>

</Project>

And we like to keep our libraries up to date! 6.0.5 is the latest stable version of System.Text.Json.

This library is targeting frameworks net6.0 and netstandard2.0 to increase compatibility, so it's .csproj actually looks more like:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFrameworks>net6.0;netstandard2.0</TargetFrameworks>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="System.Text.Json" Version="6.0.5" />
  </ItemGroup>

</Project>

jamesjoplin commented 2 years ago

Hello!

I've seen similar behavior when leveraging an extension method in the System.Net.Http.Json namespace, specifically HttpResponseMessage.Content.ReadFromJsonAsync<T>()

I've created an empty new project here with reproduction steps : https://github.com/jamesjoplin/lambda-json-missing-method Manually attempting to add System.Net.Http.Json or the System.Text.Json assemblies doesn't resolve the issue.

When deploying to AWS Lambda, the function works appropriately. If it's preferred to open a new issue with the appropriate template, more than happy to do so!

ashishdhingra commented 1 week ago

@nobowned Good afternoon. The above issue appears to be fixed using the latest Empty Lambda Function blueprint targeting .NET 8.0, System.Text.Json version 8.0.5 (current stable version) and latest AWS .NET 8.0 Mock Lambda Test Tool (0.15.2). Below is the function code along with .csprojfile: .csproj

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <GenerateRuntimeConfigurationFiles>true</GenerateRuntimeConfigurationFiles>
    <AWSProjectType>Lambda</AWSProjectType>
    <!-- This property makes the build directory similar to a publish directory and helps the AWS .NET Lambda Mock Test Tool find project dependencies. -->
    <CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies>
    <!-- Generate ready to run images during publishing to improve cold start time. -->
    <PublishReadyToRun>true</PublishReadyToRun>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Amazon.Lambda.Core" Version="2.3.0" />
    <PackageReference Include="Amazon.Lambda.S3Events" Version="3.1.0" />
    <PackageReference Include="Amazon.Lambda.Serialization.SystemTextJson" Version="2.4.4" />
    <PackageReference Include="System.Text.Json" Version="8.0.5" />
  </ItemGroup>
</Project>

Function.cs

using Amazon.Lambda.Core;
using Amazon.Lambda.S3Events;

// Assembly attribute to enable the Lambda function's JSON input to be converted into a .NET class.
[assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.SystemTextJson.DefaultLambdaJsonSerializer))]

namespace LambdaTest_Issue1264
{
    public class Function
    {

        /// <summary>
        /// A simple function that takes a string and does a ToUpper
        /// </summary>
        /// <param name="input">The event for the Lambda function handler to process.</param>
        /// <param name="context">The ILambdaContext that provides methods for logging and describing the Lambda environment.</param>
        /// <returns></returns>
        public void FunctionHandler(S3Event input, ILambdaContext context)
        {
            System.Net.Http.Json.JsonContent.Create("test", options: new System.Text.Json.JsonSerializerOptions());
        }
    }
}

Please verify the same at your end and confirm if this issue could be closed.

Thanks, Ashish

github-actions[bot] commented 3 days ago

This issue has not received a response in 5 days. If you want to keep this issue open, please just leave a comment below and auto-close will be canceled.

nobowned commented 3 days ago

@ashishdhingra System.Text.Json 8.0.5 works System.Text.Json 9.0.0 Missing Method Exception Why is this? I'm not a fan of having to pin a very specific System.Text.Json version in order to use a tool.

For reference: I can switch between 8.0.5 and 9.0.0 in a Console Application or ASP.NET Application and there are no issues. So it's not that 9.0.0 is missing a method, it's that this tool is interfering.

aws / aws-lambda-dotnet