dotnet lambda deploy-serverless skipping some files

Description

When we use the dotnet lambda deploy-serverless command some files are skipped, the same files each time but we cannot figure out why, HTML files to be percise.

Reproduction Steps

This is the command we are using dotnet lambda deploy-serverless lambda-*** --function-name lambda-*** --region us-east-1 --function-handler ***.LambdaEntryPoint::FunctionHandlerAsync --configuration Release --s3-bucket lambda-***-0bfde9b7b1656b786 --function-role lambda-***-7856d99d6d0e4 --function-memory-size 512 --function-timeout 30 --config-file aws-lambda-tools-defaults.json -t serverless.template -tp "***"="***";"***"="***";"***"="***";"***"="";"***"="***"

Logs

We can see the file being skipped during the "Zipping" phase

Environment

Build Version: Amazon Lambda Tools for .NET Core applications (5.0.0)
OS Info: Windows Server 2019 std
Build Environment: terminal dotnet
Targeted .NET Platform: .net core 3.1

Resolution

This seems to be working under version 4.0.0, maybe a downgrade?

This is a :bug: bug-report

Hi @zlatkolakisic,

Good morning.

Could you please try adding <CopyToOutputDirectory> for each HTML file in your .csproj file (example below):

<Project Sdk="Microsoft.NET.Sdk.Web">
  <PropertyGroup>
    <TargetFramework>netcoreapp3.1</TargetFramework>
    <GenerateRuntimeConfigurationFiles>true</GenerateRuntimeConfigurationFiles>
    <AWSProjectType>Lambda</AWSProjectType>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="AWSSDK.S3" Version="3.3.110.62" />
    <PackageReference Include="AWSSDK.Extensions.NETCore.Setup" Version="3.3.100.1" />
    <PackageReference Include="Amazon.Lambda.AspNetCoreServer" Version="5.1.1" />
  </ItemGroup>
  <ItemGroup>
    <None Update="test.html">
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
    </None>
  </ItemGroup>
</Project>

Here is the output of my dotnet lambda package command (on MacOS terminal) and you could see the test.html included in the ZIP file:

Amazon Lambda Tools for .NET Core applications (5.0.0)
Project Home: https://github.com/aws/aws-extensions-for-dotnet-cli, https://github.com/aws/aws-lambda-dotnet

Executing publish command
... invoking 'dotnet publish', working folder '/Users/<user>/dev/repros/DeployServerless_Issue782/AWSServerlessAPI/src/AWSServerlessAPI/bin/Release/netcoreapp3.1/publish'
... dotnet publish --output "/Users/<user>/dev/repros/DeployServerless_Issue782/AWSServerlessAPI/src/AWSServerlessAPI/bin/Release/netcoreapp3.1/publish" --configuration "Release" --framework "netcoreapp3.1" /p:GenerateRuntimeConfigurationFiles=true --runtime linux-x64 --self-contained false 
... publish: Microsoft (R) Build Engine version 16.8.0+126527ff1 for .NET
... publish: Copyright (C) Microsoft Corporation. All rights reserved.
... publish:   Determining projects to restore...
... publish:   Restored /Users/<user>/dev/repros/DeployServerless_Issue782/AWSServerlessAPI/src/AWSServerlessAPI/AWSServerlessAPI.csproj (in 271 ms).
... publish:   AWSServerlessAPI -> /Users/<user>/dev/repros/DeployServerless_Issue782/AWSServerlessAPI/src/AWSServerlessAPI/bin/Release/netcoreapp3.1/linux-x64/AWSServerlessAPI.dll
... publish:   AWSServerlessAPI -> /Users/<user>/dev/repros/DeployServerless_Issue782/AWSServerlessAPI/src/AWSServerlessAPI/bin/Release/netcoreapp3.1/publish/
Changed permissions on published file (chmod +rx Amazon.Lambda.APIGatewayEvents.dll).
Changed permissions on published file (chmod +rx AWSServerlessAPI.deps.json).
Changed permissions on published file (chmod +rx AWSSDK.Core.dll).
Changed permissions on published file (chmod +rx AWSServerlessAPI.pdb).
Changed permissions on published file (chmod +rx Amazon.Lambda.Logging.AspNetCore.dll).
Changed permissions on published file (chmod +rx AWSSDK.S3.dll).
Changed permissions on published file (chmod +rx web.config).
Changed permissions on published file (chmod +rx appsettings.json).
Changed permissions on published file (chmod +rx AWSServerlessAPI.dll).
Changed permissions on published file (chmod +rx AWSServerlessAPI).
Changed permissions on published file (chmod +rx Amazon.Lambda.ApplicationLoadBalancerEvents.dll).
Changed permissions on published file (chmod +rx AWSServerlessAPI.runtimeconfig.json).
Changed permissions on published file (chmod +rx Amazon.Lambda.Serialization.SystemTextJson.dll).
Changed permissions on published file (chmod +rx aws-lambda-tools-defaults.json).
Changed permissions on published file (chmod +rx test.html).
Changed permissions on published file (chmod +rx AWSSDK.Extensions.NETCore.Setup.dll).
Changed permissions on published file (chmod +rx Amazon.Lambda.Core.dll).
Changed permissions on published file (chmod +rx appsettings.Development.json).
Changed permissions on published file (chmod +rx Amazon.Lambda.AspNetCoreServer.dll).
Zipping publish folder /Users/<user>/dev/repros/DeployServerless_Issue782/AWSServerlessAPI/src/AWSServerlessAPI/bin/Release/netcoreapp3.1/publish to /Users/<user>/dev/repros/DeployServerless_Issue782/AWSServerlessAPI/src/AWSServerlessAPI/bin/Release/netcoreapp3.1/AWSServerlessAPI.zip
... zipping:   adding: Amazon.Lambda.APIGatewayEvents.dll (deflated 55%)
... zipping:   adding: AWSServerlessAPI.deps.json (deflated 91%)
... zipping:   adding: AWSSDK.Core.dll (deflated 71%)
... zipping:   adding: AWSServerlessAPI.pdb (deflated 48%)
... zipping:   adding: Amazon.Lambda.Logging.AspNetCore.dll (deflated 48%)
... zipping:   adding: AWSSDK.S3.dll (deflated 66%)
... zipping:   adding: web.config (deflated 37%)
... zipping:   adding: appsettings.json (deflated 27%)
... zipping:   adding: AWSServerlessAPI.dll (deflated 55%)
... zipping:   adding: AWSServerlessAPI (deflated 61%)
... zipping:   adding: Amazon.Lambda.ApplicationLoadBalancerEvents.dll (deflated 47%)
... zipping:   adding: AWSServerlessAPI.runtimeconfig.json (deflated 36%)
... zipping:   adding: Amazon.Lambda.Serialization.SystemTextJson.dll (deflated 46%)
... zipping:   adding: aws-lambda-tools-defaults.json (deflated 45%)
... zipping:   adding: test.html (deflated 28%)
... zipping:   adding: AWSSDK.Extensions.NETCore.Setup.dll (deflated 55%)
... zipping:   adding: Amazon.Lambda.Core.dll (deflated 46%)
... zipping:   adding: appsettings.Development.json (deflated 13%)
... zipping:   adding: Amazon.Lambda.AspNetCoreServer.dll (deflated 58%)
Created publish archive (/Users/<user>/dev/repros/DeployServerless_Issue782/AWSServerlessAPI/src/AWSServerlessAPI/bin/Release/netcoreapp3.1/AWSServerlessAPI.zip).
Lambda project successfully packaged: /Users/<user>/dev/repros/DeployServerless_Issue782/AWSServerlessAPI/src/AWSServerlessAPI/bin/Release/netcoreapp3.1/AWSServerlessAPI.zip

The dotnet lambda deploy-serverless command first invokes dotnet publish, ZIPs the published directory and then executes rest of the logic. Before making the changes, you could examine if the HTML files were included in the publish folder (in your case HTML files should be absent), then make the changes to .csproj (preferably include HTML files Copy to output directory via Visual Studio) and then re-execute the dotnet lambda deploy-serverless command.

Hope this helps. Please let me know if it works and thereafter we could close this issue.

Thanks, Ashish

This did not help, I added the whole wwwroot folder to be copied. The file is there in the publish directory in the bin folder but the zipping mechanism skips it and some other files, any way to debug what is going on here? I am at the end of my rope.

Hi @zlatkolakisic,

I re-tested the solution using the following steps:

Manually created wwwroot folder and added an HTML file.
Did not add <CopyToOutputDirectory> element in the .csproj file.
Ran the dotnet lambda package command from project folder.

Output:

Amazon Lambda Tools for .NET Core applications (5.0.0)
Project Home: https://github.com/aws/aws-extensions-for-dotnet-cli, https://github.com/aws/aws-lambda-dotnet

Executing publish command
... invoking 'dotnet publish', working folder '/Users/<user>/dev/repros/DeployServerless_Issue782/AWSServerlessAPI/src/AWSServerlessAPI/bin/Release/netcoreapp3.1/publish'
... dotnet publish --output "/Users/<user>/dev/repros/DeployServerless_Issue782/AWSServerlessAPI/src/AWSServerlessAPI/bin/Release/netcoreapp3.1/publish" --configuration "Release" --framework "netcoreapp3.1" /p:GenerateRuntimeConfigurationFiles=true --runtime linux-x64 --self-contained false 
... publish: Microsoft (R) Build Engine version 16.8.0+126527ff1 for .NET
... publish: Copyright (C) Microsoft Corporation. All rights reserved.
... publish:   Determining projects to restore...
... publish:   Restored /Users/<user>/dev/repros/DeployServerless_Issue782/AWSServerlessAPI/src/AWSServerlessAPI/AWSServerlessAPI.csproj (in 336 ms).
... publish:   AWSServerlessAPI -> /Users/<user>/dev/repros/DeployServerless_Issue782/AWSServerlessAPI/src/AWSServerlessAPI/bin/Release/netcoreapp3.1/linux-x64/AWSServerlessAPI.dll
... publish:   AWSServerlessAPI -> /Users/<user>/dev/repros/DeployServerless_Issue782/AWSServerlessAPI/src/AWSServerlessAPI/bin/Release/netcoreapp3.1/publish/
Changed permissions on published file (chmod +rx Amazon.Lambda.APIGatewayEvents.dll).
Changed permissions on published file (chmod +rx AWSServerlessAPI.deps.json).
Changed permissions on published file (chmod +rx AWSSDK.Core.dll).
Changed permissions on published file (chmod +rx AWSServerlessAPI.pdb).
Changed permissions on published file (chmod +rx Amazon.Lambda.Logging.AspNetCore.dll).
Changed permissions on published file (chmod +rx AWSSDK.S3.dll).
Changed permissions on published file (chmod +rx web.config).
Changed permissions on published file (chmod +rx appsettings.json).
Changed permissions on published file (chmod +rx AWSServerlessAPI.dll).
Changed permissions on published file (chmod +rx AWSServerlessAPI).
Changed permissions on published file (chmod +rx Amazon.Lambda.ApplicationLoadBalancerEvents.dll).
Changed permissions on published file (chmod +rx AWSServerlessAPI.runtimeconfig.json).
Changed permissions on published file (chmod +rx Amazon.Lambda.Serialization.SystemTextJson.dll).
Changed permissions on published file (chmod +rx aws-lambda-tools-defaults.json).
Changed permissions on published file (chmod +rx test.html).
Changed permissions on published file (chmod +rx AWSSDK.Extensions.NETCore.Setup.dll).
Changed permissions on published file (chmod +rx Amazon.Lambda.Core.dll).
Changed permissions on published file (chmod +rx appsettings.Development.json).
Changed permissions on published file (chmod +rx Amazon.Lambda.AspNetCoreServer.dll).
Zipping publish folder /Users/<user>/dev/repros/DeployServerless_Issue782/AWSServerlessAPI/src/AWSServerlessAPI/bin/Release/netcoreapp3.1/publish to /Users/<user>/dev/repros/DeployServerless_Issue782/AWSServerlessAPI/src/AWSServerlessAPI/bin/Release/netcoreapp3.1/AWSServerlessAPI.zip
... zipping:   adding: Amazon.Lambda.APIGatewayEvents.dll (deflated 55%)
... zipping:   adding: AWSServerlessAPI.deps.json (deflated 91%)
... zipping:   adding: AWSSDK.Core.dll (deflated 71%)
... zipping:   adding: AWSServerlessAPI.pdb (deflated 48%)
... zipping:   adding: Amazon.Lambda.Logging.AspNetCore.dll (deflated 48%)
... zipping:   adding: AWSSDK.S3.dll (deflated 66%)
... zipping:   adding: web.config (deflated 37%)
... zipping:   adding: appsettings.json (deflated 27%)
... zipping:   adding: AWSServerlessAPI.dll (deflated 55%)
... zipping:   adding: AWSServerlessAPI (deflated 61%)
... zipping:   adding: Amazon.Lambda.ApplicationLoadBalancerEvents.dll (deflated 47%)
... zipping:   adding: AWSServerlessAPI.runtimeconfig.json (deflated 36%)
... zipping:   adding: Amazon.Lambda.Serialization.SystemTextJson.dll (deflated 46%)
... zipping:   adding: aws-lambda-tools-defaults.json (deflated 45%)
... zipping:   adding: test.html (deflated 28%)
... zipping:   adding: AWSSDK.Extensions.NETCore.Setup.dll (deflated 55%)
... zipping:   adding: Amazon.Lambda.Core.dll (deflated 46%)
... zipping:   adding: appsettings.Development.json (deflated 13%)
... zipping:   adding: Amazon.Lambda.AspNetCoreServer.dll (deflated 58%)
... zipping:   adding: wwwroot/test_www.html (deflated 28%)
Created publish archive (/Users/<user>/dev/repros/DeployServerless_Issue782/AWSServerlessAPI/src/AWSServerlessAPI/bin/Release/netcoreapp3.1/AWSServerlessAPI.zip).
Lambda project successfully packaged: /Users/<user>/dev/repros/DeployServerless_Issue782/AWSServerlessAPI/src/AWSServerlessAPI/bin/Release/netcoreapp3.1/AWSServerlessAPI.zip

As you could see from the above output, the file wwwroot/test_www.html is included in the ZIP file by the dotnet lambda package command. Hence, I'm unable to reproduce the issue.

Could you please try/confirm the following:

Was this working in the previous version of AWS Lambda Tools?
Could you try running the dotnet publish command from the project folder and examine if the wwwroot folder along with the HTML files is copied to publish folder? Kindly let us know the result.
Since you already mentioned that the HTML file is copied to publish directory, could you clean the solution and delete the bin (and publish directory). The try to execute the dotnet lambda package command.
Share your code solution (preferably as a ZIP file)

Thanks, Ashish

Hi @ashishdhingra

Thanks for the prompt response. It turns out this is working fine on one machine and not on another, they are both identical (copies of the same VM) server 2019 machines running the same version of visual studio and now the same version of the cli tool.

The dotnet publish command works exactly as expected and outputs all the files just fine.

The wwwroot folder contains some 1800 files.

it seems that the zipping process of this tool is skipping some files, could there be some depth or character path length limitation to it (no, the paths in question don't exceed the windows limitation of 250 characters)

I cannot share this solution as it is not open source, only the project file

netcoreapp3.1 true Lambda 2.7 Debug;Release;Release_Publish_win_x64;Release_Publish_linux_x64;Release_Publish_osx_x64 0.81.39 **** **** 2020 **** 0.png PreserveNewest True

So, I have found a workaround to this, but its far from ideal and its a patch to our current cicd process for deploying our product to lambda, right before we run the dotnet lambda deploy-serverless function we run dotnet lambda package, create a package and then run the following commands to supplement the missing files

ren ${zipFilePath_original} ${zipFilePath_new} "C:\Program Files\7-Zip\7z.exe" a "${zipFilePath_new}" .\bin\Release\netcoreapp3.1\publish\wwwroot ren ${zipFilePath_new} ${zipFilePath_original}

In essence, we take the name of the package that is generated, create a new name with a .zip postfix and rename the package to the zip postfix name. then using 7zip cli we add the published wwwroot folder to the package and rename the file back to the original package name

Then we run the dotnet lambda deploy-serverless command with the -pac option and it all deploys great.

This is a temporary stopgap measure we put in but the zipping functionality of the tool still has the issue.

Hi @zlatkolakisic,

Good afternoon.

I tried the following on Windows environment:

Generated 1800 HTML files in the wwwroot folder using the following script on Windows command line:

for /l %x in (1, 1, 1800) do (
>filename_%x.html (
echo ^<!DOCTYPE html^>
echo ^<html^>
echo ^<head^>
echo   ^<meta charset="utf-8"^>
echo   ^<title^>Image Text^</title^>
echo ^</head^>
echo ^<body^>
echo ^<canvas id="myCanvas" width="2479" height="3508"^>^</canvas^>
echo ^<script^>
echo    var canvas = document.getElementById('myCanvas'^);
echo    var context = canvas.getContext('2d'^);
echo    var imageObj = new Image(^);
echo    imageObj.onload = function(^) {
echo      context.drawImage(imageObj, 0, 0^);
echo      context.font = "40px Bahnschrift";
echo      context.fillStyle = "red";
echo      context.fillText("%year%", %loc%^);
echo    }
echo    imageObj.src = 'Certi.jpg';
echo ^</script^>
echo ^</body^>
echo ^</html^>
)
)

Executed the dotnet lambda package command using Amazon Lambda Tools for .NET Core applications (5.0.0)

It successfully included all the 1800 HTML files from wwwroot:


Amazon Lambda Tools for .NET Core applications (5.0.0)
Project Home: https://github.com/aws/aws-extensions-for-dotnet-cli, https://github.com/aws/aws-lambda-dotnet

Executing publish command Deleted previous publish folder ... invoking 'dotnet publish', working folder 'C:\Users\\source\repos\DeployServerless_Issue782\AWSServerlessAPI\src\AWSServerlessAPI\bin\Release\netcoreapp3.1\publish' ... dotnet publish --output "C:\Users\\source\repos\DeployServerless_Issue782\AWSServerlessAPI\src\AWSServerlessAPI\bin\Release\netcoreapp3.1\publish" --configuration "Release" --framework "netcoreapp3.1" /p:GenerateRuntimeConfigurationFiles=true --runtime linux-x64 --self-contained false ... publish: Microsoft (R) Build Engine version 16.8.0+126527ff1 for .NET ... publish: Copyright (C) Microsoft Corporation. All rights reserved. ... publish: Determining projects to restore... ... publish: Restored C:\Users\\source\repos\DeployServerless_Issue782\AWSServerlessAPI\src\AWSServerlessAPI\AWSServerlessAPI.csproj (in 521 ms). ... publish: AWSServerlessAPI -> C:\Users\\source\repos\DeployServerless_Issue782\AWSServerlessAPI\src\AWSServerlessAPI\bin\Release\netcoreapp3.1\linux-x64\AWSServerlessAPI.dll ... publish: AWSServerlessAPI -> C:\Users\\source\repos\DeployServerless_Issue782\AWSServerlessAPI\src\AWSServerlessAPI\bin\Release\netcoreapp3.1\publish\ Zipping publish folder C:\Users\\source\repos\DeployServerless_Issue782\AWSServerlessAPI\src\AWSServerlessAPI\bin\Release\netcoreapp3.1\publish to C:\Users\\source\repos\DeployServerless_Issue782\AWSServerlessAPI\src\AWSServerlessAPI\bin\Release\netcoreapp3.1\AWSServerlessAPI.zip ... zipping: Amazon.Lambda.APIGatewayEvents.dll ... zipping: Amazon.Lambda.ApplicationLoadBalancerEvents.dll ... zipping: Amazon.Lambda.AspNetCoreServer.dll ... zipping: Amazon.Lambda.Core.dll ... zipping: Amazon.Lambda.Logging.AspNetCore.dll ... zipping: Amazon.Lambda.Serialization.SystemTextJson.dll ... zipping: appsettings.Development.json ... zipping: appsettings.json ... zipping: aws-lambda-tools-defaults.json ... zipping: AWSSDK.Core.dll ... zipping: AWSSDK.Extensions.NETCore.Setup.dll ... zipping: AWSSDK.S3.dll ... zipping: AWSServerlessAPI ... zipping: AWSServerlessAPI.deps.json ... zipping: AWSServerlessAPI.dll ... zipping: AWSServerlessAPI.pdb ... zipping: AWSServerlessAPI.runtimeconfig.json ... zipping: test.html ... zipping: web.config ... zipping: wwwroot/filename_1.html ... zipping: wwwroot/filename_10.html ... zipping: wwwroot/filename_100.html ... zipping: wwwroot/filename_1000.html ... zipping: wwwroot/filename_1001.html ... zipping: wwwroot/filename_1002.html ... zipping: wwwroot/filename_1003.html ... zipping: wwwroot/filename_1004.html ... zipping: wwwroot/filename_1005.html ... zipping: wwwroot/filename_1006.html ... zipping: wwwroot/filename_1007.html ... zipping: wwwroot/filename_1008.html ... zipping: wwwroot/filename_1009.html ... zipping: wwwroot/filename_101.html ... ... ... zipping: wwwroot/filename_99.html ... zipping: wwwroot/filename_990.html ... zipping: wwwroot/filename_991.html ... zipping: wwwroot/filename_992.html ... zipping: wwwroot/filename_993.html ... zipping: wwwroot/filename_994.html ... zipping: wwwroot/filename_995.html ... zipping: wwwroot/filename_996.html ... zipping: wwwroot/filename_997.html ... zipping: wwwroot/filename_998.html ... zipping: wwwroot/filename_999.html Created publish archive (C:\Users\\source\repos\DeployServerless_Issue782\AWSServerlessAPI\src\AWSServerlessAPI\bin\Release\netcoreapp3.1\AWSServerlessAPI.zip). Lambda project successfully packaged: C:\Users\\source\repos\DeployServerless_Issue782\AWSServerlessAPI\src\AWSServerlessAPI\bin\Release\netcoreapp3.1\AWSServerlessAPI.zip



The issue is not reproducible. Did you try it on some other system? What is the average size of your HTML file?

Not sure if it is the environmental issue and what could be done here to troubleshoot the issue. 

Thanks,
Ashish

This issue has not recieved a response in 2 weeks. If you want to keep this issue open, please just leave a comment below and auto-close will be canceled.

Hi @ashishdhingra

We have tried but could not reproduce your success, stand by for more debug information if we can provide it.

Zlatko

Hi @ashishdhingra

We have tried but could not reproduce your success, stand by for more debug information if we can provide it.

Zlatko

Hi @zlatkolakisic,

Please advise if you tried it on some other system. Also try upgrading to latest version of Lambda tools. Also try clearing your %TEMP% directory.

Thanks, Ashish

This issue has not recieved a response in 2 weeks. If you want to keep this issue open, please just leave a comment below and auto-close will be canceled.

aws / aws-lambda-dotnet