sbiscigl
commented
6 months ago Expected Behavior S3Crt library works as expected in the "provided.al2023" environment.
I tried reproducing the issue with the following set up and could not.
Dockerfile
FROM public.ecr.aws/lambda/provided:al2023
#install deps
RUN dnf install -y git cmake gcc-c++ libcurl-devel zlib-devel openssl-devel
# Clone repo
RUN git clone --depth 1 --recurse-submodules https://github.com/aws/aws-sdk-cpp && \
cd aws-sdk-cpp && \
mkdir build && \
cd build && \
cmake -DAUTORUN_UNIT_TESTS=OFF -DBUILD_ONLY="s3-crt" .. && \
cmake --build . && \
cmake --install .
# Copy Code Over \
RUN mkdir sdk-example
COPY CMakeLists.txt sdk-example/CMakeLists.txt
COPY main.cpp sdk-example/main.cpp
RUN cd sdk-example && \
mkdir build && \
cd build && \
cmake .. -DUSE_CRT=ON && \
cmake --build .
ENTRYPOINT [ "./sdk-example/build/sdk_usage_workspace" ]main.cpp
#include <aws/core/Aws.h>
#ifdef USE_CRT
#include <aws/s3-crt/S3CrtClient.h>
#include <aws/s3-crt/model/PutObjectRequest.h>
#include <aws/s3-crt/model/GetObjectRequest.h>
#else
#include <aws/s3/S3Client.h>
#include <aws/s3/model/PutObjectRequest.h>
#include <aws/s3/model/GetObjectRequest.h>
#endif
using namespace Aws;
using namespace Aws::Utils;
#ifdef USE_CRT
using namespace Aws::S3Crt;
using namespace Aws::S3Crt::Model;
#else
using namespace Aws::S3;
using namespace Aws::S3::Model;
#endif
const char* ALLOCATION_TAG = "sure";
const char* BUCKET_NAME = "yourbucketname";
auto main() -> int {
SDKOptions options;
options.loggingOptions.logLevel = Logging::LogLevel::Trace;
InitAPI(options); {
#ifdef USE_CRT
const auto client = MakeUnique<S3CrtClient>(ALLOCATION_TAG);
#else
const auto client = MakeUnique<S3Client>(ALLOCATION_TAG);
#endif
auto putRequest = PutObjectRequest().WithBucket(BUCKET_NAME).WithKey("some-test-key");
std::shared_ptr<IOStream> body = Aws::MakeShared<StringStream>(ALLOCATION_TAG,
"test string",
std::ios_base::in | std::ios_base::binary);
putRequest.SetBody(body);
const auto putResponse = client->PutObject(putRequest);
assert(putResponse.IsSuccess());
auto getRequest = GetObjectRequest().WithBucket(BUCKET_NAME).WithKey("some-test-key");
const auto getResponse = client->GetObject(getRequest);
assert(getResponse.IsSuccess());
std::cout << getResponse.GetResult().GetBody().rdbuf();
}
ShutdownAPI(options);
return 0;
}CMakeLists.txt
cmake_minimum_required(VERSION 3.13)
project(sdk_usage_workspace)
set(CMAKE_CXX_STANDARD 20)
find_package(AWSSDK REQUIRED COMPONENTS s3-crt)
if (USE_CRT)
add_definitions("-DUSE_CRT")
endif ()
add_executable(${PROJECT_NAME} "main.cpp")
target_link_libraries(${PROJECT_NAME} ${AWSSDK_LINK_LIBRARIES})when i build with
docker build -t basic-use-image .and i run with
docker run \
-e AWS_ACCESS_KEY_ID=... \
-e AWS_SECRET_ACCESS_KEY=... \
-e AWS_SESSION_TOKEN=... \
--name basic-use-image basic-use-image /sdk-example/build/sdk_usage_workspaceI see the output "test string". So this uses the provided:al2023 docker images which according to docs is what image is being used.
Since we cannot reproduce in the image, could you possibly try to update this example to replicate what you are seeing?
pr0g
github-actions[bot]
DmitriyMusatkin
Describe the bug
From an OS only environment in Lambda, using the
S3CrtClientwith the newprovided.al2023runtime, and callingGetObjecton the client, results in an internal failure:Calling code looks roughly like this:
This code is running as a native Lambda function. The infrastructure is provisioned by Terraform and previously was using
"provided"(instead of"provided.al2023"). With"provided"things worked as expected. It seems like the problem is potentially an issue to do withaws-c-commonas the enum reported comes from here.As a quick test I switched out
S3Crtwith the normalS3client libraries and things worked exactly as expected. This is a workaround for now but it definitely seems something is going wrong inside the S3Crt library.(see https://aws.amazon.com/blogs/compute/introducing-the-amazon-linux-2023-runtime-for-aws-lambda/ for more context).
Expected Behavior
S3Crt library works as expected in the
"provided.al2023"environment.Current Behavior
S3Crt suffers an internal failure.
Reproduction Steps
See description above, it should be possible to repro using the above environment (OS only environment with Amazon Linux 2023) and the S3Crt example here. What we're doing is very similar.
Possible Solution
Use the S3 client instead of S3Crt (not sure what the performance implications are at this stage though).
Additional Information/Context
No response
AWS CPP SDK version used
1.11.255 (from vcpkg)
Compiler and Version used
gcc
Operating System and version
Amazon Linux 2023