search

aws / aws-sdk-cpp

AWS SDK for C++
Apache License 2.0
1.98k stars 1.06k forks source link

Seg fault in call to InitAPI #342

Closed csimmons0 closed 7 years ago

csimmons0 commented 8 years ago

As soon as my code calls InitAPI, it hits a seg fault. This is my code, which just tries to call InitAPI and ShutdownAPI with default options:

    std::shared_ptr<LoginExecutive>
    LoginExecutive::create(
        const LoginExecutiveContext & context
        )
    {
        Aws::SDKOptions options;
        Aws::InitAPI(options);
        Aws::ShutdownAPI(options);
        return std::make_shared<LoginExecutiveImpl>(context);
    }

This is the stack trace of the crash:

(lldb) bt
* thread #1: tid = 378, 0xf76c0d33 libc.so`pthread_mutex_lock + 35, name = 'doclily.doclily', stop reason = signal SIGSEGV: invalid address (fault address: 0x4)
    frame #0: 0xf76c0d33 libc.so`pthread_mutex_lock + 35
    frame #1: 0xe1a5cbb1 libc++_shared.so`std::__ndk1::mutex::lock() + 33
    frame #2: 0xe1624af8 libaws-cpp-sdk-core.so`Aws::Utils::Crypto::OpenSSL::locking_fn(int, int, char const*, int) + 88
    frame #3: 0xe17af9ba libaws-cpp-sdk-core.so`CRYPTO_lock + 266
    frame #4: 0xe1805735 libaws-cpp-sdk-core.so`___lldb_unnamed_symbol1057$$libaws-cpp-sdk-core.so + 101
    frame #5: 0xe1805944 libaws-cpp-sdk-core.so`___lldb_unnamed_symbol1059$$libaws-cpp-sdk-core.so + 68
    frame #6: 0xe1804db8 libaws-cpp-sdk-core.so`ERR_func_error_string + 120
    frame #7: 0xe17b4bd6 libaws-cpp-sdk-core.so`ERR_load_CRYPTO_strings + 38
    frame #8: 0xe16248fd libaws-cpp-sdk-core.so`Aws::Utils::Crypto::OpenSSL::init_static_state() + 29
    frame #9: 0xe16747b2 libaws-cpp-sdk-core.so`void std::__ndk1::__invoke_void_return_wrapper<void>::__call<void (*&)()>(void (*&&&)()) + 50
    frame #10: 0xe1674695 libaws-cpp-sdk-core.so`std::__ndk1::__function::__func<void (*)(), std::__ndk1::allocator<void (*)()>, void ()>::operator()() + 53
    frame #11: 0xe1611b19 libaws-cpp-sdk-core.so`std::__ndk1::function<void ()>::operator()() const + 41
    frame #12: 0xe1689876 libaws-cpp-sdk-core.so`Aws::Utils::GetTheLights::EnterRoom(std::__ndk1::function<void ()>&&) + 294
    frame #13: 0xe1672e2b libaws-cpp-sdk-core.so`DefaultMD5Factory::InitStaticState() + 107
    frame #14: 0xe166ecd1 libaws-cpp-sdk-core.so`Aws::Utils::Crypto::InitCrypto() + 705
    frame #15: 0xe15a765f libaws-cpp-sdk-core.so`Aws::InitAPI(Aws::SDKOptions const&) + 1231
  * frame #16: 0xe3f9013e libnative-lib.so`helloworld::LoginExecutive::create(context=0xfff92ec8) + 110 at LoginExecutiveImpl.cpp:18
    frame #17: 0xe3f8a618 libnative-lib.so`::Java_com_doclily_doclily_LoginExecutive_create(jniEnv=0xf434e2b0, (null)=0xfff92f0c, j_context=0xfff92f10) + 104 at NativeLoginExecutive.cpp:28
    frame #18: 0xe30633e7
JonathanHenson commented 8 years ago

Does your code initialize OpenSSL or libcurl anywhere else?

Sent from my iPhone

On Nov 18, 2016, at 2:28 AM, Chris Simmons notifications@github.com wrote:

As soon as my code calls InitAPI, it hits a seg fault. This is my code, which just tries to call InitAPI and ShutdownAPI with default options:

std::shared_ptr<LoginExecutive>
LoginExecutive::create(
    const LoginExecutiveContext & context
    )
{
    Aws::SDKOptions options;
    Aws::InitAPI(options);
    Aws::ShutdownAPI(options);
    return std::make_shared<LoginExecutiveImpl>(context);
}

This is the stack trace of the crash:

(lldb) bt

  • thread #1: tid = 378, 0xf76c0d33 libc.sopthread_mutex_lock + 35, name = 'doclily.doclily', stop reason = signal SIGSEGV: invalid address (fault address: 0x4) frame #0: 0xf76c0d33 libc.sopthread_mutex_lock + 35 frame #1: 0xe1a5cbb1 libc++_shared.sostd::__ndk1::mutex::lock() + 33 frame #2: 0xe1624af8 libaws-cpp-sdk-core.soAws::Utils::Crypto::OpenSSL::lockingfn(int, int, char const, int) + 88 frame #3: 0xe17af9ba libaws-cpp-sdk-core.soCRYPTO_lock + 266 frame #4: 0xe1805735 libaws-cpp-sdk-core.so_lldb_unnamedsymbol1057$$libaws-cpp-sdk-core.so + 101 frame #5: 0xe1805944 libaws-cpp-sdk-core.so`lldb_unnamed_symbol1059$$libaws-cpp-sdk-core.so + 68 frame #6: 0xe1804db8 libaws-cpp-sdk-core.soERR_func_error_string + 120 frame #7: 0xe17b4bd6 libaws-cpp-sdk-core.soERR_load_CRYPTO_strings + 38 frame #8: 0xe16248fd libaws-cpp-sdk-core.soAws::Utils::Crypto::OpenSSL::init_static_state() + 29 frame #9: 0xe16747b2 libaws-cpp-sdk-core.sovoid std::ndk1::invoke_void_return_wrapper::call<void (&)()>(void (&&&)()) + 50 frame #10: 0xe1674695 libaws-cpp-sdk-core.so`std::ndk1::function::func<void (_)(), std::ndk1::allocator<void (*)()>, void ()>::operator()() + 53 frame #11: 0xe1611b19 libaws-cpp-sdk-core.so`std::ndk1::function<void ()>::operator()() const + 41 frame #12: 0xe1689876 libaws-cpp-sdk-core.soAws::Utils::GetTheLights::EnterRoom(std::__ndk1::function<void ()>&&) + 294 frame #13: 0xe1672e2b libaws-cpp-sdk-core.soDefaultMD5Factory::InitStaticState() + 107 frame #14: 0xe166ecd1 libaws-cpp-sdk-core.soAws::Utils::Crypto::InitCrypto() + 705 frame #15: 0xe15a765f libaws-cpp-sdk-core.soAws::InitAPI(Aws::SDKOptions const&) + 1231
    • frame #16: 0xe3f9013e libnative-lib.sohelloworld::LoginExecutive::create(context=0xfff92ec8) + 110 at LoginExecutiveImpl.cpp:18 frame #17: 0xe3f8a618 libnative-lib.so::Java_com_doclily_doclily_LoginExecutive_create(jniEnv=0xf434e2b0, (null)=0xfff92f0c, j_context=0xfff92f10) + 104 at NativeLoginExecutive.cpp:28 frame #18: 0xe30633e7 — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.
csimmons0 commented 8 years ago

Not to my knowledge. Should it?

csimmons0 commented 8 years ago

Actually, I now see that my code is getting called twice, which means that those libraries are getting initialized elsewhere, in the first call to InitAPI, which does succeed. I didn't expect my code to get called twice; there may be a bug on my end.

bretambrose commented 8 years ago

Would you mind sharing your build, execution, and debug (for the android app itself) setup? Android debugging is something I've struggled with (pretty much log-only so far) and the fact that you're sitting in LLDB has me very jealous.

JonathanHenson commented 8 years ago

No, if you are, you have to tell the SDK not to via the SDKOptions.

Sent from my iPhone

On Nov 18, 2016, at 9:04 AM, Chris Simmons notifications@github.com wrote:

Actually, I now see that my code is getting called twice, which means that those libraries are getting initialized elsewhere, in the first call to InitAPI, which does succeed. I didn't expect my code to get called twice; there may be a bug on my end.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

csimmons0 commented 8 years ago

I now see that the documentation for ShutdownAPI states, "Do not call any other SDK methods after calling ShutdownAPI." I'm calling InitAPI after a call to ShutdownAPI; I think this is my bug.

@bretambrose I'm not sure exactly what information you're looking for regarding my development setup, but here's some info: My OS is OS X El Capitan. Android Studio 2.2.2 and the default tools it uses (Gradle, LLDB) handle most things related to build, execution, and debug. I'm running the app on a Genymotion 2.8.0 emulator.

JonathanHenson commented 8 years ago

It's complicated. As long as you can guarantee that all memory and modules are deleted before calling shutdownAPI, it should be fine. The other day, I actually accidentally had a segfault in my own code due to this.

In general non-trivial statics and long lived memory on the heap is your enemy, and will cause results like this.

Sent from my iPhone

On Nov 18, 2016, at 9:39 AM, Chris Simmons notifications@github.com wrote:

I now see that the documentation for ShutdownAPI states, "Do not call any other SDK methods after calling ShutdownAPI." I'm calling InitAPI after a call to ShutdownAPI; I think this is my bug.

@bretambrose I'm not sure exactly what information you're looking for regarding my development setup, but here's some info: My OS is OS X El Capitan. Android Studio 2.2.2 and the default tools it uses (Gradle, LLDB) handle most things related to build, execution, and debug. I'm running the app on a Genymotion 2.8.0 emulator.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

pchen66 commented 7 years ago

I have a similar situation but only happen in a specific context. I'm trying to hook up S3 uploading feature in Unreal Engine 4 but segmentation fault always comes to me when executing Aws::InitAPI call. What confuses me is that the exact same code works on Unreal Engine 4 for Mac and Linux (using aws-doc-sdk-examples) except for Unreal Engine 4 for Linux. Also, when I compiled for -DNO_ENCRYPTION=ON, the unit test for sdk fails at AWSAuthV4SignerTest. Is there something you can point me that I should look into? Thanks in advance. @JonathanHenson

My configurations:

gdb output with backtrace

Thread 1 "UE4Editor" received signal SIGSEGV, Segmentation fault.
0x00000000029ccea0 in ?? ()
(gdb) 
(gdb) bt
#0  0x00000000029ccea0 in ?? ()
#1  0x00007fff8468e39d in std::function<std::shared_ptr<Aws::Utils::Crypto::SymmetricCipherFactory> ()>::operator()() const ()
   from /usr/local/lib/libaws-cpp-sdk-core.so
#2  0x00007fff8468de04 in Aws::InitAPI(Aws::SDKOptions const&) ()
   from /usr/local/lib/libaws-cpp-sdk-core.so
#3  0x00007fff849c6edd in UAWSClient::UploadFileToS3 (FileName=..., 
    FileAbsolutePath=...)
    at /home/ray/YSProjects/TestBed/Source/TestBed/Private/AWSClient.cpp:25
#4  0x00007fff849ca169 in UAWSClient::execUploadFileToS3 (
    this=<optimized out>, Stack=..., Z_Param__Result=<optimized out>)
    at ../../../YSProjects/TestBed/Source/TestBed/Public/AWSClient.h:19
#5  0x00007ffff66ecf22 in UFunction::Invoke (this=<optimized out>, 
    Obj=<optimized out>, Stack=..., Z_Param__Result=<optimized out>)
    at /home/ray/UnrealEngine/Engine/Source/Runtime/CoreUObject/Private/UObject/Class.cpp:4474
#6  0x00007ffff6842237 in UObject::CallFunction (this=0x7fff89218fc0, 
    Stack=..., Z_Param__Result=0x7fffffffa830, Function=<optimized out>)
    at /home/ray/UnrealEngine/Engine/Source/Runtime/CoreUObject/Private/UObject/ScriptCore.cpp:700
#7  0x00007ffff684b3f5 in FFrame::Step (this=0x7fffffffa950, 
    Z_Param__Result=0x7fffffffa830, Context=<optimized out>)
    at /home/ray/UnrealEngine/Engine/Source/Runtime/CoreUObject/Private/UObject/---Typ------Type <return> to continu-----Type <r---Type <return> to continue, or q <return> to quit---
JonathanHenson commented 7 years ago

NO_ENCRYPTION means you are supplying your own crypto code via the factories in Aws::CryptoOptions and we will not attempt to compile the platform defaults (this is largely for playstation and xbox or other embedded platforms where they may want to do hardware level crypto). If libcrypto, commoncrypto or msft cng, are on the platform you are building against, leave that option on unless you have a very special use case. If you do turn it off, you are responsible for providing your own implementation via the CryptoOptions.

Sent from my iPhone

On Jan 29, 2017, at 11:03 AM, Ray Chen notifications@github.com wrote:

I have a similar situation but only happen in a specific context. I'm trying to hook up S3 uploading feature in Unreal Engine 4 but segmentation fault always comes to me when executing Aws::InitAPI call. What confuses me is that the exact same code works on Unreal Engine 4 for Mac and Linux (using aws-doc-sdk-examples) except for Unreal Engine 4 for Linux. Also, when I compiled for -DNO_ENCRYPTION=ON, the unit test for sdk fails at AWSAuthV4SignerTest. Is there something you can point me that I should look into? Thanks in advance. @JonathanHenson

My configurations:

Ubuntu 16.04 LTS libssl-dev: 1.0.2g-1ubuntu4.5 Unreal Engine 4.14 gdb output with backtrace

Thread 1 "UE4Editor" received signal SIGSEGV, Segmentation fault. 0x00000000029ccea0 in ?? () (gdb) (gdb) bt

0 0x00000000029ccea0 in ?? ()

1 0x00007fff8468e39d in std::function<std::shared_ptr ()>::operator()() const ()

from /usr/local/lib/libaws-cpp-sdk-core.so

2 0x00007fff8468de04 in Aws::InitAPI(Aws::SDKOptions const&) ()

from /usr/local/lib/libaws-cpp-sdk-core.so

3 0x00007fff849c6edd in UAWSClient::UploadFileToS3 (FileName=...,

FileAbsolutePath=...)
at /home/ray/YSProjects/TestBed/Source/TestBed/Private/AWSClient.cpp:25

4 0x00007fff849ca169 in UAWSClient::execUploadFileToS3 (

this=<optimized out>, Stack=..., Z_Param__Result=<optimized out>)
at ../../../YSProjects/TestBed/Source/TestBed/Public/AWSClient.h:19

5 0x00007ffff66ecf22 in UFunction::Invoke (this=,

Obj=<optimized out>, Stack=..., Z_Param__Result=<optimized out>)
at /home/ray/UnrealEngine/Engine/Source/Runtime/CoreUObject/Private/UObject/Class.cpp:4474

6 0x00007ffff6842237 in UObject::CallFunction (this=0x7fff89218fc0,

Stack=..., Z_Param__Result=0x7fffffffa830, Function=<optimized out>)
at /home/ray/UnrealEngine/Engine/Source/Runtime/CoreUObject/Private/UObject/ScriptCore.cpp:700

7 0x00007ffff684b3f5 in FFrame::Step (this=0x7fffffffa950,

Z_Param__Result=0x7fffffffa830, Context=<optimized out>)
at /home/ray/UnrealEngine/Engine/Source/Runtime/CoreUObject/Private/UObject/---Typ------Type <return> to continu-----Type <r---Type <return> to continue, or q <return> to quit---

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

JonathanHenson commented 7 years ago

as for your segfault, is anywhere else in your application using libcurl or openssl?

Sent from my iPhone

On Jan 29, 2017, at 11:03 AM, Ray Chen notifications@github.com wrote:

I have a similar situation but only happen in a specific context. I'm trying to hook up S3 uploading feature in Unreal Engine 4 but segmentation fault always comes to me when executing Aws::InitAPI call. What confuses me is that the exact same code works on Unreal Engine 4 for Mac and Linux (using aws-doc-sdk-examples) except for Unreal Engine 4 for Linux. Also, when I compiled for -DNO_ENCRYPTION=ON, the unit test for sdk fails at AWSAuthV4SignerTest. Is there something you can point me that I should look into? Thanks in advance. @JonathanHenson

My configurations:

Ubuntu 16.04 LTS libssl-dev: 1.0.2g-1ubuntu4.5 Unreal Engine 4.14 gdb output with backtrace

Thread 1 "UE4Editor" received signal SIGSEGV, Segmentation fault. 0x00000000029ccea0 in ?? () (gdb) (gdb) bt

0 0x00000000029ccea0 in ?? ()

1 0x00007fff8468e39d in std::function<std::shared_ptr ()>::operator()() const ()

from /usr/local/lib/libaws-cpp-sdk-core.so

2 0x00007fff8468de04 in Aws::InitAPI(Aws::SDKOptions const&) ()

from /usr/local/lib/libaws-cpp-sdk-core.so

3 0x00007fff849c6edd in UAWSClient::UploadFileToS3 (FileName=...,

FileAbsolutePath=...)
at /home/ray/YSProjects/TestBed/Source/TestBed/Private/AWSClient.cpp:25

4 0x00007fff849ca169 in UAWSClient::execUploadFileToS3 (

this=<optimized out>, Stack=..., Z_Param__Result=<optimized out>)
at ../../../YSProjects/TestBed/Source/TestBed/Public/AWSClient.h:19

5 0x00007ffff66ecf22 in UFunction::Invoke (this=,

Obj=<optimized out>, Stack=..., Z_Param__Result=<optimized out>)
at /home/ray/UnrealEngine/Engine/Source/Runtime/CoreUObject/Private/UObject/Class.cpp:4474

6 0x00007ffff6842237 in UObject::CallFunction (this=0x7fff89218fc0,

Stack=..., Z_Param__Result=0x7fffffffa830, Function=<optimized out>)
at /home/ray/UnrealEngine/Engine/Source/Runtime/CoreUObject/Private/UObject/ScriptCore.cpp:700

7 0x00007ffff684b3f5 in FFrame::Step (this=0x7fffffffa950,

Z_Param__Result=0x7fffffffa830, Context=<optimized out>)
at /home/ray/UnrealEngine/Engine/Source/Runtime/CoreUObject/Private/UObject/---Typ------Type <return> to continu-----Type <r---Type <return> to continue, or q <return> to quit---

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

pchen66 commented 7 years ago

Not that I'm aware of. The AWS part is the only place from my application that uses libcurl and openssl.

JonathanHenson commented 7 years ago

looks like you are segfaulting in initialization because you have not built the crypto layer. Is this running on a build that has NO_ENCRYPTION turned on? If so, that's your problem.

Sent from my iPhone

On Jan 29, 2017, at 11:36 AM, Ray Chen notifications@github.com wrote:

Not that I'm aware of. The AWS part is the only place that needs libcurl and openssl.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

pchen66 commented 7 years ago

Sorry I should have clarify that. The segfault wasn't resulting from enabling NO_ENCRYPTION. I was using cmake -DBUILD_ONLY="s3" <path-to-sdk-source> across platforms, which works for all the other except Unreal Linux. NO_ENCRYPTION is what I experiment to see if that's related.

JonathanHenson commented 7 years ago

any chance you can upload some valgrind output?

Sent from my iPhone

On Jan 29, 2017, at 12:40 PM, Ray Chen notifications@github.com wrote:

Sorry I should have clarify that. The segfault wasn't resulting from enabling NO_ENCRYPTION. I was using cmake -DBUILD_ONLY="s3" across platforms, which works for all the other except Unreal Linux. NO_ENCRYPTION is what I experiment to see if that's related.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

pchen66 commented 7 years ago

Sure, I haven't used valgrind before but I'll give it a try. Unrelated question, sometime when I make some tests failed. Do you know why is that? credential is sitting in /.aws/ and never changed.

----------] Global test environment tear-down
[==========] 198 tests from 31 test cases ran. (1143 ms total)
[  PASSED  ] 195 tests.
[  FAILED  ] 3 tests, listed below:
[  FAILED  ] ProfileConfigFileAWSCredentialsProviderTest.TestDefaultConfig
[  FAILED  ] EnvironmentModifyingTest.ProfileConfigTestWithEnvVars
[  FAILED  ] EnvironmentModifyingTest.ProfileConfigTestWithEnvVarsButSpecifiedProfile
pchen66 commented 7 years ago

This valgrind output captures as soon as the Aws::InitAPI executes. Thanks for checking @JonathanHenson. I'm more than happy to provide any additional information you need.

~ valgrind ./UE4Editor "test.uproject"
...
==31245== Thread 1:
==31245== Conditional jump or move depends on uninitialised value(s)
==31245==    at 0x621CDC6: UScriptStruct::CompareScriptStruct(void const*, void const*, unsigned int) const (Class.cpp:2238)
==31245==    by 0x621CDC3: UScriptStruct::CompareScriptStruct(void const*, void const*, unsigned int) const (Class.cpp:2238)
==31245==    by 0x621CDC3: UScriptStruct::CompareScriptStruct(void const*, void const*, unsigned int) const (Class.cpp:2238)
==31245==    by 0x621CDC3: UScriptStruct::CompareScriptStruct(void const*, void const*, unsigned int) const (Class.cpp:2238)
==31245==    by 0x62F4CEF: UArrayProperty::Identical(void const*, void const*, unsigned int) const (PropertyArray.cpp:48)
==31245==    by 0x621483A: UStruct::SerializeTaggedProperties(FArchive&, unsigned char*, UStruct*, unsigned char*, UObject const*) const (Class.cpp:1129)
==31245==    by 0x62B5A76: UObject::SerializeScriptProperties(FArchive&) const (Obj.cpp:1130)
==31245==    by 0x62ABFDD: UObject::Serialize(FArchive&) (Obj.cpp:1060)
==31245==    by 0x7A0E429: AActor::Serialize(FArchive&) (Actor.cpp:538)
==31245==    by 0x8BC251E: AWorldSettings::Serialize(FArchive&) (WorldSettings.cpp:201)
==31245==    by 0x63A5D12: StaticDuplicateObjectEx(FObjectDuplicationParameters&) (UObjectGlobals.cpp:2028)
==31245==    by 0x63A5497: StaticDuplicateObject(UObject const*, UObject*, FName, EObjectFlags, UClass*, EDuplicateMode::Type, EInternalObjectFlags) (UObjectGlobals.cpp:1935)
==31245== 
==31245== Conditional jump or move depends on uninitialised value(s)
==31245==    at 0x71AEFD92: Aws::InitAPI(Aws::SDKOptions const&) (in /usr/local/lib/libaws-cpp-sdk-core.so)
==31245==    by 0x71724EDC: UAWSClient::UploadFileToS3(FString const&, FString const&) (AWSClient.cpp:25)
==31245==    by 0x71728168: UAWSClient::execUploadFileToS3(FFrame&, void*) (AWSClient.h:19)
==31245==    by 0x6225F21: UFunction::Invoke(UObject*, FFrame&, void*) (Class.cpp:4474)
==31245==    by 0x637B236: UObject::CallFunction(FFrame&, void*, UFunction*) (ScriptCore.cpp:700)
==31245==    by 0x63843F4: Step (ScriptCore.cpp:265)
==31245==    by 0x63843F4: UObject::ProcessContextOpcode(FFrame&, void*, bool) (ScriptCore.cpp:2136)
==31245==    by 0x637C088: Step (ScriptCore.cpp:265)
==31245==    by 0x637C088: UObject::ProcessInternal(FFrame&, void*) (ScriptCore.cpp:952)
==31245==    by 0x637AD12: UObject::CallFunction(FFrame&, void*, UFunction*) (ScriptCore.cpp:821)
==31245==    by 0x637C088: Step (ScriptCore.cpp:265)
==31245==    by 0x637C088: UObject::ProcessInternal(FFrame&, void*) (ScriptCore.cpp:952)
==31245==    by 0x6225F21: UFunction::Invoke(UObject*, FFrame&, void*) (Class.cpp:4474)
==31245==    by 0x637F29C: UObject::ProcessEvent(UFunction*, void*) (ScriptCore.cpp:1301)
==31245==    by 0x7A22ED8: AActor::BeginPlay() (Actor.cpp:3046)
==31245== 
==31245== Conditional jump or move depends on uninitialised value(s)
==31245==    at 0x71AF037C: std::function<std::shared_ptr<Aws::Utils::Crypto::SymmetricCipherFactory> ()>::operator()() const (in /usr/local/lib/libaws-cpp-sdk-core.so)
==31245==    by 0x71AEFDB4: Aws::InitAPI(Aws::SDKOptions const&) (in /usr/local/lib/libaws-cpp-sdk-core.so)
==31245==    by 0x71724EDC: UAWSClient::UploadFileToS3(FString const&, FString const&) (AWSClient.cpp:25)
==31245==    by 0x71728168: UAWSClient::execUploadFileToS3(FFrame&, void*) (AWSClient.h:19)
==31245==    by 0x6225F21: UFunction::Invoke(UObject*, FFrame&, void*) (Class.cpp:4474)
==31245==    by 0x637B236: UObject::CallFunction(FFrame&, void*, UFunction*) (ScriptCore.cpp:700)
==31245==    by 0x63843F4: Step (ScriptCore.cpp:265)
==31245==    by 0x63843F4: UObject::ProcessContextOpcode(FFrame&, void*, bool) (ScriptCore.cpp:2136)
==31245==    by 0x637C088: Step (ScriptCore.cpp:265)
==31245==    by 0x637C088: UObject::ProcessInternal(FFrame&, void*) (ScriptCore.cpp:952)
==31245==    by 0x637AD12: UObject::CallFunction(FFrame&, void*, UFunction*) (ScriptCore.cpp:821)
==31245==    by 0x637C088: Step (ScriptCore.cpp:265)
==31245==    by 0x637C088: UObject::ProcessInternal(FFrame&, void*) (ScriptCore.cpp:952)
==31245==    by 0x6225F21: UFunction::Invoke(UObject*, FFrame&, void*) (Class.cpp:4474)
==31245==    by 0x637F29C: UObject::ProcessEvent(UFunction*, void*) (ScriptCore.cpp:1301)
==31245== 
==31245== Use of uninitialised value of size 8
==31245==    at 0x71AF039B: std::function<std::shared_ptr<Aws::Utils::Crypto::SymmetricCipherFactory> ()>::operator()() const (in /usr/local/lib/libaws-cpp-sdk-core.so)
==31245==    by 0x71AEFDB4: Aws::InitAPI(Aws::SDKOptions const&) (in /usr/local/lib/libaws-cpp-sdk-core.so)
==31245==    by 0x71724EDC: UAWSClient::UploadFileToS3(FString const&, FString const&) (AWSClient.cpp:25)
==31245==    by 0x71728168: UAWSClient::execUploadFileToS3(FFrame&, void*) (AWSClient.h:19)
==31245==    by 0x6225F21: UFunction::Invoke(UObject*, FFrame&, void*) (Class.cpp:4474)
==31245==    by 0x637B236: UObject::CallFunction(FFrame&, void*, UFunction*) (ScriptCore.cpp:700)
==31245==    by 0x63843F4: Step (ScriptCore.cpp:265)
==31245==    by 0x63843F4: UObject::ProcessContextOpcode(FFrame&, void*, bool) (ScriptCore.cpp:2136)
==31245==    by 0x637C088: Step (ScriptCore.cpp:265)
==31245==    by 0x637C088: UObject::ProcessInternal(FFrame&, void*) (ScriptCore.cpp:952)
==31245==    by 0x637AD12: UObject::CallFunction(FFrame&, void*, UFunction*) (ScriptCore.cpp:821)
==31245==    by 0x637C088: Step (ScriptCore.cpp:265)
==31245==    by 0x637C088: UObject::ProcessInternal(FFrame&, void*) (ScriptCore.cpp:952)
==31245==    by 0x6225F21: UFunction::Invoke(UObject*, FFrame&, void*) (Class.cpp:4474)
==31245==    by 0x637F29C: UObject::ProcessEvent(UFunction*, void*) (ScriptCore.cpp:1301)
==31245== 
==31245== Jump to the invalid address stated on the next line
==31245==    at 0x0: ???
==31245==    by 0x71AEFDB4: Aws::InitAPI(Aws::SDKOptions const&) (in /usr/local/lib/libaws-cpp-sdk-core.so)
==31245==    by 0x71724EDC: UAWSClient::UploadFileToS3(FString const&, FString const&) (AWSClient.cpp:25)
==31245==    by 0x71728168: UAWSClient::execUploadFileToS3(FFrame&, void*) (AWSClient.h:19)
==31245==    by 0x6225F21: UFunction::Invoke(UObject*, FFrame&, void*) (Class.cpp:4474)
==31245==    by 0x637B236: UObject::CallFunction(FFrame&, void*, UFunction*) (ScriptCore.cpp:700)
==31245==    by 0x63843F4: Step (ScriptCore.cpp:265)
==31245==    by 0x63843F4: UObject::ProcessContextOpcode(FFrame&, void*, bool) (ScriptCore.cpp:2136)
==31245==    by 0x637C088: Step (ScriptCore.cpp:265)
==31245==    by 0x637C088: UObject::ProcessInternal(FFrame&, void*) (ScriptCore.cpp:952)
==31245==    by 0x637AD12: UObject::CallFunction(FFrame&, void*, UFunction*) (ScriptCore.cpp:821)
==31245==    by 0x637C088: Step (ScriptCore.cpp:265)
==31245==    by 0x637C088: UObject::ProcessInternal(FFrame&, void*) (ScriptCore.cpp:952)
==31245==    by 0x6225F21: UFunction::Invoke(UObject*, FFrame&, void*) (Class.cpp:4474)
==31245==    by 0x637F29C: UObject::ProcessEvent(UFunction*, void*) (ScriptCore.cpp:1301)
==31245==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==31245== 
Signal 11 caught.
Malloc Size=131076 LargeMemoryPoolOffset=131092 
Malloc Size=65535 LargeMemoryPoolOffset=196655 
Malloc Size=112917 LargeMemoryPoolOffset=309589 
==31245== 
==31245== Process terminating with default action of signal 11 (SIGSEGV): dumping core
==31245==    at 0x4E4A269: raise (pt-raise.c:35)
==31245==    by 0x53ECF2E: FLinuxCrashContext::GenerateCrashInfoAndLaunchReporter(bool) const (LinuxPlatformCrashContext.cpp:523)
==31245==    by 0x53F00B4: PlatformCrashHandler(int, siginfo_t*, void*) (LinuxPlatformCrashContext.cpp:576)
==31245==    by 0x4E4A38F: ??? (in /lib/x86_64-linux-gnu/libpthread-2.23.so)
==31245== 
==31245== HEAP SUMMARY:
==31245==     in use at exit: 378,178,593 bytes in 66,590 blocks
==31245==   total heap usage: 390,248 allocs, 323,658 frees, 8,387,603,552 bytes allocated
==31245== 
==31245== LEAK SUMMARY:
==31245==    definitely lost: 86,528 bytes in 689 blocks
==31245==    indirectly lost: 599,464 bytes in 5,344 blocks
==31245==      possibly lost: 332,731,704 bytes in 2,782 blocks
==31245==    still reachable: 44,760,897 bytes in 57,775 blocks
==31245==                       of which reachable via heuristic:
==31245==                         stdstring          : 192 bytes in 6 blocks
==31245==                         newarray           : 16 bytes in 1 blocks
==31245==                         multipleinheritance: 104 bytes in 1 blocks
==31245==         suppressed: 0 bytes in 0 blocks
==31245== Rerun with --leak-check=full to see details of leaked memory
==31245== 
==31245== For counts of detected and suppressed errors, rerun with: -v
==31245== Use --track-origins=yes to see where uninitialised values come from
==31245== ERROR SUMMARY: 11074 errors from 121 contexts (suppressed: 0 from 0)
Segmentation fault (core dumped)
JonathanHenson commented 7 years ago

This should not be possible without you explicitly setting one of the callback functions in CryptoOptions, could you upload the full text of UAWSClient::UploadfileToS3 so I can have a look?

pchen66 commented 7 years ago

Sure

static const Aws::String BUCKET = "BUCKET_NAME";
static const Aws::String AWS_ACCESS_KEY_ID = "ACCESS_KEY_ID ";
static const Aws::String AWS_SECRET_ACCESS_KEY = "SECRET_ACCESS_KEY";
static const std::string DEFAULT_FILE_PATH = "PATH";

void UAWSClient::UploadFileToS3(const FString& FileName, const FString& FileAbsolutePath)
{
    const Aws::String key_name = TCHAR_TO_UTF8(*FileName);
    std::string filePATH = DEFAULT_FILE_PATH;

    if (!FileAbsolutePath.IsEmpty())
    {
        filePATH = TCHAR_TO_UTF8(*FileAbsolutePath);
    }

    Aws::SDKOptions options;

    // Where segfault happens
    Aws::InitAPI(options);

    Aws::Client::ClientConfiguration clientConfig;
    clientConfig.region = Aws::Region::AP_NORTHEAST_1;
    clientConfig.connectTimeoutMs = 30000;
    clientConfig.requestTimeoutMs = 600000;

    auto s3_client = Aws::MakeShared<Aws::S3::S3Client>("S3Client", Aws::Auth::AWSCredentials(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY), clientConfig);

    Aws::S3::Model::PutObjectRequest object_request;
    object_request.WithBucket(BUCKET).WithKey(key_name);

    std::shared_ptr<Aws::FStream> fileToUpload = Aws::MakeShared<Aws::FStream>("FileToUpload", filePATH, std::ios::in | std::ios::binary);
    object_request.SetBody(fileToUpload);
    object_request.SetContentType("image/jpeg"); // "text/plain"

    auto put_object_outcome = s3_client->PutObject(object_request);

    if(put_object_outcome.IsSuccess()) {
        UE_LOG(LogTemp, Warning, TEXT("S3 Uploaded Successfully"));
    }
    else {
        UE_LOG(LogTemp, Warning, TEXT("S3 Uploaded Failed"));
    }

    Aws::ShutdownAPI(options);
}
JonathanHenson commented 7 years ago

key_name is going to have its allocator swapped out from under it, when you call InitAPI(). You shouldn't use any thing from one of our header files before you call InitAPI().

Also InitAPI needs to go way way up the chain inside

int main(char** argv, int argv) { Aws::InitAPI() //initialize the rest of the game and the engine. //those components will presumably do AWS things. Aws::CleanupAPI() }

Also, as written, this will segfault at ShutdownAPI anyways, because ShutdownAPI will be called before clientConfig, s3_client, and the rest gets cleaned up. You would need scoping brackets to make that work.

JonathanHenson commented 7 years ago

For reference you could have a look at the AWS_Lmbr components in the Lumberyard Engine.

adennie commented 7 years ago

Just ran into this exact symptom (same stack as the OP), on the 2nd call to InitAPI. Is it OK to call InitAPI after ShutdownAPI, or not? That is, as long as every InitAPI has a corresponding ShutdownAPI that is called before the next @InitAPI, should that work? FWIW, it seems to work fine on Windows, but not linux.

JonathanHenson commented 7 years ago

in theory, it would work, but you should not do it. Put InitAPI in the entry point of your application as early as possible. a.) Its expensive. b.) it manipulates global state. c.) on linux it alters state of openssl and libcurl. d.) Nothing guarantees the compiler does not change the ordering of your calls. e.) That will not work in a multi-threaded scenario.

Sent from my iPhone

On Jan 30, 2017, at 5:36 PM, Andy Dennie notifications@github.com wrote:

Just ran into this exact symptom (same stack as the OP), on the 2nd call to InitializeAPI. Is it OK to call InitializeAPI after ShutdownAPI, or not? That is, as long as every InitializeAPI has a corresponding ShutdownAPI that is called before the next InitializeAPI, should that work?

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub, or mute the thread.

JonathanHenson commented 7 years ago

also, that will shutdown the logging thread. It will likely not start back up on the next call to InitAPI()

Sent from my iPhone

On Jan 30, 2017, at 5:36 PM, Andy Dennie notifications@github.com wrote:

Just ran into this exact symptom (same stack as the OP), on the 2nd call to InitializeAPI. Is it OK to call InitializeAPI after ShutdownAPI, or not? That is, as long as every InitializeAPI has a corresponding ShutdownAPI that is called before the next InitializeAPI, should that work?

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub, or mute the thread.

pchen66 commented 7 years ago

It turns out that the segfault is resulting from std::string for my case. Because UE4 is using libc++ for the engine while Linux uses libstdc++ by default. After switching to libc++ it's all working fine.

bretambrose commented 7 years ago

Closing

salunkhesagar1386 commented 5 years ago

No, if you are, you have to tell the SDK not to via the SDKOptions.

Sent from my iPhone

On Nov 18, 2016, at 9:04 AM, Chris Simmons notifications@github.com wrote: Actually, I now see that my code is getting called twice, which means that those libraries are getting initialized elsewhere, in the first call to InitAPI, which does succeed. I didn't expect my code to get called twice; there may be a bug on my end. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

Hey @JonathanHenson : You mentioned in one of your post 'you have to tell the SDK not to via the SDKOptions.'. My application is getting crashed while InitAPI is getting called on Android. I also use licurl internally for other REST calls. How to tell AWS SDK not to initialize the libcurl again?? BTW the same code works fine on Windows.