search

firebase / firebase-cpp-sdk

Firebase C++ SDK
http://firebase.google.com
Apache License 2.0
271 stars 111 forks source link

App crashes when using Storage from a Firestore completion handler #356

Open mark-grimes opened 3 years ago

mark-grimes commented 3 years ago

[REQUIRED] Please fill in the following fields:

[REQUIRED] Please describe the question here:

Briefly

Is it safe to call other firebase methods from a future OnCompletion handler? That is, start a second (or more) asynchronous task when the first completes?

In detail

This is potentially a regression, but could be a misuse of the API on our part. We've been using Firebase 6.15.1 successfully for a while and decided it was past time to upgrade to 7.1.1. We have a design where configuration is stored in Firestore, which tells the app which files are required from Cloud Storage. So for one call to our internal function we need a call to Firestore followed by a call to Storage. Simplified code would be something like:

void getCorrectFile( const char* localPath, std::function<void(const std::string& errorMessage)> callback )
{
    pFirestore->Collection("configStuff").Get().OnCompletion(
        [callback=std::move(callback)]( const ::firebase::Future<firebase::firestore::QuerySnapshot>& collectionFuture )
        {
            if( collectionFuture.error() != firebase::firestore::Error::kErrorOk )
            {
                callback( "An error occurred (real code maps errors to error_codes)" );
            }
            else
            {
                // ...code to work out correct file from the QuerySnapshot...
                firebase::storage::StorageReference fileRef = pStorage->GetReference(firebasePath);

                fileRef.GetFile(localPath).OnCompletion( [callback=std::move(callback),fileRef](const firebase::Future<size_t>& getFileFuture)
                {
                    if( getFileFuture.error() != 0 ) callback( getFileFuture.error_message() );
                    else callback( "No error (real code would create default constructed error_code)" );
                } );
            }
        } );
}

As I say, this all worked fine in Firebase 6.15.1. After upgrading to 7.1.1 we now get various crashes[*]. Before investigating I thought it was worth checking we're not fundamentally abusing the thread model in the API, and it just happened to work previously.

[*] How it crashes is fairly inconsistent. It can be an assert from some mutex.h file, or an unhandled *** -[NSFileManager createDirectoryAtURL:withIntermediateDirectories:attributes:error:]: URL is nil exception. I've even seen it work once.

google-oss-bot commented 3 years ago

I found a few problems with this issue:

dconeybe commented 3 years ago

Hi @mark-grimes. Thank you for reporting this issue. Your code looks fine to me. Your usage of OnCompletion() is not a misuse of the thread model and looks perfectly acceptable. It is entirely possible that you've uncovered a bug. The behavior you are describing smells like a use-after-free situation (especially the crash in mutex.h).

To start the investigation, could you provide some of the stack traces from the crashes? You mentioned there are several different ones so if you could attach some of them to this issue that would be a good starting point.

At the same time, perhaps just ensure that callback, pFirestore, and pStorage are not being deleted prior to the lambdas specified to OnCompletion executing.

mark-grimes commented 3 years ago

Thanks for the response. pFirestore and pStorage have duration for the whole app, so they're fine. I'm 99% sure callback is still alive but checking that now. I wondered about the threading model because it seems callbacks can occur on any thread now, whereas they always seemed to be on the main thread in 6.15.1.

The main error is: ASSERT: /Users/runner/work/firebase-cpp-sdk/firebase-cpp-sdk/sdk-src/app/src/mutex.h(53): ret == 0 I'll see if I can replicate the others. The stack trace for that thread is below.

Frame 13 (where the assert occurs) is the closing brace of this code:

void sncore::src::firebase::detail::Firebase::copyFileToLocal( const std::string& firebasePath, const std::filesystem::path& localPath, std::function<void(std::error_code)> callback ) const
{
    // Firebase requires URIs and not paths. A path string will work on desktop but not on iOS.
    std::string localURL="file://" + localPath.string();

    ::firebase::storage::StorageReference fileRef = pImpl_->pStorage->GetReference(firebasePath);

    if( !fileRef.is_valid() ) return callback( std::error_code( CustomFirebaseError::FileRefNotValid, customFirebaseErrorCategory ) );

    fileRef.GetFile( localURL.c_str() ).OnCompletion( [callback=std::move(callback),fileRef](const ::firebase::Future<size_t>& getFileFuture) {
        if( getFileFuture.error() != 0 ) callback( std::error_code( getFileFuture.error(), cloudStorageErrorCategory ) );
        else callback( std::error_code() );
    } );
}

I'm not entirely sure why I have fileRef in the lambda capture, I think I previously used it as part of the error message. There's a lot of levels of wrapping code (anything sncore::...) so I don't know how useful this is to you. I'll try and boil down to a minimal failing example, but that's going to take a while.

* thread #9, queue = 'com.google.firebase.firestore.callback', stop reason = signal SIGABRT
    frame #0: 0x00000001ccbd4414 libsystem_kernel.dylib`__pthread_kill + 8
    frame #1: 0x00000001ea730b50 libsystem_pthread.dylib`pthread_kill + 272
    frame #2: 0x00000001a8032b74 libsystem_c.dylib`abort + 104
    frame #3: 0x0000000104fe120c MyApp`firebase::DefaultLogCallback(firebase::LogLevel, char const*, void*) + 68
    frame #4: 0x0000000104fe1320 MyApp`firebase::LogMessageWithCallbackV(firebase::LogLevel, char const*, char*) + 208
    frame #5: 0x0000000104fe1444 MyApp`firebase::LogAssert(char const*, ...) + 40
    frame #6: 0x000000010501c9d0 MyApp`firebase::Mutex::~Mutex() + 56
    frame #7: 0x000000010501a4dc MyApp`firebase::Mutex::~Mutex() + 32
    frame #8: 0x000000010501a57c MyApp`firebase::storage::internal::StorageReferenceInternal::~StorageReferenceInternal() + 72
    frame #9: 0x000000010501a5fc MyApp`firebase::storage::internal::StorageReferenceInternal::~StorageReferenceInternal() + 32
    frame #10: 0x00000001050111e4 MyApp`firebase::storage::internal::StorageReferenceInternalCommon::DeleteInternal(firebase::storage::StorageReference*) + 72
    frame #11: 0x0000000105011310 MyApp`firebase::storage::StorageReference::~StorageReference() + 32
    frame #12: 0x0000000105011340 MyApp`firebase::storage::StorageReference::~StorageReference() + 32
  * frame #13: 0x00000001053bedd8 MyApp`sncore::src::firebase::detail::Firebase::copyFileToLocal(this=0x00000002821618d8, firebasePath="ski_areas/ArapahoeBasin/1.1/003.sqlite", localPath=0x000000016b5a7040, callback=function<void (std::__1::error_code)> @ 0x000000016b5a6b68)>) const at Firebase.cpp:280:1
    frame #14: 0x00000001051f8784 MyApp`void sncore::src::FirebaseFileService<NSLogLogger>::copyMapToLocal<NSLogLogger>(this=0x0000000282161898, logger=0x0000000282c7b708, identifier=0x00000002821e9410, localPath=0x000000016b5a7040, callback=function<void (std::__1::error_code)> @ 0x000000016b5a72a8)>) at FirebaseFileService.hpp:172:15
    frame #15: 0x00000001051c9f18 MyApp`void sncore::src::Manager<sncore::src::LocalFileService, sncore::src::FirebaseFileService<NSLogLogger>, sncore::src::sqlite::SQLiteSkiAreaReader, sncore::src::JSONConfigReader>::loadSkiAreaAsync<SNCoreConfig, NSLogLogger>(this=0x0000000281a1f4d8, logger=0x0000000282c7b708, identifier=0x00000002821e9410, callback=function<void (sncore::SkiArea<SNCoreConfig> &&, std::__1::error_code)> @ 0x000000016b5a7558, localOnly=false)>, bool) at Manager.hpp:158:30
    frame #16: 0x000000010537e6b4 MyApp`-[MyApp loadSkiAreaWithIdentifier:completion:]::$_2::operator(this=0x00000002821e9408, descriptions=size=50, error=(__val_ = 0, __cat_ = 0x000000020154f258))(std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code) at MyApp.mm:286:40
    frame #17: 0x000000010537e3b0 MyApp`decltype(__f=0x00000002821e9408, __args=size=50, __args=0x000000016b5a7730)(std::__1::forward<std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> > >(fp0), std::__1::forward<std::__1::error_code>(fp0))) std::__1::__invoke<-[MyApp loadSkiAreaWithIdentifier:completion:]::$_2&, std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >, std::__1::error_code>(-[MyApp loadSkiAreaWithIdentifier:completion:]::$_2&, std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code&&) at type_traits:3545:1
    frame #18: 0x000000010537e324 MyApp`void std::__1::__invoke_void_return_wrapper<void>::__call<-[MyApp loadSkiAreaWithIdentifier:completion:]::$_2&, std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >, std::__1::error_code>(__args=0x00000002821e9408, __args=size=50, __args=0x000000016b5a7730) at __functional_base:348:9
    frame #19: 0x000000010537e2c0 MyApp`std::__1::__function::__alloc_func<-[MyApp loadSkiAreaWithIdentifier:completion:]::$_2, std::__1::allocator<-[MyApp loadSkiAreaWithIdentifier:completion:]::$_2>, void (std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code)>::operator(this=0x00000002821e9408, __arg=size=50, __arg=0x000000016b5a7730)(std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code&&) at functional:1546:16
    frame #20: 0x000000010537cbac MyApp`std::__1::__function::__func<-[MyApp loadSkiAreaWithIdentifier:completion:]::$_2, std::__1::allocator<-[MyApp loadSkiAreaWithIdentifier:completion:]::$_2>, void (std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code)>::operator(this=0x00000002821e9400, __arg=size=50, __arg=0x000000016b5a7730)(std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code&&) at functional:1720:12
    frame #21: 0x00000001051f4fac MyApp`std::__1::__function::__value_func<void (std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code)>::operator(this=0x0000000281a18ca0, __args=size=50, __args=0x000000016b5a7730)(std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code&&) const at functional:1873:16
    frame #22: 0x00000001051e25e0 MyApp`std::__1::function<void (std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code)>::operator(this= Lambda in File MyApp.mm at Line 286, __arg=size=50, __arg=(__val_ = 0, __cat_ = 0x000000020154f258))(std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code) const at functional:2548:12
    frame #23: 0x00000001051f4e38 MyApp`void sncore::src::Manager<sncore::src::LocalFileService, sncore::src::FirebaseFileService<NSLogLogger>, sncore::src::sqlite::SQLiteSkiAreaReader, sncore::src::JSONConfigReader>::listSkiAreasAsync<NSLogLogger>(this=0x0000000281a18c88, results=size=50, error=(__val_ = 0, __cat_ = 0x000000020154f258))>, bool)::'lambda'(std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code)::operator()(std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code) at Manager.hpp:307:13
    frame #24: 0x00000001051f4d78 MyApp`decltype(__f=0x0000000281a18c88, __args=size=50, __args=0x000000016b5a7940)(std::__1::forward<void sncore::src::Manager<sncore::src::LocalFileService, sncore::src::FirebaseFileService<NSLogLogger>, sncore::src::sqlite::SQLiteSkiAreaReader, sncore::src::JSONConfigReader>::listSkiAreasAsync<NSLogLogger>(NSLogLogger&, std::__1::function<void (std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code)>, bool)::'lambda'(std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code)&>(fp0)...)) std::__1::__invoke<void sncore::src::Manager<sncore::src::LocalFileService, sncore::src::FirebaseFileService<NSLogLogger>, sncore::src::sqlite::SQLiteSkiAreaReader, sncore::src::JSONConfigReader>::listSkiAreasAsync<NSLogLogger>(NSLogLogger&, std::__1::function<void (std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code)>, bool)::'lambda'(std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code)&, std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >, std::__1::error_code>(NSLogLogger&&, void sncore::src::Manager<sncore::src::LocalFileService, sncore::src::FirebaseFileService<NSLogLogger>, sncore::src::sqlite::SQLiteSkiAreaReader, sncore::src::JSONConfigReader>::listSkiAreasAsync<NSLogLogger>(NSLogLogger&, std::__1::function<void (std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code)>, bool)::'lambda'(std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code)&...) at type_traits:3545:1
    frame #25: 0x00000001051f4cec MyApp`void std::__1::__invoke_void_return_wrapper<void>::__call<void sncore::src::Manager<sncore::src::LocalFileService, sncore::src::FirebaseFileService<NSLogLogger>, sncore::src::sqlite::SQLiteSkiAreaReader, sncore::src::JSONConfigReader>::listSkiAreasAsync<NSLogLogger>(__args=0x0000000281a18c88, __args=size=50, __args=0x000000016b5a7940)>, bool)::'lambda'(std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code)&, std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >, std::__1::error_code>(NSLogLogger&&...) at __functional_base:348:9
    frame #26: 0x00000001051f4c74 MyApp`std::__1::__function::__alloc_func<void sncore::src::Manager<sncore::src::LocalFileService, sncore::src::FirebaseFileService<NSLogLogger>, sncore::src::sqlite::SQLiteSkiAreaReader, sncore::src::JSONConfigReader>::listSkiAreasAsync<NSLogLogger>(NSLogLogger&, std::__1::function<void (std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code)>, bool)::'lambda'(std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code), std::__1::allocator<void sncore::src::Manager<sncore::src::LocalFileService, sncore::src::FirebaseFileService<NSLogLogger>, sncore::src::sqlite::SQLiteSkiAreaReader, sncore::src::JSONConfigReader>::listSkiAreasAsync<NSLogLogger>(NSLogLogger&, std::__1::function<void (std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code)>, bool)::'lambda'(std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code)>, void (std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code)>::operator(this=0x0000000281a18c88, __arg=size=50, __arg=0x000000016b5a7940)(std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code&&) at functional:1546:16
    frame #27: 0x00000001051f1cc8 MyApp`std::__1::__function::__func<void sncore::src::Manager<sncore::src::LocalFileService, sncore::src::FirebaseFileService<NSLogLogger>, sncore::src::sqlite::SQLiteSkiAreaReader, sncore::src::JSONConfigReader>::listSkiAreasAsync<NSLogLogger>(NSLogLogger&, std::__1::function<void (std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code)>, bool)::'lambda'(std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code), std::__1::allocator<void sncore::src::Manager<sncore::src::LocalFileService, sncore::src::FirebaseFileService<NSLogLogger>, sncore::src::sqlite::SQLiteSkiAreaReader, sncore::src::JSONConfigReader>::listSkiAreasAsync<NSLogLogger>(NSLogLogger&, std::__1::function<void (std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code)>, bool)::'lambda'(std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code)>, void (std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code)>::operator(this=0x0000000281a18c80, __arg=size=50, __arg=0x000000016b5a7940)(std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code&&) at functional:1720:12
    frame #28: 0x00000001051f4fac MyApp`std::__1::__function::__value_func<void (std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code)>::operator(this=0x0000000283401a68, __args=size=50, __args=0x000000016b5a7940)(std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code&&) const at functional:1873:16
    frame #29: 0x00000001051e25e0 MyApp`std::__1::function<void (std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code)>::operator(this=0x0000000283401a68, __arg=size=50, __arg=(__val_ = 0, __cat_ = 0x000000020154f258))(std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code) const at functional:2548:12
    frame #30: 0x00000001051ee02c MyApp`void sncore::src::FirebaseFileService<NSLogLogger>::listAsync<NSLogLogger>(this=0x0000000283401a18)>)::ListingResults::resolve() at FirebaseFileService.hpp:227:17
    frame #31: 0x00000001051eddac MyApp`void sncore::src::FirebaseFileService<NSLogLogger>::listAsync<NSLogLogger>(this=0x0000000280b4c1a8, pQuerySnapshot=0x0000000280d7dbf0, error=(__val_ = 0, __cat_ = 0x000000020154f258))>)::'lambda'(firebase::firestore::QuerySnapshot const*, std::__1::error_code)::operator()(firebase::firestore::QuerySnapshot const*, std::__1::error_code) const at FirebaseFileService.hpp:250:30
    frame #32: 0x00000001051edc34 MyApp`decltype(__f=0x0000000280b4c1a8, __args=0x000000016b5a7db0, __args=0x000000016b5a7dc0)(std::__1::forward<void sncore::src::FirebaseFileService<NSLogLogger>::listAsync<NSLogLogger>(NSLogLogger&, std::__1::error_code&, std::__1::function<void (std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code)>)::'lambda'(firebase::firestore::QuerySnapshot const*, std::__1::error_code)&>(fp0)...)) std::__1::__invoke<void sncore::src::FirebaseFileService<NSLogLogger>::listAsync<NSLogLogger>(NSLogLogger&, std::__1::error_code&, std::__1::function<void (std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code)>)::'lambda'(firebase::firestore::QuerySnapshot const*, std::__1::error_code)&, firebase::firestore::QuerySnapshot const*, std::__1::error_code>(NSLogLogger&&, void sncore::src::FirebaseFileService<NSLogLogger>::listAsync<NSLogLogger>(NSLogLogger&, std::__1::error_code&, std::__1::function<void (std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code)>)::'lambda'(firebase::firestore::QuerySnapshot const*, std::__1::error_code)&...) at type_traits:3545:1
    frame #33: 0x00000001051edba8 MyApp`void std::__1::__invoke_void_return_wrapper<void>::__call<void sncore::src::FirebaseFileService<NSLogLogger>::listAsync<NSLogLogger>(__args=0x0000000280b4c1a8, __args=0x000000016b5a7db0, __args=0x000000016b5a7dc0)>)::'lambda'(firebase::firestore::QuerySnapshot const*, std::__1::error_code)&, firebase::firestore::QuerySnapshot const*, std::__1::error_code>(NSLogLogger&&...) at __functional_base:348:9
    frame #34: 0x00000001051edb30 MyApp`std::__1::__function::__alloc_func<void sncore::src::FirebaseFileService<NSLogLogger>::listAsync<NSLogLogger>(NSLogLogger&, std::__1::error_code&, std::__1::function<void (std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code)>)::'lambda'(firebase::firestore::QuerySnapshot const*, std::__1::error_code), std::__1::allocator<void sncore::src::FirebaseFileService<NSLogLogger>::listAsync<NSLogLogger>(NSLogLogger&, std::__1::error_code&, std::__1::function<void (std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code)>)::'lambda'(firebase::firestore::QuerySnapshot const*, std::__1::error_code)>, void (firebase::firestore::QuerySnapshot const*, std::__1::error_code)>::operator(this=0x0000000280b4c1a8, __arg=0x000000016b5a7db0, __arg=0x000000016b5a7dc0)(firebase::firestore::QuerySnapshot const*&&, std::__1::error_code&&) at functional:1546:16
    frame #35: 0x00000001051ec44c MyApp`std::__1::__function::__func<void sncore::src::FirebaseFileService<NSLogLogger>::listAsync<NSLogLogger>(NSLogLogger&, std::__1::error_code&, std::__1::function<void (std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code)>)::'lambda'(firebase::firestore::QuerySnapshot const*, std::__1::error_code), std::__1::allocator<void sncore::src::FirebaseFileService<NSLogLogger>::listAsync<NSLogLogger>(NSLogLogger&, std::__1::error_code&, std::__1::function<void (std::__1::vector<sncore::io::SkiAreaDescription, std::__1::allocator<sncore::io::SkiAreaDescription> >&&, std::__1::error_code)>)::'lambda'(firebase::firestore::QuerySnapshot const*, std::__1::error_code)>, void (firebase::firestore::QuerySnapshot const*, std::__1::error_code)>::operator(this=0x0000000280b4c1a0, __arg=0x000000016b5a7db0, __arg=0x000000016b5a7dc0)(firebase::firestore::QuerySnapshot const*&&, std::__1::error_code&&) at functional:1720:12
    frame #36: 0x00000001053c7cd0 MyApp`std::__1::__function::__value_func<void (firebase::firestore::QuerySnapshot const*, std::__1::error_code)>::operator(this=0x0000000280011ab0, __args=0x000000016b5a7db0, __args=0x000000016b5a7dc0)(firebase::firestore::QuerySnapshot const*&&, std::__1::error_code&&) const at functional:1873:16
    frame #37: 0x00000001053c7b30 MyApp`std::__1::function<void (firebase::firestore::QuerySnapshot const*, std::__1::error_code)>::operator(this=0x0000000280011ab0, __arg=0x0000000280d7dbf0, __arg=(__val_ = 0, __cat_ = 0x000000020154f258))(firebase::firestore::QuerySnapshot const*, std::__1::error_code) const at functional:2548:12
    frame #38: 0x00000001053c790c MyApp`sncore::src::firebase::detail::Firebase::getCollection(this=0x0000000280011aa8, future=0x000000016b5a8620)>)::$_1::operator()(firebase::Future<firebase::firestore::QuerySnapshot> const&) const at Firebase.cpp:224:21
    frame #39: 0x00000001053c7728 MyApp`decltype(__f=0x0000000280011aa8, __args=0x000000016b5a8620)>)::$_1&>(fp)(std::__1::forward<firebase::Future<firebase::firestore::QuerySnapshot> const&>(fp0))) std::__1::__invoke<sncore::src::firebase::detail::Firebase::getCollection(char const*, std::__1::function<void (firebase::firestore::QuerySnapshot const*, std::__1::error_code)>)::$_1&, firebase::Future<firebase::firestore::QuerySnapshot> const&>(sncore::src::firebase::detail::Firebase::getCollection(char const*, std::__1::function<void (firebase::firestore::QuerySnapshot const*, std::__1::error_code)>)::$_1&, firebase::Future<firebase::firestore::QuerySnapshot> const&) at type_traits:3545:1
    frame #40: 0x00000001053c76b8 MyApp`void std::__1::__invoke_void_return_wrapper<void>::__call<sncore::src::firebase::detail::Firebase::getCollection(__args=0x0000000280011aa8, __args=0x000000016b5a8620)>)::$_1&, firebase::Future<firebase::firestore::QuerySnapshot> const&>(sncore::src::firebase::detail::Firebase::getCollection(char const*, std::__1::function<void (firebase::firestore::QuerySnapshot const*, std::__1::error_code)>)::$_1&, firebase::Future<firebase::firestore::QuerySnapshot> const&) at __functional_base:348:9
    frame #41: 0x00000001053c7658 MyApp`std::__1::__function::__alloc_func<sncore::src::firebase::detail::Firebase::getCollection(char const*, std::__1::function<void (firebase::firestore::QuerySnapshot const*, std::__1::error_code)>)::$_1, std::__1::allocator<sncore::src::firebase::detail::Firebase::getCollection(char const*, std::__1::function<void (firebase::firestore::QuerySnapshot const*, std::__1::error_code)>)::$_1>, void (firebase::Future<firebase::firestore::QuerySnapshot> const&)>::operator(this=0x0000000280011aa8, __arg=0x000000016b5a8620)(firebase::Future<firebase::firestore::QuerySnapshot> const&) at functional:1546:16
    frame #42: 0x00000001053c5ea8 MyApp`std::__1::__function::__func<sncore::src::firebase::detail::Firebase::getCollection(char const*, std::__1::function<void (firebase::firestore::QuerySnapshot const*, std::__1::error_code)>)::$_1, std::__1::allocator<sncore::src::firebase::detail::Firebase::getCollection(char const*, std::__1::function<void (firebase::firestore::QuerySnapshot const*, std::__1::error_code)>)::$_1>, void (firebase::Future<firebase::firestore::QuerySnapshot> const&)>::operator(this=0x0000000280011aa0, __arg=0x000000016b5a8620)(firebase::Future<firebase::firestore::QuerySnapshot> const&) at functional:1720:12
    frame #43: 0x0000000104ffc980 MyApp`std::__1::__function::__value_func<void (firebase::FutureBase const&)>::operator()(firebase::FutureBase const&) const + 88
    frame #44: 0x0000000104ffc91c MyApp`std::__1::function<void (firebase::FutureBase const&)>::operator()(firebase::FutureBase const&) const + 60
    frame #45: 0x0000000104ffab9c MyApp`firebase::CallStdFunction(firebase::FutureBase const&, void*) + 48
    frame #46: 0x0000000104ff9d4c MyApp`firebase::ReferenceCountedFutureImpl::RunCallback(firebase::FutureBase*, void (*)(firebase::FutureBase const&, void*), void*) + 68
    frame #47: 0x0000000104ff9c40 MyApp`firebase::ReferenceCountedFutureImpl::ReleaseMutexAndRunCallbacks(firebase::FutureHandle const&) + 212
    frame #48: 0x0000000104fb52c0 MyApp`void firebase::ReferenceCountedFutureImpl::CompleteInternal<firebase::firestore::QuerySnapshot, void firebase::firestore::Promise<firebase::firestore::QuerySnapshot>::SetValue<firebase::firestore::QuerySnapshot, void>(firebase::firestore::QuerySnapshot)::'lambda'(firebase::firestore::QuerySnapshot*)>(firebase::FutureHandle const&, int, char const*, void const&) + 256
    frame #49: 0x0000000104fb519c MyApp`void firebase::ReferenceCountedFutureImpl::Complete<firebase::firestore::QuerySnapshot, void firebase::firestore::Promise<firebase::firestore::QuerySnapshot>::SetValue<firebase::firestore::QuerySnapshot, void>(firebase::firestore::QuerySnapshot)::'lambda'(firebase::firestore::QuerySnapshot*)>(firebase::SafeFutureHandle<firebase::firestore::QuerySnapshot>, int, char const*, void const&) + 76
    frame #50: 0x0000000104fb4f84 MyApp`void firebase::firestore::Promise<firebase::firestore::QuerySnapshot>::SetValue<firebase::firestore::QuerySnapshot, void>(firebase::firestore::QuerySnapshot) + 132
    frame #51: 0x0000000104fb4dfc MyApp`std::__1::unique_ptr<firebase::firestore::core::EventListener<firebase::firestore::api::QuerySnapshot>, std::__1::default_delete<firebase::firestore::core::EventListener<firebase::firestore::api::QuerySnapshot> > > firebase::firestore::ListenerWithPromise<firebase::firestore::api::QuerySnapshot, firebase::firestore::QuerySnapshot>(firebase::firestore::Promise<firebase::firestore::QuerySnapshot>)::'lambda'(firebase::firestore::util::StatusOr<firebase::firestore::api::QuerySnapshot>)::operator()(firebase::firestore::util::StatusOr<firebase::firestore::api::QuerySnapshot>) + 180
    frame #52: 0x0000000104fb4cfc MyApp`decltype(std::__1::forward<firebase::firestore::api::QuerySnapshot>(fp)(std::__1::forward<firebase::firestore::QuerySnapshot>(fp0)...)) std::__1::__invoke<std::__1::unique_ptr<firebase::firestore::core::EventListener<firebase::firestore::api::QuerySnapshot>, std::__1::default_delete<firebase::firestore::core::EventListener<firebase::firestore::api::QuerySnapshot> > > firebase::firestore::ListenerWithPromise<firebase::firestore::api::QuerySnapshot, firebase::firestore::QuerySnapshot>(firebase::firestore::Promise<firebase::firestore::QuerySnapshot>)::'lambda'(firebase::firestore::util::StatusOr<firebase::firestore::api::QuerySnapshot>)&, firebase::firestore::util::StatusOr<firebase::firestore::api::QuerySnapshot> >(firebase::firestore::api::QuerySnapshot&&, firebase::firestore::QuerySnapshot&&...) + 120
    frame #53: 0x0000000104fb4c50 MyApp`void std::__1::__invoke_void_return_wrapper<void>::__call<std::__1::unique_ptr<firebase::firestore::core::EventListener<firebase::firestore::api::QuerySnapshot>, std::__1::default_delete<firebase::firestore::core::EventListener<firebase::firestore::api::QuerySnapshot> > > firebase::firestore::ListenerWithPromise<firebase::firestore::api::QuerySnapshot, firebase::firestore::QuerySnapshot>(firebase::firestore::Promise<firebase::firestore::QuerySnapshot>)::'lambda'(firebase::firestore::util::StatusOr<firebase::firestore::api::QuerySnapshot>)&, firebase::firestore::util::StatusOr<firebase::firestore::api::QuerySnapshot> >(firebase::firestore::api::QuerySnapshot&&...) + 64
    frame #54: 0x0000000104fb4c04 MyApp`std::__1::__function::__alloc_func<std::__1::unique_ptr<firebase::firestore::core::EventListener<firebase::firestore::api::QuerySnapshot>, std::__1::default_delete<firebase::firestore::core::EventListener<firebase::firestore::api::QuerySnapshot> > > firebase::firestore::ListenerWithPromise<firebase::firestore::api::QuerySnapshot, firebase::firestore::QuerySnapshot>(firebase::firestore::Promise<firebase::firestore::QuerySnapshot>)::'lambda'(firebase::firestore::util::StatusOr<firebase::firestore::api::QuerySnapshot>), std::__1::allocator<std::__1::unique_ptr<firebase::firestore::core::EventListener<firebase::firestore::api::QuerySnapshot>, std::__1::default_delete<firebase::firestore::core::EventListener<firebase::firestore::api::QuerySnapshot> > > firebase::firestore::ListenerWithPromise<firebase::firestore::api::QuerySnapshot, firebase::firestore::QuerySnapshot>(firebase::firestore::Promise<firebase::firestore::QuerySnapshot>)::'lambda'(firebase::firestore::util::StatusOr<firebase::firestore::api::QuerySnapshot>)>, void (firebase::firestore::util::StatusOr<firebase::firestore::api::QuerySnapshot>)>::operator()(firebase::firestore::util::StatusOr<firebase::firestore::api::QuerySnapshot>&&) + 64
    frame #55: 0x0000000104fb343c MyApp`std::__1::__function::__func<std::__1::unique_ptr<firebase::firestore::core::EventListener<firebase::firestore::api::QuerySnapshot>, std::__1::default_delete<firebase::firestore::core::EventListener<firebase::firestore::api::QuerySnapshot> > > firebase::firestore::ListenerWithPromise<firebase::firestore::api::QuerySnapshot, firebase::firestore::QuerySnapshot>(firebase::firestore::Promise<firebase::firestore::QuerySnapshot>)::'lambda'(firebase::firestore::util::StatusOr<firebase::firestore::api::QuerySnapshot>), std::__1::allocator<std::__1::unique_ptr<firebase::firestore::core::EventListener<firebase::firestore::api::QuerySnapshot>, std::__1::default_delete<firebase::firestore::core::EventListener<firebase::firestore::api::QuerySnapshot> > > firebase::firestore::ListenerWithPromise<firebase::firestore::api::QuerySnapshot, firebase::firestore::QuerySnapshot>(firebase::firestore::Promise<firebase::firestore::QuerySnapshot>)::'lambda'(firebase::firestore::util::StatusOr<firebase::firestore::api::QuerySnapshot>)>, void (firebase::firestore::util::StatusOr<firebase::firestore::api::QuerySnapshot>)>::operator()(firebase::firestore::util::StatusOr<firebase::firestore::api::QuerySnapshot>&&) + 64
    frame #56: 0x0000000104faf764 MyApp`std::__1::__function::__value_func<void (firebase::firestore::util::StatusOr<firebase::firestore::api::QuerySnapshot>)>::operator()(firebase::firestore::util::StatusOr<firebase::firestore::api::QuerySnapshot>&&) const + 88
    frame #57: 0x0000000104faf674 MyApp`std::__1::function<void (firebase::firestore::util::StatusOr<firebase::firestore::api::QuerySnapshot>)>::operator()(firebase::firestore::util::StatusOr<firebase::firestore::api::QuerySnapshot>) const + 52
    frame #58: 0x0000000104faf43c MyApp`firebase::firestore::core::EventListener<firebase::firestore::api::QuerySnapshot>::Create(std::__1::function<void (firebase::firestore::util::StatusOr<firebase::firestore::api::QuerySnapshot>)>)::CallbackEventListener::OnEvent(firebase::firestore::util::StatusOr<firebase::firestore::api::QuerySnapshot>) + 112
    frame #59: 0x0000000107cc08ec FirebaseFirestore`firebase::firestore::api::Query::GetDocuments(this=<unavailable>, maybe_snapshot=StatusOr<firebase::firestore::api::QuerySnapshot> @ 0x000000016b5a9c88)::ListenOnce::OnEvent(firebase::firestore::util::StatusOr<firebase::firestore::api::QuerySnapshot>) at query_core.cc:152:20 [opt]
    frame #60: 0x0000000107cc116c FirebaseFirestore`firebase::firestore::api::Query::AddSnapshotListener(this=<unavailable>, maybe_snapshot=<unavailable>)::Converter::OnEvent(firebase::firestore::util::StatusOr<firebase::firestore::core::ViewSnapshot>) at query_core.cc:201:23 [opt]
    frame #61: 0x0000000107c10b44 FirebaseFirestore`void std::__1::__invoke_void_return_wrapper<void>::__call<firebase::firestore::core::AsyncEventListener<firebase::firestore::core::ViewSnapshot>::OnEvent(firebase::firestore::util::StatusOr<firebase::firestore::core::ViewSnapshot>)::'lambda'()&>(firebase::firestore::core::ViewSnapshot&&...) [inlined] firebase::firestore::core::AsyncEventListener<firebase::firestore::core::ViewSnapshot>::OnEvent(this=0x000000010cbed018)::'lambda'()::operator()() const at event_listener.h:146:31 [opt]
    frame #62: 0x0000000107c10b04 FirebaseFirestore`void std::__1::__invoke_void_return_wrapper<void>::__call<firebase::firestore::core::AsyncEventListener<firebase::firestore::core::ViewSnapshot>::OnEvent(firebase::firestore::util::StatusOr<firebase::firestore::core::ViewSnapshot>)::'lambda'()&>(firebase::firestore::core::ViewSnapshot&&...) [inlined] decltype(__f=0x000000010cbed018)(std::__1::forward<firebase::firestore::core::AsyncEventListener<firebase::firestore::core::ViewSnapshot>::OnEvent(firebase::firestore::util::StatusOr<firebase::firestore::core::ViewSnapshot>)::'lambda'()&>(fp0)...)) std::__1::__invoke<firebase::firestore::core::AsyncEventListener<firebase::firestore::core::ViewSnapshot>::OnEvent(firebase::firestore::util::StatusOr<firebase::firestore::core::ViewSnapshot>)::'lambda'()&>(firebase::firestore::core::ViewSnapshot&&, firebase::firestore::core::AsyncEventListener<firebase::firestore::core::ViewSnapshot>::OnEvent(firebase::firestore::util::StatusOr<firebase::firestore::core::ViewSnapshot>)::'lambda'()&...) at type_traits:3545 [opt]
    frame #63: 0x0000000107c10b04 FirebaseFirestore`void std::__1::__invoke_void_return_wrapper<void>::__call<firebase::firestore::core::AsyncEventListener<firebase::firestore::core::ViewSnapshot>::OnEvent(__args=0x000000010cbed018)::'lambda'()&>(firebase::firestore::core::ViewSnapshot&&...) at __functional_base:348 [opt]
    frame #64: 0x0000000107cf3ba8 FirebaseFirestore`firebase::firestore::util::Task::ExecuteAndRelease() [inlined] std::__1::__function::__value_func<void ()>::operator(this=<unavailable>)() const at functional:1873:16 [opt]
    frame #65: 0x0000000107cf3b94 FirebaseFirestore`firebase::firestore::util::Task::ExecuteAndRelease() [inlined] std::__1::function<void ()>::operator(this=<unavailable>)() const at functional:2548 [opt]
    frame #66: 0x0000000107cf3b94 FirebaseFirestore`firebase::firestore::util::Task::ExecuteAndRelease(this=0x0000000283a56640) at task.cc:102 [opt]
    frame #67: 0x000000010c81d6c0 libdispatch.dylib`_dispatch_client_callout + 20
    frame #68: 0x000000010c825354 libdispatch.dylib`_dispatch_lane_serial_drain + 736
    frame #69: 0x000000010c8260c0 libdispatch.dylib`_dispatch_lane_invoke + 448
    frame #70: 0x000000010c832644 libdispatch.dylib`_dispatch_workloop_worker_thread + 1520
    frame #71: 0x00000001ea731814 libsystem_pthread.dylib`_pthread_wqthread + 276
dconeybe commented 3 years ago

Update: I've been analyzing the stack trace (which is very helpful, thank you!) and I have more questions than answers at this point. But I wanted to reply to let you know that investigation is ongoing.

It looks like the Mutex object in StorageReferenceInternal is being double-deleted. However, it doesn't explain why didn't experience this crash until upgrading from 6.15.1 to 7.1.1.

This is the Mutex that I suspect is being double-deleted: https://github.com/firebase/firebase-cpp-sdk/blob/6181e9452c6f82a2bf0a46ba975dd56d44041a77/storage/src/ios/storage_reference_ios.h#L189

This is the code that defines the failing assertion: https://github.com/firebase/firebase-cpp-sdk/blob/6181e9452c6f82a2bf0a46ba975dd56d44041a77/app/src/mutex.h#L52-L53

You mentioned that the capture of fileRef is not strictly required. If you remove that capture, does it fix the crash? If it does, it could point to some bad accounting in the copy constructors of StorageReferenceInternal.

mark-grimes commented 3 years ago

Removing fileRef makes no difference.

mark-grimes commented 3 years ago

Some other naive testing, considering I don't really understand the internals. If I fake the Firestore data so that I can go straight to the file download, it works fine. I.e. it's probably not a regression in the file download code (mine or firebase) - it's only when called from a Firestore handler. As mentioned, the failure is in the closing brace so probably nothing to do with the GetFile handler. If I leave out the fileRef.GetFile(...) (I replace with a callback call with an error) it doesn't trigger the assert, so instantiating fileRef is okay at this point but triggering GetFile is not.

dconeybe commented 3 years ago

That makes it sound like the change in the threading model could have uncovered a concurrency bug. Is there a way that you could hack up your code to call copyFileToLocal() on the main thread, just to see if it crashes?

mark-grimes commented 3 years ago

Not easily. It's coded as a pure C++ library which is later included in an iOS app, and anything like dispatch_async( dispatch_get_main_queue(), ^{ ... } ); isn't available at that layer. It's probably easier for me to code up a failing example from scratch.

dconeybe commented 3 years ago

Would using dispatch_async_f() specifying dispatch_get_main_queue() for the queue work?

https://developer.apple.com/documentation/dispatch/1452834-dispatch_async_f

https://developer.apple.com/documentation/dispatch/1452921-dispatch_get_main_queue

mark-grimes commented 3 years ago

I had a look at the code base again, and the relevant part is actually in an Objective-C++ shim that then calls into the pure C++. So I was able to hack it easily, and everything works fine if I shift the call to download the file from Firebase Storage onto the main thread. I need to check other parts of the code but this could potentially be a long a term solution for us. Presumably (from your first comment) this isn't meant to be required though?

dconeybe commented 3 years ago

That's great information. You are correct that it should not be required to use the Storage SDK exclusively from the main thread. This looks like a bug of some sort in the Firebase C++ SDK that should be investigated. The workaround of forcing the callback to run on the main thread sounds reasonable until there is a proper fix for this.

dconeybe commented 3 years ago

Hi @mark-grimes. I've tried to reproduce this crash but have had no success. There is nothing obvious from our code that would cause this issue. And the normally-helpful stack trace that you provided doesn't explain why the assertion in ~Mutex() is failing. So I'm kind of out of options to get a fix out for this.

One idea that comes to mind would be to reproduce the crash with the Address, Thread, and Undefined Behavior sanitizers. Address Sanitizer is probably the most relevant one in this case. This may point to some issues that may help narrow things down. See https://developer.apple.com/documentation/xcode/diagnosing_memory_thread_and_crash_issues_early for details.

I'll keep this issue open for a few more days to see if any ideas come to mind. If there is no progress by then and you have a satisfactory workaround, I'll close it.

dconeybe commented 3 years ago

@mark-grimes Another question... if you remove the call to fileRef.GetFile() from your code, does it still crash?

mark-grimes commented 3 years ago

I tried with the address sanitizer early on, and got loads of crashes where it said the sanitizer couldn't allocate memory. There was also one where it said it couldn't allocate a negative amount of bytes. I'll read that article and give it another go.

I also tried removing the call to fileRef.GetFile() (and replaced with a callback call with an error). Everything runs fine, apart from the obvious not downloading the file.

I'll try the sanitizers you suggested and if that doesn't show anything I'll try and put together a public minimal failing example for you to try.

dconeybe commented 3 years ago

@mark-grimes Thank you for trying those additional suggestions. That is useful information. If I were to provide you with a custom build with some extra logging, would you be willing to use it and reproduce the crash?

mark-grimes commented 3 years ago

Sure, if it helps you figure out what's going on.

dconeybe commented 3 years ago

The custom build is in progress and the C++ SDK with the additional logging can be downloaded once it's complete: https://github.com/firebase/firebase-cpp-sdk/actions/runs/742313931. Once you integrate this custom build into your code base, please reproduce the issue and capture the output. There will be a bunch of log lines beginning with "zzyzx" which will help isolate which mutex is causing issues. Could you attach those logs to a comment on this issue?

Looking at the commit history, you are correct that the callbacks used to happen on the main thread. Starting in 6.16.0 with commit 8d92e73b56308499c05ebc2d147fa3b0d79ad1e2 the callbacks were moved to a dedicated callback thread on iOS.

You mentioned that a workaround that worked for you was to "force" the callbacks to occur on the main thread. You can also explicitly specify any dispatch queue to use for callbacks by calling firebase::firestore::Settings::set_dispatch_queue(dispatch_queue_t queue):

https://github.com/firebase/firebase-cpp-sdk/blob/1f03fab24807ce3e4d5298797197136212d76900/firestore/src/include/firebase/firestore/settings.h#L158-L166

It looks like when the change to move callbacks to a non-main thread was submitted the author forgot to update the comment "By default, the main queue is used". I'm going to make that change.

dconeybe commented 3 years ago

FYI I've created #368 to fix the documentation issues surrounding setting the dispatch queue to use for Firestore callbacks.

dconeybe commented 3 years ago

@mark-grimes There are some unrelated compilation errors in our code base right now preventing me from providing you with the custom-built SDK. I'll post back here once it's ready, hopefully today.

dconeybe commented 3 years ago

I finally have a custom build for you to try out: https://github.com/firebase/firebase-cpp-sdk/actions/runs/748849601. Could you reproduce the crash with this build and attach the logs? I'm especially interested in the lines beginning with "zzyzx" that are printed to stdout.

mark-grimes commented 3 years ago

I've tried this twice with fresh downloads of that package and clean builds, and I get zero log lines with "zzyzx". You meant "zzyzx" literally right? I added ::firebase::SetLogLevel( ::firebase::kLogLevelVerbose );

Here's another stack trace though of a crash I get less frequently, at near enough the same point:

2021-04-15 11:14:34.930485+0100 Snonav[86502:7044301] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '*** -[NSFileManager createDirectoryAtURL:withIntermediateDirectories:attributes:error:]: URL is nil'
*** First throw call stack:
(0x19ec0486c 0x1b3c20c50 0x19fe6c094 0x107edaf88 0x107ee45cc 0x19f3de154 0x19ff53850 0x19fe40740 0x19ff55ca4 0x19fe403c8 0x19ff5674c 0x19ff561d4 0x10c46d850 0x10c45d6c0 0x10c460668 0x10c45fa00 0x10c470ae0 0x10c471488 0x1ea7317d8 0x1ea73876c)
libc++abi.dylib: terminating with uncaught exception of type NSException
*** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '*** -[NSFileManager createDirectoryAtURL:withIntermediateDirectories:attributes:error:]: URL is nil'
terminating with uncaught exception of type NSException
(lldb) bt
* thread #44, queue = 'com.google.GTMSessionFetcher.NSURLSessionDelegateQueue (QOS: UNSPECIFIED)', stop reason = signal SIGABRT
  * frame #0: 0x00000001ccbd4414 libsystem_kernel.dylib`__pthread_kill + 8
    frame #1: 0x00000001ea730b50 libsystem_pthread.dylib`pthread_kill + 272
    frame #2: 0x00000001a8032b74 libsystem_c.dylib`abort + 104
    frame #3: 0x00000001b3d26cf8 libc++abi.dylib`abort_message + 132
    frame #4: 0x00000001b3d17e4c libc++abi.dylib`demangling_terminate_handler() + 308
    frame #5: 0x00000001b3c20f64 libobjc.A.dylib`_objc_terminate() + 144
    frame #6: 0x00000001b3d260e0 libc++abi.dylib`std::__terminate(void (*)()) + 20
    frame #7: 0x00000001b3d2606c libc++abi.dylib`std::terminate() + 44
    frame #8: 0x00000001b3c20ed4 libobjc.A.dylib`objc_terminate + 16
    frame #9: 0x000000010c45d6d4 libdispatch.dylib`_dispatch_client_callout + 40
    frame #10: 0x000000010c460668 libdispatch.dylib`_dispatch_continuation_pop + 584
    frame #11: 0x000000010c45fa00 libdispatch.dylib`_dispatch_async_redirect_invoke + 692
    frame #12: 0x000000010c470ae0 libdispatch.dylib`_dispatch_root_queue_drain + 364
    frame #13: 0x000000010c471488 libdispatch.dylib`_dispatch_worker_thread2 + 140
    frame #14: 0x00000001ea7317d8 libsystem_pthread.dylib`_pthread_wqthread + 216

I'll try an put together a minimal failing example.

mark-grimes commented 3 years ago

Okay, I have a minimal public repository which has the same bug:

https://github.com/mark-grimes/FirebaseCppDoubleCallback

The assert isn't triggered every single time, but it's higher than 50%. Hopefully you can trace through that and figure out what is going on.

dconeybe commented 3 years ago

Thank you for trying out that custom build. I'm going to try out your app to see if I can reproduce because if I can then this whole "custom build" thing won't be required anymore. Thanks for putting the work into the repro app! That will be immensely helpful if I can indeed use it to reproduce.

dconeybe commented 3 years ago

I was able to reproduce with your app! This should greatly help investigation.

dconeybe commented 3 years ago

Update: My investigation lead to a dead end so I've handed off this issue to the team that has expertise in the Storage SDK.

mark-grimes commented 2 years ago

Is there any progress on this from the Storage team? We're having separate issues related to having to run Firestore completion handlers on the main thread, and would prefer not to if Storage can be invoked from a completion handler on another thread.

OpenJeDi commented 4 months ago

I know this issue is old, but I am having the same issue.

I am trying to update an application that has been running fine for five years. In my case, I am using windows. I think the same issue happens on our other platforms (at least MacOS), but I am not completely sure.

The logic is the same: I try to download a data file in the completion callback of a firestore document read.

While investigating, I see I have had the same issue with functions.

The same workaround I wrote for the functions calls, also works here: Instead of using the OnCompletion callback of the future, I run my download code in the background and wait for the future to be completed.

When I do that no crash happens. If I do a GetFile the application crashes, even if I don't set a OnCompletion callback.

Basically: if the Future instance is destroyed before the download is complete, the application crashes, in AcquireMutex in my case.