0.21 pocketcoin-qt exception on launch when loading wallet

[tawmaz]

Pocketcoin-qt fails with exception when attempting to load wallet, with message "CDataStream::read(): end of data: iostream error". Exception occurs in function RecentRequestsTableModel::addNewRequest when serializing requests from the wallet. Error is not seen with 0.20.19 build.

Stack below:

libc.so.6!__GI_raise(int sig) (/build/glibc-sMfBJT/glibc-2.31/sysdeps/unix/sysv/linux/raise.c:50)
libc.so.6!__GI_abort() (/build/glibc-sMfBJT/glibc-2.31/stdlib/abort.c:79)
libstdc++.so.6![Unknown/Just-In-Time compiled code] (Unknown Source:0)
libstdc++.so.6!std::terminate() (Unknown Source:0)
libstdc++.so.6!__cxa_throw (Unknown Source:0)
ReadCompactSize<CDataStream>(CDataStream & is, bool range_check) (/usr/include/c++/9/system_error:151)
Unserialize<CDataStream, char>(CDataStream & is, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > & str) (/media/data/src/pocketnet.core.sqlite/src/serialize.h:737)
UnserializeMany<CDataStream, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > & arg, CDataStream & s) (/media/data/src/pocketnet.core.sqlite/src/serialize.h:1055)
UnserializeMany<CDataStream, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > & arg, CDataStream & s) (/media/data/src/pocketnet.core.sqlite/src/serialize.h:1058)
UnserializeMany<CDataStream, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > & arg, CDataStream & s) (/media/data/src/pocketnet.core.sqlite/src/serialize.h:1058)
UnserializeMany<CDataStream, long&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&>(long & arg, CDataStream & s) (/media/data/src/pocketnet.core.sqlite/src/serialize.h:1058)
UnserializeMany<CDataStream, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, long&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > & arg, CDataStream & s) (/media/data/src/pocketnet.core.sqlite/src/serialize.h:1058)
UnserializeMany<CDataStream, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, long&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > & arg, CDataStream & s) (/media/data/src/pocketnet.core.sqlite/src/serialize.h:1058)
UnserializeMany<CDataStream, int&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, long&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&>(int & arg, CDataStream & s) (/media//data/src/pocketnet.core.sqlite/src/serialize.h:1058)
SerReadWriteMany<CDataStream, int&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, long&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&>(CDataStream & s) (/media//data/src/pocketnet.core.sqlite/src/serialize.h:1070)
SendCoinsRecipient::SerializationOps<CDataStream, SendCoinsRecipient, CSerActionUnserialize>(CDataStream & s, SendCoinsRecipient & obj) (/media/data/src/pocketnet.core.sqlite/src/qt/sendcoinsrecipient.h:56)
SendCoinsRecipient::Unser<CDataStream>(SendCoinsRecipient & obj, CDataStream & s) (/media/data/src/pocketnet.core.sqlite/src/qt/sendcoinsrecipient.h:47)
SendCoinsRecipient::Unserialize<CDataStream>(CDataStream & s, SendCoinsRecipient * const this) (/media/data/src/pocketnet.core.sqlite/src/qt/sendcoinsrecipient.h:47)
Unserialize<CDataStream, SendCoinsRecipient&>(SendCoinsRecipient & a, CDataStream & is) (/media/data/src/pocketnet.core.sqlite/src/serialize.h:707)
UnserializeMany<CDataStream, SendCoinsRecipient&>(SendCoinsRecipient & arg, CDataStream & s) (/media/data/src/pocketnet.core.sqlite/src/serialize.h:1057)
UnserializeMany<CDataStream, unsigned int&, SendCoinsRecipient&>(unsigned int & arg, CDataStream & s) (/media/data/src/pocketnet.core.sqlite/src/serialize.h:1058)
UnserializeMany<CDataStream, long&, unsigned int&, SendCoinsRecipient&>(long & arg, CDataStream & s) (/media/data/src/pocketnet.core.sqlite/src/serialize.h:1058)
UnserializeMany<CDataStream, int&, long&, unsigned int&, SendCoinsRecipient&>(int & arg, CDataStream & s) (/media/data/src/pocketnet.core.sqlite/src/serialize.h:1058)
SerReadWriteMany<CDataStream, int&, long&, unsigned int&, SendCoinsRecipient&>(CDataStream & s) (/media/data/src/pocketnet.core.sqlite/src/serialize.h:1070)
RecentRequestEntry::SerializationOps<CDataStream, RecentRequestEntry, CSerActionUnserialize>(CDataStream & s, RecentRequestEntry & obj) (/media/data/src/pocketnet.core.sqlite/src/qt/recentrequeststablemodel.h:30)
RecentRequestEntry::Unser<CDataStream>(RecentRequestEntry & obj, CDataStream & s) (/media/data/src/pocketnet.core.sqlite/src/qt/recentrequeststablemodel.h:27)
RecentRequestEntry::Unserialize<CDataStream>(CDataStream & s, RecentRequestEntry * const this) (/media/data/src/pocketnet.core.sqlite/src/qt/recentrequeststablemodel.h:27)
Unserialize<CDataStream, RecentRequestEntry&>(RecentRequestEntry & a, CDataStream & is) (/media/data/src/pocketnet.core.sqlite/src/serialize.h:707)
CDataStream::operator>><RecentRequestEntry&>(RecentRequestEntry & obj, CDataStream * const this) (/media/data/src/pocketnet.core.sqlite/src/streams.h:459)
RecentRequestsTableModel::addNewRequest(RecentRequestsTableModel * const this,  recipient) (/media/data/src/pocketnet.core.sqlite/src/qt/recentrequeststablemodel.cpp:186)
RecentRequestsTableModel::RecentRequestsTableModel(RecentRequestsTableModel * const this, WalletModel * parent) (/media/data/src/pocketnet.core.sqlite/src/qt/recentrequeststablemodel.cpp:24)
WalletModel::WalletModel(WalletModel * const this, std::unique_ptr<interfaces::Wallet, std::default_delete<interfaces::Wallet> > wallet,  client_model, const PlatformStyle * platformStyle, QObject * parent) (/media/data/src/pocketnet.core.sqlite/src/qt/walletmodel.cpp:54)
WalletController::getOrCreateWallet(WalletController * const this, std::unique_ptr<interfaces::Wallet, std::default_delete<interfaces::Wallet> > wallet) (/usr/include/c++/9/bits/move.h:74)
WalletController::WalletController(WalletController * const this,  client_model, const PlatformStyle * platform_style, QObject * parent) (/usr/include/c++/9/bits/move.h:74)
PocketcoinApplication::initializeResult(PocketcoinApplication * const this, bool success, interfaces::BlockAndHeaderTipInfo tip_info) (/media/data/src/pocketnet.core.sqlite/src/qt/pocketcoin.cpp:377)
QtPrivate::FunctorCall<QtPrivate::IndexesList<0, 1>, QtPrivate::List<bool, interfaces::BlockAndHeaderTipInfo>, void, void (PocketcoinApplication::*)(bool, interfaces::BlockAndHeaderTipInfo)>::call(void ** arg, PocketcoinApplication * o, void (PocketcoinApplication::*)(PocketcoinApplication * const, bool, interfaces::BlockAndHeaderTipInfo) f) (/media/data/src/pocketnet.core.sqlite/depends/x86_64-pc-linux-gnu/include/QtCore/qobjectdefs_impl.h:168)
QtPrivate::FunctionPointer<void (PocketcoinApplication::*)(bool, interfaces::BlockAndHeaderTipInfo)>::call<QtPrivate::List<bool, interfaces::BlockAndHeaderTipInfo>, void>(void ** arg, PocketcoinApplication * o, QtPrivate::FunctionPointer<void (PocketcoinApplication::*)(bool, interfaces::BlockAndHeaderTipInfo)>::Function f) (/media/data/src/pocketnet.core.sqlite/depends/x86_64-pc-linux-gnu/include/QtCore/qobjectdefs_impl.h:169)
QtPrivate::QSlotObject<void (PocketcoinApplication::*)(bool, interfaces::BlockAndHeaderTipInfo), QtPrivate::List<bool, interfaces::BlockAndHeaderTipInfo>, void>::impl(int which, QtPrivate::QSlotObjectBase * this_, QObject * r, void ** a, bool * ret) (/media/data/src/pocketnet.core.sqlite/depends/x86_64-pc-linux-gnu/include/QtCore/qobject_impl.h:120)
QObject::event(QEvent*) (Unknown Source:0)
QGuiApplication::event(QEvent*) (Unknown Source:0)
QApplication::event(QEvent*) (Unknown Source:0)
QApplicationPrivate::notify_helper(QObject*, QEvent*) (Unknown Source:0)
QApplication::notify(QObject*, QEvent*) (Unknown Source:0)
QCoreApplication::notifyInternal2(QObject*, QEvent*) (Unknown Source:0)
QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) (Unknown Source:0)
QEventDispatcherUNIX::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) (Unknown Source:0)
QUnixEventDispatcherQPA::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) (Unknown Source:0)
QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) (Unknown Source:0)
QCoreApplication::exec() (Unknown Source:0)
GuiMain(int argc, char ** argv) (/media/data/src/pocketnet.core.sqlite/src/qt/pocketcoin.cpp:702)
libc.so.6!__libc_start_main(int (*)(int, char **, char **) main, int argc, char ** argv, int (*)(int, char **, char **) init, void (*)(void) fini, void (*)(void) rtld_fini, void * stack_end) (/build/glibc-sMfBJT/glibc-2.31/csu/libc-start.c:308)
_start() (/usr/include/c++/9/ext/new_allocator.h:89)

[tawmaz]

It appears to be only an issue with pocketcoin-qt and the RecentRequestsTableModel class which is only used by pocketcoin-qt. pocketcoind was able to load my wallet correctly, fully sync, and display my balance. I will check to see if there are some changes which did not get merged over from bitcoin.

[tawmaz]

In the 0.21 branch there appears to have been some big changes to walletmodel.cpp which is in the code stack of this crash in this commit here: https://github.com/pocketnetteam/pocketnet.core/commit/87dc9c573ee651cb6472b7362a8ef3b05c0f420c

The code appears to be different from both bitcoin core and 0.20 branch.

[lostystyg]

In the 0.21 branch there appears to have been some big changes to walletmodel.cpp which is in the code stack of this crash in this commit here: 87dc9c5

The code appears to be different from both bitcoin core and 0.20

The code is absolutely the same as in bitcoin core at af591f2 because there were no changes from us in walletmodel.cpp except renaming all to pocket.

Also, i'm unable to reproduce this, my wallet is loaded well. I've tried both wallets created with 0.20 and 0.21 and both were loaded well. However, i used fresh created wallets, probably this is the difference. @tawmaz, can you provide more info on how i can reproduce this?

[tawmaz]

Attached the testnet wallet file which was used to repro this.

wallet.zip

[lostystyg]

The problem is in serialization for SendCoinsRecipient class. e020490188648a1444119c6115241b2cc506b5b0 introduced serialization changes and removed serializing field sPaymentRequest from SendCoinsRecipient class for 0.20. Later, for 0.21 SendCoinsRecipient class was extracted from walletmodel.h to different file and sPaymentRequest was returned because it is situated in bitcoin upstream. Absence of this field is the reason why error doesn't occur in 0.20. @tawmaz, can you remember what was the reason for removing sPaymentRequest from serialization? However, in 0.21 sPaymentRequest seems to be no longer meaningful and is used just to not break serialization of older wallets, but i don't think we could safely just remove it because it is deserialized before auth_merchant_str and so deleting sPaymentRequest will prevent from correct deserializing auth_merchant_str and any potentially new added field after it. Thus, if we really can't handle sPaymentRequest and even don't need it, we should also remove auth_merchant_str and take care of not adding some more fields to wallet recipients that can be a shotgun in the future.

Notes:

• reverting e020490188648a1444119c6115241b2cc506b5b0 causes the same error as currently for 0.21
• deleting sPaymentRequest from serialization for 0.21 here results in successful wallet loading.

[lostystyg]

@andyoknen said he can test this issue's reproducibility on a lot of different wallets, so we can be sure that this is not a specific problem to the wallet @tawmaz attached.

[tawmaz]

It looks like it was my mistake when porting to the new serialization functions. It would be worth enabling qt/test/wallettests.cpp which would catch this issue.

[tawmaz]

One idea would be to perform the deserialization in a try-catch, and if we encounter the overrun serialization issue perform the incorrect serialization from 0.20.19 to update the wallet to the new format. Let me know if you want me to retake this issue @lostystyg