search

yugabyte / yugabyte-db

YugabyteDB - the cloud native distributed SQL database for mission-critical applications.
https://www.yugabyte.com
Other
9.02k stars 1.08k forks source link

[PITR]Cores found during bank workload in test_ysql_snapshots_bank_after_schedule_nemesis #24953

Open agsh-yb opened 4 days ago

agsh-yb commented 4 days ago

Jira Link: DB-14090

Description

Core dump found after sqlbanktransfer with PITR test run with nemesis version: 2.25.0.0-b254

Screenshot 2024-11-18 at 2 17 30 PM

attached logs in jira

lldb) target create "/home/yugabyte/master/bin/yb-master" --core "/home/yugabyte/cores/core_yb.1730755019.async-tasks.41569"
Core file '/home/yugabyte/cores/core_yb.1730755019.async-tasks.41569' (aarch64) was loaded.
(lldb) bt all
* thread #1, name = 'yb-master', stop reason = signal SIGABRT
  * frame #0: 0x0000ffffadf96274 libc.so.6`raise + 172
    frame #1: 0x0000ffffadf80a2c libc.so.6`abort + 276
    frame #2: 0x0000aaaac2e028a4 yb-master`abort_message + 164
    frame #3: 0x0000aaaac2e024a4 yb-master`__cxa_pure_virtual + 20
    frame #4: 0x0000aaaac135ec10 yb-master`yb::master::RetryingTSRpcTask::DoRpcCallback(this=0x000035a7bc33f8e0) at async_rpc_tasks.cc:588:59
    frame #5: 0x0000aaaac262f598 yb-master`yb::ThreadPool::DispatchThread(this=0x000035a7bf852fc0, permanent=false) at threadpool.cc:612:22
    frame #6: 0x0000aaaac262a738 yb-master`yb::Thread::SuperviseThread(void*) [inlined] std::__1::__function::__value_func<void ()>::operator()[abi:ue170006](this=0x000035a7bdd0e3f0) const at function.h:517:16
    frame #7: 0x0000aaaac262a724 yb-master`yb::Thread::SuperviseThread(void*) [inlined] std::__1::function<void ()>::operator()(this=0x000035a7bdd0e3f0) const at function.h:1168:12
    frame #8: 0x0000aaaac262a724 yb-master`yb::Thread::SuperviseThread(arg=0x000035a7bdd0e390) at thread.cc:884:3
    frame #9: 0x0000ffffadf278b8 libpthread.so.0`start_thread + 392
    frame #10: 0x0000ffffadf83afc libc.so.6`thread_start + 12
  thread #2, stop reason = signal 0
    frame #0: 0x0000ffffadf2dc60 libpthread.so.0`pthread_cond_wait@@GLIBC_2.17 + 528
    frame #1: 0x0000aaaac0d8a830 yb-master`yb::TerminationMonitor::WaitForTermination() [inlined] void std::__1::condition_variable::wait<yb::TerminationMonitor::WaitForTermination()::$_0>(this=0x000035a7bfdb2238, __lk=<unavailable>, __pred=(unnamed class) @ x19) at condition_variable.h:148:5
    frame #2: 0x0000aaaac0d8a818 yb-master`yb::TerminationMonitor::WaitForTermination(this=0x000035a7bfdb2200) at termination_monitor.cc:68:16
    frame #3: 0x0000aaaac0969854 yb-master`main [inlined] yb::master::MasterMain(argc=<unavailable>, argv=<unavailable>) at master_main.cc:162:24
    frame #4: 0x0000aaaac0968d3c yb-master`main(argc=<unavailable>, argv=<unavailable>) at master_main.cc:190:12
    frame #5: 0x0000ffffadf84384 libc.so.6`__libc_start_main + 220
    frame #6: 0x0000aaaac07de034 yb-master`_start + 52

Issue Type

kind/bug

Warning: Please confirm that this issue does not contain any sensitive information

agsh-yb commented 2 days ago

in 2.20.8.0-b50:
F1119 16:55:56.817930 45613 hybrid_clock.cc:177] Too big clock skew is detected: 0.515s, while max allowed is: 0.500s; clock_skew_force_crash_bound_usec=60000000