yugabyte / yugabyte-db

YugabyteDB - the cloud native distributed SQL database for mission-critical applications.
https://www.yugabyte.com
Other
8.88k stars 1.05k forks source link

[DocDB] Single node cluster may hit Too big clock skew FATAL on Mac #13260

Open qvad opened 2 years ago

qvad commented 2 years ago

Jira Link: DB-2908

Description

Started local node cluster on Mac M1 using yugabyte-db repository and following commands:

./yb_build.sh release --no-tests --skip-java-build
bin/yugabyted start

While evaluating optimizer test hit following FATAL.

F20220711 23:50:10 /Users/dsherstobitov/code/yugabyte-db/src/yb/server/hybrid_clock.cc:177] Too big clock skew is detected: 0.629s, while max allowed is: 0.500s; clock_skew_force_crash_bound_usec=60000000
    @        0x10621a510  google::LogDestination::LogToSinks()
    @        0x106219648  google::LogMessage::SendToLog()
    @        0x10621a004  google::LogMessage::Flush()
    @        0x10621e038  google::LogMessageFatal::~LogMessageFatal()
    @        0x10621af54  google::LogMessageFatal::~LogMessageFatal()
    @        0x104645d8c  yb::server::HybridClock::NowWithError()
    @        0x104645378  yb::server::HybridClock::NowRange()
    @        0x106276744  yb::client::internal::AsyncRpcBase<>::AsyncRpcBase()
    @        0x106276098  yb::client::internal::ReadRpc::ReadRpc()
    @        0x106280678  std::__1::make_shared<>()
    @        0x1062803cc  yb::client::internal::Batcher::CreateRpc()
    @        0x10627f510  yb::client::internal::Batcher::ExecuteOperations()
    @        0x10627ed2c  yb::client::internal::Batcher::AllLookupsDone()
    @        0x10627e0c4  yb::client::internal::Batcher::TabletLookupFinished()
    @        0x106284570  std::__1::__invoke_void_return_wrapper<>::__call<>()
    @        0x10631a920  _ZN2yb15ScopeExitLambdaIZNS_6client8internal9MetaCache19DoLookupTabletByKeyINS_10SharedLockINSt3__118shared_timed_mutexEEEEEbRKNS6_10shared_ptrIKNS1_7YBTableEEERKNS9_IKNS1_27VersionedTablePartitionListEEERKNS9_IKNS6_12basic_stringIcNS6_11char_traitsIcEENS6_9allocatorIcEEEEEENS6_6chrono10time_pointINS_15CoarseMonoClockENSU_8durationIxNS6_5ratioILl1ELl1000000000EEEEEEEPNS6_8functionIFvRKNS_6ResultI13scoped_refptrINS2_12RemoteTabletEEEEEEEPSR_EUlvE_ED1Ev
    @        0x1063090f8  yb::client::internal::MetaCache::DoLookupTabletByKey<>()
    @        0x106308794  yb::client::internal::MetaCache::LookupTabletByKey()
    @        0x10627d7c4  yb::client::internal::Batcher::FlushAsync()
    @        0x10632695c  yb::client::(anonymous namespace)::FlushBatcherAsync()
    @        0x10632653c  yb::client::YBSession::FlushAsync()
    @        0x1039a6a50  yb::tserver::PgClientSession::Perform()
    @        0x1039a23ec  yb::tserver::PgClientServiceImpl::Impl::DoPerform()
    @        0x10399c484  yb::tserver::PgClientServiceImpl::Impl::Perform()
    @        0x1046cfed0  std::__1::__function::__func<>::operator()()
    @        0x1046c6d80  yb::tserver::PgClientServiceIf::Handle()
    @        0x1078339d0  yb::rpc::ServicePoolImpl::Handle()
    @        0x1077e2010  yb::rpc::InboundCall::InboundCallTask::Run()
    @        0x1078417d0  yb::rpc::(anonymous namespace)::Worker::Execute()
    @        0x1083a2960  yb::Thread::SuperviseThread()
    @        0x19e57026c  _pthread_start
    @        0x19e56b08c  thread_start
tedyu commented 2 years ago

I think this is unique to MacOS.