rails-sqlserver / tiny_tds

TinyTDS - Simple and fast FreeTDS bindings for Ruby using DB-Library.
Other
605 stars 191 forks source link

ruby 3.3.0 -> 3.3.1 new error "machine stack overflow in critical region (fatal)" #557

Open bf4 opened 2 months ago

bf4 commented 2 months ago

I'm not positive what this is related to, but figured I'd log an issue in case it means something to someone.

Ubuntu Linux, Heroku-20 image

Ruby 3.3.1

Tiny Tds 2.1.7

The only backtrace I have is

gems/tiny_tds-2.1.7/lib/tiny_tds/client.rb:60:in `connect': machine stack overflow in critical region (fatal)

and it's not reproducible every time.

RbConfig::CONFIG ```ruby {"DESTDIR"=>"", "MAJOR"=>"3", "MINOR"=>"3", "TEENY"=>"1", "PATCHLEVEL"=>"55", "INSTALL"=>"/usr/bin/install -c", "EXEEXT"=>"", "prefix"=>"/app/vendor/ruby-3.3.1", "ruby_install_name"=>"ruby", "RUBY_INSTALL_NAME"=>"ruby", "RUBY_SO_NAME"=>"ruby", "exec"=>"exec", "ruby_pc"=>"ruby-3.3.pc", "CC_WRAPPER"=>"", "PACKAGE"=>"ruby", "BUILTIN_TRANSSRCS"=>" enc/trans/newline.c", "MKMF_VERBOSE"=>"0", "MANTYPE"=>"man", "vendorarchhdrdir"=>"/app/vendor/ruby-3.3.1/include/ruby-3.3.0/vendor_ruby/x86_64-linux", "sitearchhdrdir"=>"/app/vendor/ruby-3.3.1/include/ruby-3.3.0/site_ruby/x86_64-linux", "rubyarchhdrdir"=>"/app/vendor/ruby-3.3.1/include/ruby-3.3.0/x86_64-linux", "vendorhdrdir"=>"/app/vendor/ruby-3.3.1/include/ruby-3.3.0/vendor_ruby", "sitehdrdir"=>"/app/vendor/ruby-3.3.1/include/ruby-3.3.0/site_ruby", "rubyhdrdir"=>"/app/vendor/ruby-3.3.1/include/ruby-3.3.0", "RUBY_SEARCH_PATH"=>"", "UNIVERSAL_INTS"=>"", "UNIVERSAL_ARCHNAMES"=>"", "configure_args"=> " '--disable-install-doc' '--prefix' '/tmp/d20240423-1-qthlgt/prefix' '--enable-load-relative' '--enable-shared' '--enable-yjit'" , "CONFIGURE"=>"configure", "vendorarchdir"=>"/app/vendor/ruby-3.3.1/lib/ruby/vendor_ruby/3.3.0/x86_64-linux", "vendorlibdir"=>"/app/vendor/ruby-3.3.1/lib/ruby/vendor_ruby/3.3.0", "vendordir"=>"/app/vendor/ruby-3.3.1/lib/ruby/vendor_ruby", "sitearchdir"=>"/app/vendor/ruby-3.3.1/lib/ruby/site_ruby/3.3.0/x86_64-linux", "sitelibdir"=>"/app/vendor/ruby-3.3.1/lib/ruby/site_ruby/3.3.0", "sitedir"=>"/app/vendor/ruby-3.3.1/lib/ruby/site_ruby", "rubyarchdir"=>"/app/vendor/ruby-3.3.1/lib/ruby/3.3.0/x86_64-linux", "rubylibdir"=>"/app/vendor/ruby-3.3.1/lib/ruby/3.3.0", "ruby_version"=>"3.3.0", "sitearch"=>"x86_64-linux", "arch"=>"x86_64-linux", "sitearchincludedir"=>"/app/vendor/ruby-3.3.1/include/x86_64-linux", "archincludedir"=>"/app/vendor/ruby-3.3.1/include/x86_64-linux", "sitearchlibdir"=>"/app/vendor/ruby-3.3.1/lib/x86_64-linux", "archlibdir"=>"/app/vendor/ruby-3.3.1/lib/x86_64-linux", "libdirname"=>"libdir", "RUBY_EXEC_PREFIX"=>"", "RUBY_LIB_VERSION"=>"", "RUBY_LIB_VERSION_STYLE"=>"3\t/* full */", "RI_BASE_NAME"=>"ri", "ridir"=>"/app/vendor/ruby-3.3.1/share/ri", "rubysitearchprefix"=>"/app/vendor/ruby-3.3.1/lib/ruby/x86_64-linux", "rubyarchprefix"=>"/app/vendor/ruby-3.3.1/lib/ruby/x86_64-linux", "MAKEFILES"=>"Makefile GNUmakefile", "USE_LLVM_WINDRES"=>"", "PLATFORM_DIR"=>"", "COROUTINE_TYPE"=>"amd64", "THREAD_MODEL"=>"pthread", "SYMBOL_PREFIX"=>"", "EXPORT_PREFIX"=>"", "COMMON_HEADERS"=>"", "COMMON_MACROS"=>"", "COMMON_LIBS"=>"", "MAINLIBS"=>"-lz -lrt -lrt -lgmp -ldl -lcrypt -lm -lpthread ", "ENABLE_SHARED"=>"yes", "DLDSHARED"=>"gcc -shared", "DLDLIBS"=>"-lc", "SOLIBS"=>"-lz -lrt -lrt -lgmp -ldl -lcrypt -lm -lpthread ", "LIBRUBYARG_SHARED"=>"-Wl,-rpath,'${ORIGIN}/../lib' -Wl,-rpath,'${ORIGIN}/../lib' -lruby", "LIBRUBYARG_STATIC"=> "-Wl,-rpath,'${ORIGIN}/../lib' -Wl,-rpath,'${ORIGIN}/../lib' -lruby-static -lz -lrt -lrt -lgmp -ldl -lcrypt -lm -lpthread ", "LIBRUBYARG"=>"-Wl,-rpath,'${ORIGIN}/../lib' -Wl,-rpath,'${ORIGIN}/../lib' -lruby", "LIBRUBY"=>"libruby.so.3.3.1", "LIBRUBY_ALIASES"=>"libruby.so.3.3 libruby.so", "LIBRUBY_SONAME"=>"libruby.so.3.3", "LIBRUBY_SO"=>"libruby.so.3.3.1", "LIBRUBY_A"=>"libruby-static.a", "RUBYW_INSTALL_NAME"=>"", "rubyw_install_name"=>"", "EXTDLDFLAGS"=>"", "EXTLDFLAGS"=>"", "strict_warnflags"=>"", "warnflags"=> "-Wall -Wextra -Wdeprecated-declarations -Wdiv-by-zero -Wduplicated-cond -Wimplicit-function-declaration -Wimplicit-int -Wpointer -arith -Wwrite-strings -Wold-style-definition -Wimplicit-fallthrough=0 -Wmissing-noreturn -Wno-cast-function-type -Wno-constant-log ical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-packed-bitfield-compat -Wno-parentheses-eq uality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wsuggest-attribute=format -Wsuggest-attr ibute=noreturn -Wunused-variable -Wmisleading-indentation -Wundef", "debugflags"=>"-g", "optflags"=>"-O3 -fno-fast-math", "NULLCMD"=>":", "ENABLE_DEBUG_ENV"=>"", "DLNOBJ"=>"dln.o", "RJIT_SUPPORT"=>"yes", "YJIT_OBJ"=>"yjit.o", "YJIT_LIBS"=>"yjit/target/release/libyjit.a", "CARGO_BUILD_ARGS"=>"", "YJIT_SUPPORT"=>"yes", "CARGO"=>"", "RUSTC"=>"rustc", "INSTALL_STATIC_LIBRARY"=>"no", "EXECUTABLE_EXTS"=>"", "ARCHFILE"=>"", "LIBRUBY_RELATIVE"=>"yes", "EXTOUT"=>".ext", "PREP"=>"miniruby", "CROSS_COMPILING"=>"no", "TEST_RUNNABLE"=>"yes", "rubylibprefix"=>"/app/vendor/ruby-3.3.1/lib/ruby", "setup"=>"Setup", "SOEXT"=>"so", "TRY_LINK"=>"", "PRELOADENV"=>"LD_PRELOAD", "LIBPATHENV"=>"LD_LIBRARY_PATH", "RPATHFLAG"=>" -Wl,-rpath,%1$-s", "LIBPATHFLAG"=>" -L%1$-s", "LINK_SO"=>"", "ADDITIONAL_DLDFLAGS"=>"", "ENCSTATIC"=>"", "EXTSTATIC"=>"", "ASMEXT"=>"S", "LIBEXT"=>"a", "DLEXT"=>"so", "LDSHAREDXX"=>"g++ -shared", "LDSHARED"=>"gcc -shared", "CCDLFLAGS"=>"-fPIC", "STATIC"=>"", "ARCH_FLAG"=>"", "DLDFLAGS"=>"-Wl,--compress-debug-sections=zlib", "ALLOCA"=>"", "EGREP"=>"/usr/bin/grep -E", "GREP"=>"/usr/bin/grep", "dsymutil"=>"", "codesign"=>"", "cleanlibs"=>"", "POSTLINK"=>":", "incflags"=>"", "WERRORFLAG"=>"-Werror", "RUBY_DEVEL"=>"", "CHDIR"=>"cd -P", "RMALL"=>"rm -fr", "RMDIRS"=>"rmdir --ignore-fail-on-non-empty -p", "RMDIR"=>"rmdir --ignore-fail-on-non-empty", "CP"=>"cp", "RM"=>"rm -f", "PKG_CONFIG"=>"pkg-config", "DOXYGEN"=>"", "DOT"=>"", "MKDIR_P"=>"/usr/bin/mkdir -p", "INSTALL_DATA"=>"/usr/bin/install -c -m 644", "INSTALL_SCRIPT"=>"/usr/bin/install -c", "INSTALL_PROGRAM"=>"/usr/bin/install -c", "SET_MAKE"=>"", "LN_S"=>"ln -s", "DLLWRAP"=>"", "WINDRES"=>"", "ASFLAGS"=>"", "ARFLAGS"=>"rcD ", "try_header"=>"", "CC_VERSION_MESSAGE"=> "gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0\nCopyright (C) 2019 Free Software Foundation, Inc.\nThis is free software; see the sou rce for copying conditions. There is NO\nwarranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.", "CC_VERSION"=>"gcc --version", "CSRCFLAG"=>"", "COUTFLAG"=>"-o ", "OUTFLAG"=>"-o ", "CPPOUTFILE"=>"-o conftest.i", "GNU_LD"=>"yes", "GCC"=>"yes", "CPP"=>"gcc -E", "CXXFLAGS"=>"", "OBJEXT"=>"o", "CPPFLAGS"=>" ", "LDFLAGS"=>"-L. -fstack-protector-strong -rdynamic -Wl,-export-dynamic -Wl,--no-as-needed", "CFLAGS"=> "-O3 -fno-fast-math -g -Wall -Wextra -Wdeprecated-declarations -Wdiv-by-zero -Wduplicated-cond -Wimplicit-function-declaration -W implicit-int -Wpointer-arith -Wwrite-strings -Wold-style-definition -Wimplicit-fallthrough=0 -Wmissing-noreturn -Wno-cast-function- type -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-packed-bitfield-comp at -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wsuggest-attribute =format -Wsuggest-attribute=noreturn -Wunused-variable -Wmisleading-indentation -Wundef -fPIC", "STRIP"=>"strip -S -x", "RANLIB"=>"gcc-ranlib", "OBJDUMP"=>"objdump", "OBJCOPY"=>"objcopy", "NM"=>"gcc-nm", "LD"=>"ld", "CXX"=>"g++", "AS"=>"as", "AR"=>"gcc-ar", "CC"=>"gcc", "wasmoptflags"=>"", "WASMOPT"=>"", "target_os"=>"linux", "target_vendor"=>"pc", "target_cpu"=>"x86_64", "target"=>"x86_64-pc-linux", "host_os"=>"linux", "host_vendor"=>"pc", "host_cpu"=>"x86_64", "host"=>"x86_64-pc-linux", "build_os"=>"linux-gnu", "build_vendor"=>"pc", "build_cpu"=>"x86_64", "build"=>"x86_64-pc-linux-gnu", "RUBY_VERSION_NAME"=>"ruby-3.3.0", "RUBYW_BASE_NAME"=>"rubyw", "RUBY_BASE_NAME"=>"ruby", "RUBY_PROGRAM_VERSION"=>"3.3.1", "RUBY_API_VERSION"=>"3.3", "HAVE_GIT"=>"yes", "GIT"=>"git", "cxxflags"=>"", "cppflags"=>"", "cflags"=> "-O3 -fno-fast-math -g -Wall -Wextra -Wdeprecated-declarations -Wdiv-by-zero -Wduplicated-cond -Wimplicit-function-declaration -W implicit-int -Wpointer-arith -Wwrite-strings -Wold-style-definition -Wimplicit-fallthrough=0 -Wmissing-noreturn -Wno-cast-function- type -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-packed-bitfield-comp at -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wsuggest-attribute =format -Wsuggest-attribute=noreturn -Wunused-variable -Wmisleading-indentation -Wundef", "MAKEDIRS"=>"/usr/bin/mkdir -p", "target_alias"=>"", "host_alias"=>"", "build_alias"=>"", "LIBS"=>"-lm -lpthread ", "ECHO_T"=>"", "ECHO_N"=>"-n", "ECHO_C"=>"", "DEFS"=>"", "mandir"=>"/app/vendor/ruby-3.3.1/share/man", "localedir"=>"/app/vendor/ruby-3.3.1/share/locale", "libdir"=>"/app/vendor/ruby-3.3.1/lib", : "psdir"=>"/app/vendor/ruby-3.3.1/share/doc/ruby", "pdfdir"=>"/app/vendor/ruby-3.3.1/share/doc/ruby", "dvidir"=>"/app/vendor/ruby-3.3.1/share/doc/ruby", "htmldir"=>"/app/vendor/ruby-3.3.1/share/doc/ruby", "infodir"=>"/app/vendor/ruby-3.3.1/share/info", "docdir"=>"/app/vendor/ruby-3.3.1/share/doc/ruby", "oldincludedir"=>"/usr/include", "includedir"=>"/app/vendor/ruby-3.3.1/include", "runstatedir"=>"/app/vendor/ruby-3.3.1/var/run", "localstatedir"=>"/app/vendor/ruby-3.3.1/var", "sharedstatedir"=>"/app/vendor/ruby-3.3.1/com", "sysconfdir"=>"/app/vendor/ruby-3.3.1/etc", "datadir"=>"/app/vendor/ruby-3.3.1/share", "datarootdir"=>"/app/vendor/ruby-3.3.1/share", "libexecdir"=>"/app/vendor/ruby-3.3.1/libexec", "sbindir"=>"/app/vendor/ruby-3.3.1/sbin", "bindir"=>"/app/vendor/ruby-3.3.1/bin", "exec_prefix"=>"/app/vendor/ruby-3.3.1", "PACKAGE_URL"=>"", "PACKAGE_BUGREPORT"=>"", "PACKAGE_STRING"=>"", "PACKAGE_VERSION"=>"", "PACKAGE_TARNAME"=>"", "PACKAGE_NAME"=>"", "PATH_SEPARATOR"=>":", "SHELL"=>"/bin/bash", "UNICODE_VERSION"=>"15.0.0", "UNICODE_EMOJI_VERSION"=>"15.0", "platform"=>"x86_64-linux", "archdir"=>"/app/vendor/ruby-3.3.1/lib/ruby/3.3.0/x86_64-linux", "topdir"=>"/app/vendor/ruby-3.3.1/lib/ruby/3.3.0/x86_64-linux"} ```
bf4 commented 2 months ago

possibly related to https://github.com/grpc/grpc/pull/36125

andyundso commented 2 months ago

I updated a couple of apps now to Ruby 3.2.4 and freetds v1.4.12. will let you know if I run into similar issues.

bf4 commented 2 months ago

Thanks @andyundso I pointed someone (byroot) to take a quick look at the c-code and they don't think it's related to the same grpc bug. In fact, even though this was a new error for us, we've now seen it again, though only once, since downgrading back to 3.3.0. So, it appears unlikely it's related to the Ruby 3.3.0 change, but it's definitely a failure which points here.. and unfortunately I haven't been able to reproduce on demand. My guess would be it has to do with connection handling and either 1) having the remote close the connection without and tiny_tds not handling it gracefully or 2) the remote server process died and tiny_tds did not handle it gracefully or 3) having the server sent a kill signal and it not being handled gracefully, though this seems less likely in this case.

andyundso commented 2 months ago

judging from the tests, we simulate a slow_close and a timeout using Toxiproxy. more details are documented in #481 and in #445.

Toxiproxy would also support to simulate a TCPRESET - this is something we could add tests for and see if this would trigger the error mentioned by you. and even if not, it is probably still a good idea to test it.