`cannot allocate memory in static TLS block` although enough memory is available

[MagicLike]

Steps to reproduce the problem

1. Run the update procedures as described here
2. Run RAILS_ENV=production SKIP_POST_DEPLOYMENT_MIGRATIONS=true bundle exec rails db:migrate

Expected behaviour

db migrations should run normally

Actual behaviour

rails interupts and throws an error

Detailed description

Here is the terminal output:

mastodon@sekundenklebertransportverbot:~/live$ RAILS_ENV=production SKIP_POST_DEPLOYMENT_MIGRATIONS=true bundle exec rails db:migrate --trace
bin/rails aborted!
LoadError: /usr/lib/aarch64-linux-gnu/libjemalloc.so.2: cannot allocate memory in static TLS block - /home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/pg-1.5.7/lib/pg_ext.so (LoadError)
/home/mastodon/.rbenv/versions/3.3.5/lib/ruby/3.3.0/bundled_gems.rb:75:in `require'
/home/mastodon/.rbenv/versions/3.3.5/lib/ruby/3.3.0/bundled_gems.rb:75:in `block (2 levels) in replace_require'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/bootsnap-1.18.4/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/pg-1.5.7/lib/pg.rb:49:in `block in <module:PG>'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/pg-1.5.7/lib/pg.rb:37:in `block in <module:PG>'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/pg-1.5.7/lib/pg.rb:42:in `<module:PG>'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/pg-1.5.7/lib/pg.rb:6:in `<main>'
<internal:/home/mastodon/.rbenv/versions/3.3.5/lib/ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:37:in `require'
<internal:/home/mastodon/.rbenv/versions/3.3.5/lib/ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:37:in `require'
/home/mastodon/.rbenv/versions/3.3.5/lib/ruby/3.3.0/bundled_gems.rb:75:in `block (2 levels) in replace_require'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/bundler-2.5.18/lib/bundler/runtime.rb:60:in `block (2 levels) in require'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/bundler-2.5.18/lib/bundler/runtime.rb:55:in `each'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/bundler-2.5.18/lib/bundler/runtime.rb:55:in `block in require'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/bundler-2.5.18/lib/bundler/runtime.rb:44:in `each'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/bundler-2.5.18/lib/bundler/runtime.rb:44:in `require'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/bundler-2.5.18/lib/bundler.rb:212:in `require'
/home/mastodon/live/config/application.rb:23:in `<main>'
/home/mastodon/.rbenv/versions/3.3.5/lib/ruby/3.3.0/bundled_gems.rb:75:in `require'
/home/mastodon/.rbenv/versions/3.3.5/lib/ruby/3.3.0/bundled_gems.rb:75:in `block (2 levels) in replace_require'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/bootsnap-1.18.4/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
/home/mastodon/live/Rakefile:6:in `<main>'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/rake_module.rb:29:in `load'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/rake_module.rb:29:in `load_rakefile'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/application.rb:740:in `raw_load_rakefile'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/application.rb:126:in `block in load_rakefile'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/application.rb:214:in `standard_exception_handling'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/application.rb:125:in `load_rakefile'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/railties-7.1.4/lib/rails/commands/rake/rake_command.rb:43:in `block in with_rake'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/rake_module.rb:59:in `with_application'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/railties-7.1.4/lib/rails/commands/rake/rake_command.rb:41:in `with_rake'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/railties-7.1.4/lib/rails/commands/rake/rake_command.rb:20:in `perform'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/railties-7.1.4/lib/rails/command.rb:156:in `invoke_rake'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/railties-7.1.4/lib/rails/command.rb:73:in `block in invoke'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/railties-7.1.4/lib/rails/command.rb:149:in `with_argv'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/railties-7.1.4/lib/rails/command.rb:69:in `invoke'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/railties-7.1.4/lib/rails/commands.rb:18:in `<main>'
/home/mastodon/.rbenv/versions/3.3.5/lib/ruby/3.3.0/bundled_gems.rb:75:in `require'
/home/mastodon/.rbenv/versions/3.3.5/lib/ruby/3.3.0/bundled_gems.rb:75:in `block (2 levels) in replace_require'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/bootsnap-1.18.4/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
bin/rails:4:in `<main>'

This might suggest that ruby/rails ran out of memory, but I still have multiple Gigabytes available... I also already tried to run the suggested fixes in mastodon/mastodon#15751, which only result in more errors:

mastodon@sekundenklebertransportverbot:~/live$ RAILS_ENV=production SKIP_POST_DEPLOYMENT_MIGRATIONS=true bundle exec rails db:migrate --trace
ERROR: ld.so: object '/path/to/libjemalloc.so.2' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/lib/arm-linux-gnueabihf/libjemalloc.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/path/to/libjemalloc.so.2' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/lib/arm-linux-gnueabihf/libjemalloc.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/path/to/libjemalloc.so.2' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/lib/arm-linux-gnueabihf/libjemalloc.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/path/to/libjemalloc.so.2' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/lib/arm-linux-gnueabihf/libjemalloc.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/path/to/libjemalloc.so.2' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/lib/arm-linux-gnueabihf/libjemalloc.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/path/to/libjemalloc.so.2' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/lib/arm-linux-gnueabihf/libjemalloc.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/path/to/libjemalloc.so.2' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/lib/arm-linux-gnueabihf/libjemalloc.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/path/to/libjemalloc.so.2' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/lib/arm-linux-gnueabihf/libjemalloc.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/path/to/libjemalloc.so.2' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/lib/arm-linux-gnueabihf/libjemalloc.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/path/to/libjemalloc.so.2' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/lib/arm-linux-gnueabihf/libjemalloc.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/path/to/libjemalloc.so.2' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/lib/arm-linux-gnueabihf/libjemalloc.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/path/to/libjemalloc.so.2' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/lib/arm-linux-gnueabihf/libjemalloc.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/path/to/libjemalloc.so.2' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/lib/arm-linux-gnueabihf/libjemalloc.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/path/to/libjemalloc.so.2' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/lib/arm-linux-gnueabihf/libjemalloc.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/path/to/libjemalloc.so.2' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/lib/arm-linux-gnueabihf/libjemalloc.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/path/to/libjemalloc.so.2' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/lib/arm-linux-gnueabihf/libjemalloc.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/path/to/libjemalloc.so.2' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/lib/arm-linux-gnueabihf/libjemalloc.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/path/to/libjemalloc.so.2' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/lib/arm-linux-gnueabihf/libjemalloc.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/path/to/libjemalloc.so.2' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/lib/arm-linux-gnueabihf/libjemalloc.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/path/to/libjemalloc.so.2' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/lib/arm-linux-gnueabihf/libjemalloc.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/path/to/libjemalloc.so.2' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/lib/arm-linux-gnueabihf/libjemalloc.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/path/to/libjemalloc.so.2' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/lib/arm-linux-gnueabihf/libjemalloc.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/path/to/libjemalloc.so.2' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/lib/arm-linux-gnueabihf/libjemalloc.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
bin/rails aborted!
LoadError: /usr/lib/aarch64-linux-gnu/libjemalloc.so.2: cannot allocate memory in static TLS block - /home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/pg-1.5.7/lib/pg_ext.so (LoadError)
/home/mastodon/.rbenv/versions/3.3.5/lib/ruby/3.3.0/bundled_gems.rb:75:in `require'
/home/mastodon/.rbenv/versions/3.3.5/lib/ruby/3.3.0/bundled_gems.rb:75:in `block (2 levels) in replace_require'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/bootsnap-1.18.4/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/pg-1.5.7/lib/pg.rb:49:in `block in <module:PG>'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/pg-1.5.7/lib/pg.rb:37:in `block in <module:PG>'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/pg-1.5.7/lib/pg.rb:42:in `<module:PG>'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/pg-1.5.7/lib/pg.rb:6:in `<main>'
<internal:/home/mastodon/.rbenv/versions/3.3.5/lib/ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:37:in `require'
<internal:/home/mastodon/.rbenv/versions/3.3.5/lib/ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:37:in `require'
/home/mastodon/.rbenv/versions/3.3.5/lib/ruby/3.3.0/bundled_gems.rb:75:in `block (2 levels) in replace_require'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/bundler-2.5.18/lib/bundler/runtime.rb:60:in `block (2 levels) in require'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/bundler-2.5.18/lib/bundler/runtime.rb:55:in `each'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/bundler-2.5.18/lib/bundler/runtime.rb:55:in `block in require'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/bundler-2.5.18/lib/bundler/runtime.rb:44:in `each'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/bundler-2.5.18/lib/bundler/runtime.rb:44:in `require'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/bundler-2.5.18/lib/bundler.rb:212:in `require'
/home/mastodon/live/config/application.rb:23:in `<main>'
/home/mastodon/.rbenv/versions/3.3.5/lib/ruby/3.3.0/bundled_gems.rb:75:in `require'
/home/mastodon/.rbenv/versions/3.3.5/lib/ruby/3.3.0/bundled_gems.rb:75:in `block (2 levels) in replace_require'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/bootsnap-1.18.4/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
/home/mastodon/live/Rakefile:6:in `<main>'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/rake_module.rb:29:in `load'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/rake_module.rb:29:in `load_rakefile'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/application.rb:740:in `raw_load_rakefile'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/application.rb:126:in `block in load_rakefile'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/application.rb:214:in `standard_exception_handling'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/application.rb:125:in `load_rakefile'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/railties-7.1.4/lib/rails/commands/rake/rake_command.rb:43:in `block in with_rake'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/rake_module.rb:59:in `with_application'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/railties-7.1.4/lib/rails/commands/rake/rake_command.rb:41:in `with_rake'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/railties-7.1.4/lib/rails/commands/rake/rake_command.rb:20:in `perform'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/railties-7.1.4/lib/rails/command.rb:156:in `invoke_rake'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/railties-7.1.4/lib/rails/command.rb:73:in `block in invoke'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/railties-7.1.4/lib/rails/command.rb:149:in `with_argv'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/railties-7.1.4/lib/rails/command.rb:69:in `invoke'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/railties-7.1.4/lib/rails/commands.rb:18:in `<main>'
/home/mastodon/.rbenv/versions/3.3.5/lib/ruby/3.3.0/bundled_gems.rb:75:in `require'
/home/mastodon/.rbenv/versions/3.3.5/lib/ruby/3.3.0/bundled_gems.rb:75:in `block (2 levels) in replace_require'
/home/mastodon/live/vendor/bundle/ruby/3.3.0/gems/bootsnap-1.18.4/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
bin/rails:4:in `<main>'

But maybe I am missing something...

Mastodon instance

soc.sekundenklebertransportverbot.de

Mastodon version

v4.3.0-alpha.5+glitch

Technical details

• Ruby version: ruby 3.3.5 (2024-09-03 revision ef084cc8f4) [aarch64-linux]
• Node.js version: v20.17.0

[ThisIsMissEm]

See: https://github.com/mastodon/mastodon/issues/31472

[MagicLike]

thx for the hint, deleting vendor/bundle fixed it <3