Bug: Segmentation Fault Error at indexing blocks [180001..182000]

[adriankae]

Describe the bug Everytime I start electrs I get a seg fault error at indexing the blocks. Specifically at blocks [180001..182000]. It's always in the same spot. Any help would be appreciated.

Electrs version v.0.10.5

To Reproduce Steps to reproduce the behavior:

1. Configure and start electrs
2. Wait
3. See error

Expected behavior Indexing throws no errors.

Configuration

electrs.toml

Didn't find it in the directory.

[2024-07-12T16:42:00.610Z INFO  electrs::index] indexing 2000 blocks: [180001..182000]
Segmentation fault (core dumped)

running with gdb:

Thread 8 "rocksdb:high" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff58006c0 (LWP 3120)]
0x0000555555ba5775 in rocksdb::CompressionContextCache::Rep::GetZSTDUncompressData() ()

Environment variables: ELECTRS_X=Y;... Arguments: --foo

System running electrs

• Deployment method: manual from source
• OS name and version: Ubuntu Server 24.04 LTS

Electrum client Client name (if not upstream desktop Electrum) and version: no electrum installed

Additional context n/a

[adriankae]

error running with gdb:

[2024-07-12T16:55:11.385Z INFO electrs::index] indexing 2000 blocks: [180001..182000]

Thread 8 "rocksdb:high" received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7ffff58006c0 (LWP 3120)] 0x0000555555ba5775 in rocksdb::CompressionContextCache::Rep::GetZSTDUncompressData() ()

[Kixunil]

Did you link dynamically or statically?

[adriankae]

Did you link dynamically or statically?

I linked statically.

[Kixunil]

That's really weird, probably has something to do with C++ compilation. Makes me rethink my previous claim that maybe pure Rust impl isn't that interesting.

[adriankae]

When I try to link dynamically i get these errors:

error[E0425]: cannot find function `rocksdb_block_based_options_set_block_cache_compressed` in crate `ffi`
   --> /home/electrs/.cargo/registry/src/index.crates.io-6f17d22bba15001f/electrs-rocksdb-0.19.0-e3/src/db_options.rs:532:18
    |
532 |             ffi::rocksdb_block_based_options_set_block_cache_compressed(self.inner, cache);
    |                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: a function with a similar name exists: `rocksdb_block_based_options_set_block_cache`
    |
   ::: /home/electrs/electrs/target/release/build/electrs-librocksdb-sys-9f28095edbfce9a7/out/bindings.rs:3:59227
    |
3   | ... { pub fn rocksdb_block_based_options_set_block_cache (options : * mut rocksdb_block_based_table_options_t , block_cache : * mut rocksdb_cache_t) ; ...
    |       ---------------------------------------------------------------------------------------------------------------------------------------------- similarly named function `rocksdb_block_based_options_set_block_cache` defined here

error[E0425]: cannot find function `rocksdb_block_based_options_set_block_cache_compressed` in crate `ffi`
   --> /home/electrs/.cargo/registry/src/index.crates.io-6f17d22bba15001f/electrs-rocksdb-0.19.0-e3/src/db_options.rs:553:18
    |
553 |             ffi::rocksdb_block_based_options_set_block_cache_compressed(self.inner, cache.0.inner);
    |                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: a function with a similar name exists: `rocksdb_block_based_options_set_block_cache`
    |
   ::: /home/electrs/electrs/target/release/build/electrs-librocksdb-sys-9f28095edbfce9a7/out/bindings.rs:3:59227
    |
3   | ... { pub fn rocksdb_block_based_options_set_block_cache (options : * mut rocksdb_block_based_table_options_t , block_cache : * mut rocksdb_cache_t) ; ...
    |       ---------------------------------------------------------------------------------------------------------------------------------------------- similarly named function `rocksdb_block_based_options_set_block_cache` defined here

error[E0061]: this function takes 9 arguments but 5 arguments were supplied
    --> /home/electrs/.cargo/registry/src/index.crates.io-6f17d22bba15001f/electrs-rocksdb-0.19.0-e3/src/db_options.rs:2525:13
     |
2525 |               ffi::rocksdb_options_set_plain_table_factory(
     |  _____________^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^-
2526 | |                 self.inner,
2527 | |                 options.user_key_length,
2528 | |                 options.bloom_bits_per_key,
2529 | |                 options.hash_table_ratio,
2530 | |                 options.index_sparseness,
2531 | |             );
     | |_____________- multiple arguments are missing
     |
note: function defined here
    --> /home/electrs/electrs/target/release/build/electrs-librocksdb-sys-9f28095edbfce9a7/out/bindings.rs:3:88313
     |
3    | ...size) ; } extern "C" { pub fn rocksdb_options_set_plain_table_factory (arg1 : * mut rocksdb_options...
     |                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
help: provide the arguments
     |
2525 |             ffi::rocksdb_options_set_plain_table_factory(self.inner, options.user_key_length, options.bloom_bits_per_key, options.hash_table_ratio, options.index_sparseness, /* usize */, /* i8 */, /* u8 */, /* u8 */);
     |                                                         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Some errors have detailed explanations: E0061, E0425.
For more information about an error, try `rustc --explain E0061`.
error: could not compile `electrs-rocksdb` (lib) due to 3 previous errors

[Kixunil]

Yes, Ubuntu has a different rocksdb version than Debian. I wish they didn't break the API so much.

[adriankae]

Is there anything I could do? I'm pretty much out of ideas...

[Kixunil]

I don't have many ideas either. Maybe try switching compiler to clang? (If you used g++, or vice-versa).

[romanz]

@adriankae Does the error reproduce if the index DB is rebuilt from scratch?

[bkjaerland]

I have the exact same issue. For me it did not help to rebuild the DB from scratch.

[bkjaerland]

Found the issue for me, I was missing the build-essential package. After installing it and rebuilding electrs it works like a charm.

[Kixunil]

How come it even compiled without it?! That's super weird.

[antonilol]

Is this a rocksdb issue? I think somewhere a compile time error should be introduced.

[xjmzx]

Confirming the same issue (at same block height) and the same fix build-essential was not installed.

OS: Ubuntu 24.04.1 LTS x86_64 Kernel: 6.8.0-41-generic

[jozbone]

Just confirming as well that I was seeing the same issue at the same block height on Ubuntu 24.04. Installed "build-essential" and rebuilt electrs (static linking). After that I was able to run it and get past the previously failing block height point.

[Kixunil]

I don't think we need more reports, but new people coming to the issue can just thumbs up this comment: https://github.com/romanz/electrs/issues/1061#issuecomment-2308759609

What would be really helpful is to abort compilation if this is happening but that requires some serious digging and will likely need an issue filed against the cc crate (but only once it's confirmed to belong there - please don't pester them about it now).

If someone figures out more information about this that would explain why this happens that'd be great. Sadly I don't have that much time for this kind of thing lately.