facebook / rocksdb

A library that provides an embeddable, persistent key-value store for fast storage.
http://rocksdb.org
GNU General Public License v2.0
28.72k stars 6.34k forks source link

A fatal error has been detected by the Java Runtime Environment: EXCEPTION_ILLEGAL_INSTRUCTION (0xc000001d) #12786

Open SanketArdalkar opened 5 months ago

SanketArdalkar commented 5 months ago

It shows problematic frame as C [librocksdbjni14464789980629058905.dll+0x546c29] what should i do. i created jar file of project containing database operation and inserted into another projects extension i.e hivemq brokers extension and this error occured even if i deleted jar file still problem persist.

A fatal error has been detected by the Java Runtime Environment:

#  EXCEPTION_ILLEGAL_INSTRUCTION (0xc000001d) at pc=0x00007ffb94946c29, pid=9408, tid=11796
#
# JRE version: Java(TM) SE Runtime Environment 18.9 (11.0.22+9) (build 11.0.22+9-LTS-219)
# Java VM: Java HotSpot(TM) 64-Bit Server VM 18.9 (11.0.22+9-LTS-219, mixed mode, tiered, compressed oops, g1 gc, windows-amd64)
# Problematic frame:
# C  [librocksdbjni14464789980629058905.dll+0x546c29]
#
# No core dump will be written. Minidumps are not enabled by default on client versions of Windows
#
# If you would like to submit a bug report, please visit:
#   https://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
adamretter commented 5 months ago

There is not enough information here to act on. Can you provide the stack trace please?

SanketArdalkar commented 5 months ago

hs_err_pid12608.log check above file. Do you think it would work if i delete hivemq and again download it ? librocksdb files are present in hivemq application or they are part of java? TIA

stefan-zobel commented 5 months ago

That CPU is a Westmere-EP microarchitecture from 2010 (before Sandy Bridge) which only has SSE4.2 instruction set extensions. It's almost impossible that a recent rocksdbjni build can run on this CPU. You'd need a portable build with flags -DPORTABLE=1 (and possibly also -DSNAPPY_HAVE_BMI2=0).

See https://github.com/facebook/rocksdb/issues/11096 which is a very similar issue.

SanketArdalkar commented 5 months ago

before inserting jar in the application it was running fine and i have removed the jar and i want that application should run like it was running before inserting the jar i dont want that jar containing database to run with application. help me so that i can run my application like it was running before insertion of jar. TIA

adamretter commented 5 months ago

That CPU is a Westmere-EP microarchitecture from 2010 (before Sandy Bridge) which only has SSE4.2 instruction set extensions. It's almost impossible that a recent rocksdbjni build can run on this CPU. You'd need a portable build with flags DPORTABLE=1 (and possibly also -DSNAPPY_HAVE_BMI2=0)

@stefan-zobel All RocksJava releases published to Maven central are already built with PORTABLE=1

adamretter commented 5 months ago

Do you think it would work if i delete hivemq and again download it ? librocksdb files are present in hivemq application or they are part of java?

@SanketArdalkar We cannot comment on the HiveMQ project - I suggest you contact them.

RocksJava includes the RocksDB native library. I don't know what HiveMQ do.

stefan-zobel commented 5 months ago

That CPU is a Westmere-EP microarchitecture from 2010 (before Sandy Bridge) which only has SSE4.2 instruction set extensions. It's almost impossible that a recent rocksdbjni build can run on this CPU. You'd need a portable build with flags DPORTABLE=1 (and possibly also -DSNAPPY_HAVE_BMI2=0)

@stefan-zobel All RocksJava releases published to Maven central are already built with PORTABLE=1

@adamretter That means AVX2 is disabled for the builds on Maven central?

adamretter commented 5 months ago

If that is blocked by PORTABLE=1, then yes

stefan-zobel commented 5 months ago

If that is blocked by PORTABLE=1, then yes

@adamretter Thanks for the confirmation. According to https://github.com/facebook/rocksdb/wiki/Building-on-Windows the /arch:AVX2 flag is blocked by -DPORTABLE=1.

ajkr commented 4 months ago

disassembly shows:

   180546c29:   c4 42 50 f5 c6          bzhi   %ebp,%r14d,%r8d

bzhi is part of BMI2: https://en.wikipedia.org/wiki/X86_Bit_manipulation_instruction_set#BMI2_(Bit_Manipulation_Instruction_Set_2)

I also don't know why the compiler outputs BMI2 instructions when portable is set.

stefan-zobel commented 4 months ago

@ajkr I've seen that bzhi instruction in a similar issue (https://github.com/facebook/rocksdb/issues/11096). In that case it came in through snappy.

ajkr commented 4 months ago

Thank you for the pointer. I have not found anywhere we are compiling Snappy for the Windows build, although we do compile it for the non-Windows builds so it wouldn't be too surprising.

rhubner commented 4 months ago

Hello @ajkr

we compile windows releases with snappy. Also with LZ4, zlib, zstd. I will double check what arguments are used to compile compression libraries.

Radek

cc: @adamretter

ajkr commented 4 months ago

Thanks for the info. I guess it would be fair to set -DSNAPPY_HAVE_BMI2=0 considering compilation of RocksDB itself does not use those instructions with PORTABLE=1.

rhubner commented 4 months ago

Hello @ajkr ,

I can confirm it's Snappy. I found exact instructions in disassembled snappy static library. For next release we will compile with -DSNAPPY_HAVE_BMI2=0

@SanketArdalkar Before we fix this in next release I can suggest as workaround to disable snappy compression or use different compression. For example zstd.

Radek

stefan-zobel commented 4 months ago

@rhubner Not related, but while you are at it I believe liblz4_static.lib should be compiled with LZ4_DLL_EXPORT=0. The vcxproj file in the download defines LZ4_DLL_EXPORT=1 which leads to the LZ4 functions being exported from librocksdbjni-win64.dll (which doesn't seem to make a lot of sense). LZ4_exports

rhubner commented 4 months ago

@stefan-zobel Interesting,

we had opposite problem in #12246 and #12226 where I didn't know how to export RocksDB C API in librocksdbjni-win64.dll. Maybe I should check how LZ4 get compiled and exported and then I can do the same for RocksDB C API.

What tool are you using to inspect .dll files?

stefan-zobel commented 4 months ago

What tool are you using to inspect .dll files?

@rhubner The screenshot is from an an age-old Dependency Walker (https://www.dependencywalker.com/) from 2006. There are better tools nowadays but I can't remember a name right now.