search

adoptium / adoptium-support

For end-user problems reported with our binary distributions
Apache License 2.0
45 stars 15 forks source link

x86 and x64 parrallel operation on Windows 10 #56

Closed knttr closed 4 years ago

knttr commented 4 years ago

32-Bit at top of path - run 64-Bit app

Failed to load the JNI shared library "C:\Program Files (x86)\AdoptOpenJDK\jre-8.0.242.08-hotspot\bin\client\jvm.dll".

What have I missed? What else must I consider to run both 32- and 64-Bit Java applications?

aahlenst commented 4 years ago

It's unclear for me what behaviour you expect: Should x64 java be able to call into x86 native code or should Windows "magically" select the right java from %PATH%?

Previously I had both 32- and 64-Bit Java JRE installed and I could run both 32-Bit and 64-Bit Java applications without issues.

Could you please give more details (versions plus JVM vendor)?

Do you have a sample project that we can run to reproduce the issue?

knttr commented 4 years ago

@aahlenst: Please accept my apologies for the late reply .

The previous JVM vendor is Oracle. Somehow Oracle manages to "magically" selects the correct Java.

The two projects below behave like the projects I am using:

javaver product-win32.win32.x86.zip javaver product-win32.win32.x86_64.zip

Thank you in advance for your assistance.

aahlenst commented 4 years ago

I tested various combinations (Azul Zulu 8, Oracle JDK 8 and Oracle JDK 13). It only works with Oracle JDK 8. Therefore I suspect there is some secret sauce being used by the executables in C:\ProgramData\Oracle\Java\javapath\. I do not know whether it is possible to determine whether a Java program uses 32 or 64 bit components before launching it. If it is, we could provide something similar. Checking whether an exe is 32 bit or 64 bit is not enough in this case.

@karianna Do you have an idea or do you know somebody who we could ask?

karianna commented 4 years ago

I tested various combinations (Azul Zulu 8, Oracle JDK 8 and Oracle JDK 13). It only works with Oracle JDK 8. Therefore I suspect there is some secret sauce being used by the executables in C:\ProgramData\Oracle\Java\javapath\. I do not know whether it is possible to determine whether a Java program uses 32 or 64 bit components before launching it. If it is, we could provide something similar. Checking whether an exe is 32 bit or 64 bit is not enough in this case.

@karianna Do you have an idea or do you know somebody who we could ask?

I don't in the short term although perhaps a search through the registry might highlight some special settings that Oracle may use?

@knttr Is that something you can do on your system?

knttr commented 4 years ago

@karianna I have some time neext week, so I'll will give it a go. I might need some hints as to what I'm looking for..... I'm neither programmer nor a windows registry buff, but I have the feeling I'm about to learn something.

tresf commented 4 years ago

It only works with Oracle JDK 8. Therefore I suspect there is some secret sauce

As far as I can recall, they didn't place the entire JVM on the path meaning all of the .dll files weren't exposed. As an ugly stop-gap, you could place a java.bat (where sane) which modifies the %PATH% variable just before running by stripping out the respective conflict.

To that point, changing directories to the java.exe before launch (e.g. "working directory") should put prioritization on the relative .dll files by putting them ahead of the %PATH% values in the library search order. This probably isn't desired, but may be a viable workaround.

Another workaround is dynamically preppending the path to the correct files again in the %PATH% (even if redundant) just before launch, which should also set priority.

Finally, you can install without setting the PATH variable, which is an option in the AdoptOpenJDK installer.

aahlenst commented 4 years ago

@tresf The question I am having is more this one: How can I determine which is the right JDK for the Java program being launched? I must figure out whether it is 32 bit or not and the pick the right JDK. Or did I misunderstand your comment?

tresf commented 4 years ago

How can I determine which is the right JDK for the Java program being launched?

This depends how it's launched as well as the program. Do you have some cases you can share? Detecting Java isn't easy and the community really needs a helper for this, but no single standard currently exists.

This won't be as much of an issue as software developers begin to distribute Java with their applications (then any .dll conflicts become the ownership of that software project), but this change takes time. Worse, some Java applications don't receive updates, or don't have a team large enough to maintain this stuff.

Some examples of Apps that are Java under the covers and manage this all for you:

Some companies have taken this to an extreme... for example, IntellIJ now authors their own runtime (called the JetBrains Runtime, or "jbr" as well as their own "Dark mode" theme called "Darkula") which as time of writing this have features that even OpenJDK are much slower to adopt (e.g. starting with IDEA 2018.2/JBR8 offered better HiDPI support than Java 8).

What does this mean for someone looking to use 32-bit and 64-bit? Well, for starters, I'd recommend not using the PATH ever unless you can control it. Second, it means that you'll need some form of launcher to find where Java is installed and which version to prioritize based on some rules.

If you can share some more about your use-case, I'd be happy to offer some insight. You're not the only person suffering these issues, so if we can find a solution that works for you it'll be a great reference for others.

Note, at the time of writing this, the 32-bit + 64-bit problem is mostly a Windows 64-bit problem. MacOS can't run 32-bit binaries starting with 10.15 Catalina and Linux has historically required a much more manual process -- leaving the responsibility of compatibility and conflicts up to the end-user.

Also at the time of writing this, Windows doesn't have a universal, standardized scripting system for this type of stuff, so even projects like IntelliJ are using batch (.bat) files to detect the environment.

tresf commented 4 years ago

If you can share some more about your use-case

@aahlenst I'm realizing after re-reading that I may be addressing you as the OP. Sorry if any of my questions were misdirected, if so, I invite the OP to reply to them. 😅

aahlenst commented 4 years ago

@tresf No problem. @knttr shared an example in https://github.com/AdoptOpenJDK/openjdk-support/issues/56#issuecomment-588932766.

I looked into it a few weeks ago. With Oracle JDK up until and including version 8 the right thing happens: 32 bit or 64 bit JDK is selected depending on the needs of the program. Does not work with any OpenJDK. So I see two possible scenarios:

  1. There's secret sauce in Oracle JDK that can do this. I have no idea how. But I'd really like to know.
  2. The sample programs are hardcoded to work with Oracle JDK and we do not provide the necessary shims (if we can at all).

This is not really my area of expertise. I'm supporting server-side software and developers and those usually have different problems.

tresf commented 4 years ago

I'm not sure what is meant by "Correct version is selected". What's "selecting" Java? I downloaded the two zip files, they appear to be .exe files, not .jar files.

02/18/20  12:15 PM                37 build.properties
03/26/20  11:45 AM    <DIR>          configuration
02/18/20  02:07 PM           319,488 dLISTconfig.exe
03/26/20  11:45 AM    <DIR>          p2
02/18/20  11:38 AM               679 feature.xml
03/26/20  11:45 AM    <DIR>          plugins
03/26/20  11:45 AM    <DIR>          target
03/26/20  11:45 AM    <DIR>          features
02/18/20  02:07 PM               282 dLISTconfig.ini
02/18/20  02:07 PM            24,576 eclipsec.exe
02/18/20  02:07 PM            18,553 artifacts.xml

Digging a bit deeper, it appears there's an plugin called Equinox launcher that's assisting with this process. Would it be better to see if they have support for non-Oracle JVMs? https://wiki.eclipse.org/Equinox_Launcher#Finding_a_VM.2C_Using_JNI_Invocation_or_Executing_Java

knttr commented 4 years ago

@aahlenst I just questioned the programmer who wrote the two sample programs: these are not hardcoded to use Oracle JDK.

When reading through all the comments above, I presume it will be a waste of time to sieve through the registry, as this seems to be a "path" issue. (I'm out of my depth here!😕)

For our own applications, we're reverting to @tresf's suggestion "to distribute Java with their applications".

The issue for me as a user is that I'm stuck using Oracle, since some applications, which require an installed JRE, are only available as either 64 Bit or 32 Bit versions.

@tresf comments "You're not the only person suffering these issues, so if we can find a solution that works for you it'll be a great reference for others." This leaves me in a bind - do a leave this issue open or do I close it?

knttr commented 4 years ago

(Sorry, I closed the issue due to coordination problems between my mouse and myself....)

As I do not know whether I can really be of help in this issue, I would appreciate your comments on whether to leave this issue open or not.

Thank your all your insights and comments!

tresf commented 4 years ago

I think the bug report deserves a unit test which includes:

  1. Test to perform
  2. Expected results
  3. Actual results

I say this at a high-level because the .exe files that are provided aren't Java at all, so help with this issue requires an understanding of a utility who's very purpose is to help the JVM selection, which should probably happen in coordination with Adopt (the Eclipse or IBM devs help explain JVM selection).

What I've seen from reading some of the source code of Equinox is that it uses some system properties about the JVM that started to help understand which JVM is currently running, but I'm uncertain if it actually helps select the target JVM.

For these reasons, I still strongly feel that this is a PATH pollution problem which has existed forever with this launcher, just exposed by Adopt since it adds itself to the PATH rather blindly.

I feel that a temporary, viable workaround is to:

  1. Uninstall all version of Adopt
  2. Reinstall Adopt WITHOUT the PATH option selected and WITH the JavaSoft registry selected
  3. Test the launcher

This allows a launcher to use the old Oracle technique for detecting the JVM without worrying about foo.dll (x86_64) comes before or after foo.dll (x86) in the library search order.

Unfortunately, I'm unable to test this theory because I'm not entirely sure what the steps are to reproduce. Regardless, the solution is going to help anyone in the future relying on this Equinox technology, even if it means patching Equinox itself.

aahlenst commented 4 years ago

@knttr I'm reasonably confident that the problem is not in AdoptOpenJDK nor in OpenJDK but rather in Eclipse Equinox. We recently had different problems with 32/64 bit selection on Windows. Please reach out to the Equinox people. Bug tracker: https://bugs.eclipse.org/bugs/buglist.cgi?component=Launcher&product=Equinox&resolution=---. If we need to provide additional/different settings, we're happy to work with them. If you open a bug, please post a link here. I'll close this issue for the time being.

@tresf Thanks a lot for your thorough analysis and help.

tresf commented 4 years ago

I'm reasonably confident that the problem is not in AdoptOpenJDK nor in OpenJDK but rather in Eclipse Equinox.

Agreed. I re-read the Equinox documentation several times now and I think it's related to the default JNI-launch technique. Equinox uses jvm.dll unless in special edge-cases and I can see how PATH pollution could break that.

What I don't understand is why this fails in the first place. For example, if Program Files\ occurs before Program Files (x86)\ in the PATH, shouldn't it just ALWAYS prefer 64-bit due to path search order?

Edit: I had forgotten there are separate 32-bit and 64-bit launchers, so I'd guess they should be self-bitness aware and have their own preference.

Perhaps it's a simple as fixing Equinox to rearrange the PATH on launch:

... of course, this is potentially undefined behavior as now PATH search order is obfuscated, but probably fixes 99% of issues.

An even better strategy would be to look for Adopt through other means and not trust PATH at all. I have a suspicion this is what happens with the Oracle version. If you can find the right bitness JVM through the registry, you can prepend that to PATH.

If Equinox knows what platform it prefers, it needs to make a few more checks before falling back onto the PATH. Unfortunately -- at the time of writing this -- I don't think there's a universally adopted, non-Oracle way for identifying JDKs on the system. I currently use HKLM\SOFTWARE\Classes\AdoptOpenJDK.jarfile\shell\open\command, but that's undoubtedly a hack.

tresf commented 4 years ago

... and perhaps this is easier than all of the above. By default a x86 process reads values from the 32-bit registry and a x86_64 process reads values from the 64-bit registry so if registry is acting as a bitness detector, the correct JVM would "just work" (and thus fail to load at all, or fallback to command-line invocation if missing for that bitness).