Closed buchholzmd closed 9 months ago
Welcome, @buchholzmd ! Let's see if we can figure this out.
Upon starting the py5 kernel, it immediately dies. When using the created conda env as the kernel, the kernel dies after importing py5.
Starting the py5 kernel imports py5 for you, so let's focus on why the kernel dies when you import py5. And that's a pretty horrific error message you have there. There must be a problem of some kind with how the JDK was installed.
Can you provide the output to py5_tools.get_jvm_debug_info()
? Do the JAVA_HOME and default jvm path values make sense to you for your setup?
Here's what I get on my Mac computer:
% ipython
import Python 3.8.16 | packaged by conda-forge | (default, Feb 1 2023, 16:05:36)
Type 'copyright', 'credits' or 'license' for more information
IPython 8.12.2 -- An enhanced Interactive Python. Type '?' for help.
In [1]: import py5_tools
py
In [2]: py5_tools.get_jvm_debug_info()
Out[2]:
{'JAVA_HOME environment variable': '/usr/local/Cellar/openjdk/17.0.2/',
'jvm version': (0, 0, 0),
'default jvm path': '/usr/local/Cellar/openjdk/17.0.2/libexec/openjdk.jdk/Contents/Home/lib/libjli.dylib'}
Thanks for a quick response! To be clear - I did not install Java on my computer, but only within the conda environment (not sure if that matters though).
After running the debugging method I get the following:
After installing Java on my computer (as well as installing it in the conda env using openjdk
), I now get a new (and less ugly) error message:
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x000000018fbd5808, pid=5308, tid=259
#
# JRE version: OpenJDK Runtime Environment Zulu20.28+85-CA (20.0+36) (build 20+36)
# Java VM: OpenJDK 64-Bit Server VM Zulu20.28+85-CA (20+36, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, bsd-aarch64)
# Problematic frame:
# C [libobjc.A.dylib+0x9808] objc_msgSend+0x8
#
# No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /Users/matthewbuchholz/Documents/Art/genart/hs_err_pid5308.log
#
# If you would like to submit a bug report, please visit:
# http://www.azul.com/support/
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
was unable to install Java 17 (or 20) using install-jdk due to a 404 error
The install-jdk library supports installing various JDKs and JREs builds. You can specify a different vendor besides the default Adoptium. You can also get the download URL it is attempting to use for debugging purposes:
In [6]: import jdk
In [7]: jdk.get_download_url('17')
Out[7]: 'https://api.adoptium.net/v3/binary/latest/17/ga/mac/x64/jdk/hotspot/normal/eclipse'
In [8]: jdk.get_download_url('17', vendor='Azul')
Out[8]: 'https://cdn.azul.com/zulu/bin/zulu17.42.19-ca-jdk17.0.7-macosx_x64.tar.gz'
Both of those URLs work for me right now. But if the default option gives you a 404 error, you can always try a different one.
I suspect there is something amiss with how the JDK was installed on your computer. I know that install-jdk
does it correctly.
Thanks for a quick response! To be clear - I did not install Java on my computer, but only within the conda environment (not sure if that matters though).
That's it! Please don't install Java with anaconda. The JREs/JDKs that it will install are problematic. I've seen weird dependencies where installing matplotlib will downgrade your Java version, which makes no sense to me. I suggest starting with a new Anaconda environment and installing Java with install-jdk
, perhaps using a different vendor.
I believe you are correct that this is an issue with my JDK install... unfortunately, I cannot resolve the issue yet.
So far, I have removed Java and JDK from my mac and started from scratch. I was able to get install-jdk
to work by trying another vendor, as mentioned. First issue that arises is a permissions error from trying to run jdk.install
when not the root user, so the only way I can install using install-jdk
is with the following command:
sudo python -c "import jdk; print('Java installed to', jdk.install('17', vendor='Azul'))"
After this step, when running java --version
in the terminal, I receive the message stating that java is not installed. The only fix to this is downloading the Java .dmg from Oracle's website directly. However, now the output of the py5_tools
method is the following:
I think my problem boils down to configuring the correct directory for $JAVA_HOME
, but I have yet to find the correct setup and am further confused that the default jvm path is now /Library/Java/JavaVirtualMachines/
(which does not work for $JAVA_HOME
).
First issue that arises is a permissions error from trying to run jdk.install when not the root user, so the only way I can install using
install-jdk
is with the following command:
Hmmm, something's wrong here. You shouldn't need sudo permission because it will by default install it in $HOME/.jdk/<VERSION>
or $HOME/.jre/<VERSION>
. Is there something funny with your $HOME
variable? If not, can you try opening an issue for this with for that library's repo at https://github.com/jyksnw/install-jdk? I've corresponded with the maintainer of that project before, he is diligent about getting back to people. Feel free to @ me in the issue you create there if you would like.
I think my problem boils down to configuring the correct directory for
$JAVA_HOME
, but I have yet to find the correct setup and am further confused that the default jvm path is now/Library/Java/JavaVirtualMachines/
(which does not work for $JAVA_HOME).
But before opening an issue, can you try setting your JAVA_HOME
to /Library/Java/JavaVirtualMachines/jdk-17.jdk
? And if that doesn't work, try each subdirectory navigating towards the bin
subdirectory that contains the java
executable?
Interesting, so a few things:
/Users/myname
JAVA_HOME
to /Library/Java/JavaVirtualMachines/jdk-17.jdk
yields zsh: /Library/Java/JavaVirtualMachines/jdk-17.jdk/ not found
... the same goes for trying /opt/homebrew/Cellar/openjdk@17/17.0.7/
. I have confirmed that both directories do in fact exist... sighpy5
issue as much as it's user error ... I've used py5
in the past without issue and just excited to get back into it againAfter setting JAVA_HOME
, what do you see when you type echo $JAVA_HOME
? In the Python kernel, what happens when you look in os.environ['JAVA_HOME']
?
In [1]: import os
In [2]: os.environ['JAVA_HOME']
Out[2]: '/usr/lib/jvm/java-17-openjdk'
(My machine here is Linux w/ Java installed by the OS in /usr/lib/jvm
)
I tried setting JAVA_HOME
to be /Library/Java/JavaVirtualMachines/jdk-17.jdk/
, /opt/homebrew/Cellar/openjdk@17/17.0.7/
and /opt/homebrew/Cellar/openjdk@17/
. Each time os.environ['JAVA_HOME']
reports the correct, corresponding values. Still kernel is dying upon importing py5
Hmmm, I don't understand what the issue is. Can you work on getting install-jdk
to install without sudo? That, so far, has proven to be a reliable way for others to install Java on OSX. Where is install-jdk
trying to install to anyway? Do you have some permission issue where you can't create new directories in your home directory?
Yes, I started looking around a bit at past issues for install-jdk
but with no luck. I'll open an issue like you suggested
It's trying to install in $HOME/.jdk/<VERSION>
, so yes, I am having permissions denied in my home directory for some reason
Can you create any directories in your home directory? $HOME/temp/
? If not, you should fix that first. It has something to do with ACLs, I believe. Is this a new computer? I vaguely remember doing something to address this when I got a mac a few years ago.
@hx2A Yes, this is a new Mac, so this must be the case. What is weird is that I did grant permissions for folders under $HOME
, and yes I can mkdir
from $HOME
, so there is clearly some finer-grain permissions needed that I need to sort out.
Thanks for the help, once I figure out how to run this not as root, I reckon things will be good to go!
Some progress: for some reason, when using install-jdk
the directories it creates are owned by root rather than user. Not sure if this is an issue with how conda/miniforge is configured on my machine or my mac in general but will update once that's solved
It's not Ananconda. Previously you said you first ran install-jdk
with sudo
, which is why root owns that directory. Can you try removing that directory completely and running install-jdk
without sudo
? If that doesn't work, figure out what the problem is without using sudo
. I don't see why you need sudo
rights if you can use mkdir
in $HOME
. Some confusing Mac thing.
Ok, thank you. The problem was that I forgot to remove the full jdk
directory. Now I can run install-jdk
without sudo
. After doing that java --version
still says the Java runtime cannot be found, but setting JAVA_HOME
to be $HOME/.jdk/zulu17.42.19-ca-jdk17.0.7-macosx_aarch64/zulu-17.jdk/Contents/Home
, Java can find the OpenJDK runtime. But unfortunately the kernel again dies upon importing py5, with the same nasty NSException
Can you try this in a regular Python interpreter:
>>> import jpype
>>> jpype.startJVM()
?
If this doesn't work, it must be a jpype problem, an issue with the JVM, or an issue your machine. In that case, I would try other JVMs options provided by install-jdk. Can you try with the default Adoptium?
If that does work, it a py5 problem. I'm running out of ideas for what to do though.
So starting JVM with jpype
does work. This is all with, Azul btw.
Adoptium was giving the 404 error with install-jdk. I can try to download Adoptium either with brew or from a webpage and just setting JAVA_HOME based on that install maybe?
Including the log for reference
Can you double check you have the latest version of install-jdk?
$ pip install install-jdk --upgrade
Requirement already satisfied: install-jdk in /home/jim/INSTALL/anaconda3/envs/py5/lib/python3.8/site-packages (1.0.4)
It should be 1.0.4.
Next, here is the download URL for adoptium Java 17:
>>> jdk.get_download_url('17')
'https://api.adoptium.net/v3/binary/latest/17/ga/linux/x64/jdk/hotspot/normal/eclipse'
Does that URL work for you in a browser? Is install-jdk still giving a 404 error? That should have been a temporary thing. Do you get a different download URL than me?
You can also try installing Java with the appropriate DMG installer from here:
https://www.oracle.com/java/technologies/downloads/#jdk17-mac
That should be really easy to get working. Then remove or adjust $JAVA_HOME
for the new location. If you need to figure out where it installed Java to, try which java
in a terminal.
Yes, I have 1.0.4, and btw I'm using python 3.9.
The url you provided (for linux) does work, but for mac arm it's not working: running the get_url
method it yields
https://api.adoptium.net/v3/binary/latest/17/ga/mac/arm/jdk/hotspot/normal/eclipse
However, I changed to this architecture and this url works:
https://api.adoptium.net/v3/binary/latest/17/ga/mac/aarch64/jdk/hotspot/normal/eclipse
Still receive the same error of kernel dying.
And btw, I initially tried the DMG install from Oracle's website. Just tried again with JAVA_HOME
set to the same path as which java
(/usr/bin/java/) as well as the actual Home directory installed (/Library/Java/JavaVirtualMachines/jdk-17.jdk/Contents/Home/). Still kernel dies with same stack trace as in the log earlier
Maybe this is an architecture issue? Perhaps you get a 404 error because Adoptium does not have a build for the architecture install-jdk detects is correct for your machine? You can't install a build for a different architecture. This could explain why it is crashing the way that it is.
Possibly... but what confuses me is that aarch64 is the correct architecture. And further, from my understanding aarch64 and arm64 are equivalent. So not sure what the difference between arm
and aarch64
is here (on the install-jdk
pypi page they list all the architectures that can be detected).
I've opened an issue with install-jdk
and hopefully can get some clarification.
The different architectures are confusing to me, especially on the mac computers. Let's see what the install-jdk maintainer has to say.
Side note: are you at NYU Courant? I graduated from there in 2005 with a MS in Math & Finance.
Agreed - will keep this thread updated! And thanks for all the help btw
And yes I am! I actually checked out your blog a few days ago and saw you were an alum. I watched the video of your ITP thesis and found it really cool! I actually worked with spherical convolutions a few times before and your work would have provided some useful references xD
Any updates? Are you still trying to get py5 working on your machine?
@buchholzmd Is this complete? Can I close this issue?
I am going to close this. Please open a new issue for further mac problems.
I followed these install steps but was unable to install Java 17 (or 20) using
install-jdk
due to a 404 error, so had to useopenjdk
.Upon starting the py5 kernel, it immediately dies. When using the created conda env as the kernel, the kernel dies after importing
py5
.I am using a mac M2. Jupyter gives the following error: