search

oracle / graal

GraalVM compiles Java applications into native executables that start instantly, scale fast, and use fewer compute resources 🚀
https://www.graalvm.org
Other
20.34k stars 1.63k forks source link

[GR-46852] Should Graal/mx.sdk recongize JDKs with Info.plist in binary format? #6825

Open navyxliu opened 1 year ago

navyxliu commented 1 year ago

Feature request

Please include the following information:

Is your feature request related to a problem? Please describe. I can't use some other JDK binary distributions to bootstrap graalvm on MacOS. eg. Corretto-17 also supports CI.

here are logs I got. It's passed JVMCIVersionCheck.java but fails in _patch_darwin_jdk of mx.sdk. 

➜  compiler git:(master) mx -v build
OS variant detected: none
Component Polyglot Launcher does not specify dependencies
env MX_HOME=/Users/xxinliu/Devel/mx MX__SUITEMODEL=sibling MX_SUBPROCESS_COMMAND_FILE=/var/folders/4r/g1rh5y0j74vgfvk_bp8tq8l00000gr/T/mx_subprocess_command.6fwwa0ll \
/Library/Java/JavaVirtualMachines/amazon-corretto-17.jdk/Contents/Home/bin/java -Xlog:disable /Users/xxinliu/Devel/graal/compiler/src/jdk.internal.vm.compiler/src/org/graalvm/compiler/hotspot/JVMCIVersionCheck.java
Component Disassembler does not specify dependencies
Cannot find 'ni' component; removing macros: ['Truffle Macro']
Components: ['Polyglot Launcher', 'Graal SDK', 'GraalVM Launcher Common', 'LLVM.org toolchain', 'Truffle API', 'Truffle', 'Truffle DSL Processor', 'Truffle NFI', 'Truffle NFI LIBFFI', 'ICU4J', 'ANTLR4', 'TRegex', 'GraalVM compiler', 'Disassembler']
No dist name for ['antlr4', 'cmp', 'dis', 'icu4j', 'llp', 'nfi', 'nfi-libffi', 'poly', 'rgx', 'sdk', 'sdkl', 'tfl', 'tfla', 'tflp']
Cannot find 'ni' component; removing macros: ['Truffle Macro']
Components: ['Polyglot Launcher', 'Graal SDK', 'GraalVM Launcher Common', 'LLVM.org toolchain', 'Truffle API', 'Truffle', 'Truffle DSL Processor', 'Truffle NFI', 'Truffle NFI LIBFFI', 'ICU4J', 'ANTLR4', 'TRegex', 'GraalVM compiler', 'Disassembler']
Traceback (most recent call last):
  File "/Users/xxinliu/Devel/mx/mx_enter.py", line 44, in <module>
    mx._main_wrapper()
  File "/Users/xxinliu/Devel/mx/mx.py", line 18385, in _main_wrapper
    main()
  File "/Users/xxinliu/Devel/mx/mx.py", line 18292, in main
    primary_suite().recursive_post_init()
  File "/Users/xxinliu/Devel/mx/mx.py", line 2580, in recursive_post_init
    self.visit_imports(Suite._init_metadata_visitor)
  File "/Users/xxinliu/Devel/mx/mx.py", line 2627, in visit_imports
    visitor(self, suite_import, **extra_args)
  File "/Users/xxinliu/Devel/mx/mx.py", line 2593, in _init_metadata_visitor
    imported_suite.visit_imports(imported_suite._init_metadata_visitor)
  File "/Users/xxinliu/Devel/mx/mx.py", line 2627, in visit_imports
    visitor(self, suite_import, **extra_args)
  File "/Users/xxinliu/Devel/mx/mx.py", line 2594, in _init_metadata_visitor
    imported_suite._init_metadata()
  File "/Users/xxinliu/Devel/mx/mx.py", line 2611, in _init_metadata
    self._load_metadata()
  File "/Users/xxinliu/Devel/mx/mx.py", line 2860, in _load_metadata
    self.mx_register_dynamic_suite_constituents(_register_project, _register_distribution)
  File "/Users/xxinliu/Devel/graal/sdk/mx.sdk/mx_sdk.py", line 153, in mx_register_dynamic_suite_constituents
    mx_sdk_vm_impl.mx_register_dynamic_suite_constituents(register_project, register_distribution)
  File "/Users/xxinliu/Devel/graal/sdk/mx.sdk/mx_sdk_vm_impl.py", line 3057, in mx_register_dynamic_suite_constituents
    _final_graalvm_distribution = get_final_graalvm_distribution()
  File "/Users/xxinliu/Devel/graal/sdk/mx.sdk/mx_sdk_vm_impl.py", line 2867, in get_final_graalvm_distribution
    _final_graalvm_distribution = GraalVmLayoutDistribution(_graalvm_base_name, stage1=False)
  File "/Users/xxinliu/Devel/graal/sdk/mx.sdk/mx_sdk_vm_impl.py", line 861, in __init__
    super(GraalVmLayoutDistribution, self).__init__(
  File "/Users/xxinliu/Devel/graal/sdk/mx.sdk/mx_sdk_vm_impl.py", line 506, in __init__
    incl_list = _patch_darwin_jdk()
  File "/Users/xxinliu/Devel/graal/sdk/mx.sdk/mx_sdk_vm_impl.py", line 440, in _patch_darwin_jdk
    root = etreeParse(orig_info_plist)
  File "/Users/xxinliu/.pyenv/versions/3.9.4/lib/python3.9/xml/etree/ElementTree.py", line 1229, in parse
    tree.parse(source, parser)
  File "/Users/xxinliu/.pyenv/versions/3.9.4/lib/python3.9/xml/etree/ElementTree.py", line 580, in parse
    self._root = parser._parse_whole(source)
xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 1, column 8

Describe the solution you'd like. A clear and concise description of what you want to happen.

The rootcause is that current mx_sdk_vm_impl.py can't recognize 'info.plist' in binary format from some jdk distributions. I inspect the jdks provides by oraclelabs. They are all in xml format, so it's fine with them. 

$file ./amazon-corretto-17.jdk/Contents/Info.plist
./amazon-corretto-17.jdk/Contents/Info.plist: Apple binary property list

$hexdump -C ./amazon-corretto-17.jdk/Contents/Info.plist  | head -2
00000000  62 70 6c 69 73 74 30 30  dc 01 02 03 04 05 06 07  |bplist00........|
00000010  08 09 0a 0b 0c 0d 0e 0f  10 11 12 13 14 15 16 17  |................|

There's a simple solution. plutil provided by MacOS can easily convert the binary plist to textual xml1. All we need to do is to wrap etreeParse(orig_info_plist) with a try/catch construct.

Describe who do you think will benefit the most. GraalVM users, GraalVM contributors, developers of libraries and frameworks which depend on GraalVM, or somebody else?

Graal developers would have more selection of jdks and oraclelabs could switch to the binary Info.plist freely.

Describe alternatives you've considered. A clear and concise description of any alternative solutions or features you've considered.

Instead of converting info.plist in memory, it's possible that mx.sdk script converts the binary plist to a textual file. It'll involve in a temporary file. It's more complex and less secure.

Additional context. Add any other context about the feature request here. For example, link to the relevant projects, documentation, standards.

Express whether you'd like to help contributing this feature If you'd like to contribute, please read the contribution guide.

yes.

oubidar-Abderrahim commented 1 year ago

Hi, Thank you for your suggestion, we'll take a look into it shortly

navyxliu commented 1 year ago

hi, @oubidar-Abderrahim I can contribute a patch to fix this issue and I have signed OCA. please let me know if you are interested in supporting binary Info.plist on MacOS.

toddjonker commented 2 months ago

@navyxliu Did you get a patch together? I'm facing the same issue.

navyxliu commented 1 month ago

Hi, @toddjonker

I didn't make a patch. it's because it's unclear Graal would take it or it's a package issue of jdk.

That being said, this problem is easy to work around. ./sdk/mx.sdk/mx_sdk_vm_impl.py expects the plist file in XML format. we could use plutil to convert plist file in Corretto to XML.

thanks, --lx