search

ome / bioformats

Bio-Formats is a Java library for reading and writing data in life sciences image file formats. It is developed by the Open Microscopy Environment. Bio-Formats is released under the GNU General Public License (GPL); commercial licenses are available from Glencoe Software.
https://www.openmicroscopy.org/bio-formats
GNU General Public License v2.0
377 stars 242 forks source link

May I have access to the data repository please ; I need this for GraalVM #4045

Open CGDogan opened 1 year ago

CGDogan commented 1 year ago

I'm working on running bioformats through GraalVM. GraalVM needs java to run with a testing agent because it can handle refleciton during runtime only if such classes were accessed beforehand. https://bio-formats.readthedocs.io/en/v6.14.0/developers/commit-testing.html suggests how to test and for this I need access to test images. export ANT_OPTS=-agentlib:native-image-agent=config-merge-dir=META-INF/native-image is just needed before running ant, but one should first set PATH and JAVA_HOME to GraalVM SDK. This will generate META-INF/ which will contain a record of reflection classes accessed.

dgault commented 1 year ago

Hi @CGDogan, Im afraid the full data repository is not available as many of the files are private. You can find the publicly available sample files at https://downloads.openmicroscopy.org/images/

The repository tests are generally used to check for any regressions in existing datasets, so I wouldn't be too concerned about running those tests yourself. These are tests that we run every night, so we can run the full test suite against any PR you have and report any issues back to you.

CGDogan commented 1 year ago

Thanks for the answer and no problem @dgault . Would you however be able to run the following on your side before you run all the tests? This will generate a list of classes at a new folder META-INF/ that you can inspect; this will not contain any data from the test files.

wget -q "https://github.com/graalvm/graalvm-ce-builds/releases/download/jdk-20.0.1/graalvm-community-jdk-20.0.1_linux-x64_bin.tar.gz" -O graal.tar.gz
tar -xzvf graal.tar.gz
rm graal.tar.gz
export JAVA_HOME=/graalvm-community-openjdk-20.0.1+9.1
export PATH="/graalvm-community-openjdk-20.0.1+9.1/bin:$PATH"
export ANT_OPTS=-agentlib:native-image-agent=config-merge-dir=META-INF/native-image

Or as a patch to the Dockerfile: https://github.com/ome/bioformats/compare/develop...CGDogan:graalmeta

This needs to be run before the ant command. Ant will then call native-image-agent, which was made for the GraalVM tool named native-image. Native-image requires knowing all classes that can be called with reflection, otherwise it crashes. So the best way to train native-image is to run this tracer before running tests with good coverage. The better the test coverage, the lower the likelihood of crashing.

Afterwards, I'll need components/test-suite/META-INF folder compressed. That would be excellent.

GraalVM would be a nice goal for v7.0.0 of this project, since it allows writing efficient bindings for many languages.

Please let me know what you think.