rcatwood
commented
1 year ago I can import the file created by the following Python script: (3.7, Anaconda )
200,200,200,200 4D array, uint16 data type, just containing ‘ones’ (script code below)
It does not import in the ‘default plugin’ as it’s too big, but it does import into the PSI plugin. Maybe you accidentally created some special data type, can you make the data into ‘np.uint16’ before writing in an HDF5 ? Is the PSI plugin expected to read netcdf4 files at all? I don’t see mention of such files in the documentation.
But …
it does not work when the memory is restricted in Imagej even if ‘virtual stack’ is ticked, should I expect this to work with 4D?
it fails with memory error for a bigger 4d array (300,300,300,300) with ‘null pointer exception’
However , though our software team has made reconstructed time-series tomography output as 4d HDF5 datasets, I don’t really like it :/ -- I’d rather have a series of 3-d datasets. (what do you do at Tomcat ? )
1 import numpy as np 2 import h5py 3 4 aaa=np.ones((200,200,200,200),dtype=np.uint16) 5 fff=h5py.File("fourdee_med.h5","w") 6 fff.create_dataset("/entry/data",data=aaa ) 7 fff.close() 8
From: Pescatore23 @.> Sent: 02 December 2022 10:12 To: paulscherrerinstitute/ch.psi.imagej.hdf5 @.> Cc: Subscribed @.***> Subject: [paulscherrerinstitute/ch.psi.imagej.hdf5] hdf.hdf5lib.exceptions.HDF5Exception: Invalid int size (Issue #8)
Hi,
I convert the reconstructed tiff-Stacks to a netcdf4-file containing the image data as 4D-uint16-array using python with xarray and numpy. Trying to import it with hdf5 vibez into ImageJ, I get a "Length is too large" error (roughly 13 billion). It works for smaller dataset.
However, with the psi-hdf5-loader on ra in the "TOMCAT"-Fiji and locally I get following error "Invalid int size":
Dec 02, 2022 10:57:04 AM ch.psi.imagej.hdf5.HDF5Reader open WARNING: Error while opening: /das/home/fische_r/DASCOELY/Data10/disk1/T_3III_scan_04_test.nc java.lang.Exception: Failed to read scalar dataset: Invalid int size at hdf.object.h5.H5ScalarDS.read(H5ScalarDS.java:820) at ch.psi.imagej.hdf5.HDF5Reader.open(HDF5Reader.java:221) at ch.psi.imagej.hdf5.HDF5Reader.open(HDF5Reader.java:51) at ch.psi.imagej.hdf5.HDF5Reader.run(HDF5Reader.java:36) at ij.IJ.runUserPlugIn(IJ.java:241) at ij.IJ.runPlugIn(IJ.java:204) at ij.Executer.runCommand(Executer.java:151) at ij.Executer.run(Executer.java:69) at java.lang.Thread.run(Thread.java:748) Caused by: hdf.hdf5lib.exceptions.HDF5Exception: Invalid int size at hdf.object.h5.H5Utils.getTotalSelectedSpacePoints(H5Utils.java:118) at hdf.object.h5.H5ScalarDS.scalarDatasetCommonIO(H5ScalarDS.java:902) at hdf.object.h5.H5ScalarDS.read(H5ScalarDS.java:816) ... 8 more
Is this somethin you have encountered before and have a workaround? I will try to play around with the dataformat and datasize to find the issue and a solution.
Best, Robert @.**@.>
— Reply to this email directly, view it on GitHubhttps://github.com/paulscherrerinstitute/ch.psi.imagej.hdf5/issues/8, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AC4TESVLQNPOUCLKX6P3AATWLHDOFANCNFSM6AAAAAASRX4LUM. You are receiving this because you are subscribed to this thread.Message ID: @.**@.>>
-- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
Pescatore23
Hi,
I convert the reconstructed tiff-Stacks to a netcdf4-file containing the image data as 4D-uint16-array using python with xarray and numpy. Trying to import it with hdf5 vibez into ImageJ, I get a "Length is too large" error (roughly 13 billion). It works for smaller dataset.
However, with the psi-hdf5-loader on ra in the "TOMCAT"-Fiji and locally I get following error "Invalid int size":
Dec 02, 2022 10:57:04 AM ch.psi.imagej.hdf5.HDF5Reader open WARNING: Error while opening: /das/home/fische_r/DASCOELY/Data10/disk1/T_3III_scan_04_test.nc java.lang.Exception: Failed to read scalar dataset: Invalid int size at hdf.object.h5.H5ScalarDS.read(H5ScalarDS.java:820) at ch.psi.imagej.hdf5.HDF5Reader.open(HDF5Reader.java:221) at ch.psi.imagej.hdf5.HDF5Reader.open(HDF5Reader.java:51) at ch.psi.imagej.hdf5.HDF5Reader.run(HDF5Reader.java:36) at ij.IJ.runUserPlugIn(IJ.java:241) at ij.IJ.runPlugIn(IJ.java:204) at ij.Executer.runCommand(Executer.java:151) at ij.Executer.run(Executer.java:69) at java.lang.Thread.run(Thread.java:748) Caused by: hdf.hdf5lib.exceptions.HDF5Exception: Invalid int size at hdf.object.h5.H5Utils.getTotalSelectedSpacePoints(H5Utils.java:118) at hdf.object.h5.H5ScalarDS.scalarDatasetCommonIO(H5ScalarDS.java:902) at hdf.object.h5.H5ScalarDS.read(H5ScalarDS.java:816) ... 8 more
Is this somethin you have encountered before and have a workaround? I will try to play around with the dataformat and datasize to find the issue and a solution.
Best, Robert robert.fischer@psi.ch