marcdegraef
commented
1 year ago Thanks for letting us know. I will update the code to include the Oxford HDF5 format... It would help if I could have access to a small example HDF5 file.
Regards, Marc.
On 10/21/22 6:34 AM, Philip-Go wrote:
Hi everybody,
I'm using EMsoft 5.0 in a nightly-build copiled package and was using it with OxfordBinaries .ebsp from AZtec versions 3.1 to 4.3. In that combo, the DI went smoothly and resulted great mappings of nano-crystalline metals and deformed microstructures.
However, recently one of our EBSD systems got an update of the AZtec software, from 3.1 to AZtec 6.0. With this update some changes in the OxfordBinary file must have occured, because EMEBSDDIpreview can't access the .ebsp any more. This can be related to the HDF5 file, because now the OxfordHDF5 file format .h5oina contains the uncompressed Kikuchi patternswhich are nicely accessible, for example in Python via h5py and matplotlib. The path to access them is '1/EBSD/Data/Processed Patterns'. Unfortunately, the EMsoft inputtype OxfordHDF leads to an hard coded break. From the code I could find out that as of 2019 the OxfordHDF5 did not contain any Kikuchi patterns, so no routine was implemented.
However, now the Kikuchi patterns are there. I'm not that familiar with HDF5 yet, but I asumed that it's advantage is data accessability independent of the hardware manufacturer. So I tried to run EMEBSDDIpreview on the OxfordHDF with inputtype BrukerHDF, which failed, and TSLHDF, which then was succesfull! Of course I tried the dictionary indexing EMEBSDDI on the OxfordHDF with the TSLHDF, which started out nicely. But eventually, the programm failed during Pattern Preprocessing inbetween row 260 and 265 of 500 rows with the error:
|Completed row 260 of 500 rows| |Error code : -1| |returned by routine HDF_readHyperslabIntegerArray3D:h5dread_f:Processed Patterns|
Interestingly, when I define a ROI from 1st to 250th column, the DI programm runs smoothly and returns a proper .ctf file:
|Completed row 250 of 250 rows| |-> experimental patterns preprocessed| |Number of experimental patterns processed per second : 34883.7| |-> computing Average Dot Product map (ADP)| |-> Number of threads set to 8| |Oct 21 2022 11:56:12.770 AM| |min./max. dot product = 0.768612 / 0.867969; 1.2% complete| |min./max. dot product = 0.768987 / 0.870098; 2.4% complete| etc...
The dataset contains 375000 datapoint/kikuchi patterns in 750 columns and 500 rows, Kikuchi patterns have a size of 168x128 px.
My question is know, if someone has implemented but not released a routine for reading OxfordHDF files containing Kikuchi patterns or maybe has an idea, why the TSLHDF set only works until a specific row. I am neither experienced with HDF5 nor Fortran. But following the error-code, the error occurs in the h5read_f line in the HDF_readHyperslabIntegerArray3D procedure.
— Reply to this email directly, view it on GitHub https://github.com/EMsoft-org/EMsoft/issues/164, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB26VWH2P6CAH5UTUYMX4ILWEJWSZANCNFSM6AAAAAARLBDWBY. You are receiving this because you are subscribed to this thread.Message ID: @.***>
-- Marc De Graef Professor Department of Materials Science and Engineering 130 Roberts Engineering Hall Carnegie Mellon University 5000 Forbes Avenue Pittsburgh, PA 15213-3890 USA
Ph: (412) 268-8527 Fx: (412) 268-7596 @.*** web:http://materials.cmu.edu/degraef
Admin. Asst.: Marygrace Antkowski Phone: (412) 268-7240
hakonanes
Philip-Go
Is it simply due to another plotting convention of the kikuchi pattern? For experimental patterns from AZtec, such as here, the origin is in the upper left corner, viewing from the detector.
drowenhorst-nrl
Note that there are several regions where the computation did not produce the expected result. This is likely due to bad data in the file, but I'll have to take a closer look at that. Interestingly, the EMsoft code continues to read the file despite numerous HDF5 errors being reported. I will try to run the Dictionary Indexing code as well, just to see what comes out. More later.
Marc.
Hi everybody,
I'm using EMsoft 5.0 in a nightly-build copiled package and was using it with OxfordBinaries .ebsp from AZtec versions 3.1 to 4.3. In that combo, the DI went smoothly and resulted great mappings of nano-crystalline metals and deformed microstructures.
However, recently one of our EBSD systems got an update of the AZtec software, from 3.1 to AZtec 6.0. With this update some changes in the OxfordBinary file must have occured, because EMEBSDDIpreview can't access the .ebsp any more. This can be related to the HDF5 file, because now the OxfordHDF5 file format .h5oina contains the uncompressed Kikuchi patternswhich are nicely accessible, for example in Python via h5py and matplotlib. The path to access them is '1/EBSD/Data/Processed Patterns'. Unfortunately, the EMsoft inputtype OxfordHDF leads to an hard coded break. From the code I could find out that as of 2019 the OxfordHDF5 did not contain any Kikuchi patterns, so no routine was implemented.
However, now the Kikuchi patterns are there. I'm not that familiar with HDF5 yet, but I asumed that it's advantage is data accessability independent of the hardware manufacturer. So I tried to run EMEBSDDIpreview on the OxfordHDF with inputtype BrukerHDF, which failed, and TSLHDF, which then was succesfull! Of course I tried the dictionary indexing EMEBSDDI on the OxfordHDF with the TSLHDF, which started out nicely. But eventually, the programm failed during Pattern Preprocessing inbetween row 260 and 265 of 500 rows with the error:
Completed row 260 of 500 rowsError code : -1returned by routine HDF_readHyperslabIntegerArray3D:h5dread_f:Processed PatternsInterestingly, when I define a ROI from 1st to 250th column, the DI programm runs smoothly and returns a proper .ctf file:
Completed row 250 of 250 rows-> experimental patterns preprocessedNumber of experimental patterns processed per second : 34883.7-> computing Average Dot Product map (ADP)-> Number of threads set to 8Oct 21 2022 11:56:12.770 AMmin./max. dot product = 0.768612 / 0.867969; 1.2% completemin./max. dot product = 0.768987 / 0.870098; 2.4% completeetc...The dataset contains 375000 datapoint/kikuchi patterns in 750 columns and 500 rows, Kikuchi patterns have a size of 168x128 px.
My question is know, if someone has implemented but not released a routine for reading OxfordHDF files containing Kikuchi patterns or maybe has an idea, why the TSLHDF set only works until a specific row. I am neither experienced with HDF5 nor Fortran. But following the error-code, the error occurs in the h5read_f line in the HDF_readHyperslabIntegerArray3D procedure.
edit: what I maybe need to mention: if I chose the ROI from row 250 to 500 (lower half), the pattern pre processing breaks after 10 lines, probably corresponding to the exact same line/pattern where processing of the whole data set fails.
Thank you for any ideas or comments