search

visit-dav / summer-projects

A place to manage activity on summer projects
1 stars 0 forks source link

Compile preliminary notes from look at CT data with VisIt on LLNL CZ #24

Closed markcmiller86 closed 2 years ago

markcmiller86 commented 2 years ago

Taking a preliminary look at MicroCT dataset with VisIt on LLN CZ systems.

Data is at path /p/lustre1/miller86/ctdata/MicroCT_Datasets/22Mondy_BigToe_10um/BigToe_Rec . I created two .imgvol (that is for Image-Volume and is a simple ascii file listing all the names of the image files to be treated as the "volume") files there. One was created by listing all the .tif files using ffind . -name '*_IR*.tif' > visit_vol.imgvol. The other was created by keeping only every 20th image from the first one using the command sed -n '1p;0~10p' visit_vol.imgvol > visit_vol_10xfewer_slices.imgvol.

The images are 4032x4032x8907. That is, each image is 4032x4032 and there are 8907 images.

To start to view it, I launched VisIt with the command...

/usr/gapps/visit/bin/visit -o visit_vol.imgvol

I've already started to do this on pascal.llnl.gov using 4-8 nodes and 144 to 288 processors. Iso-volume operator has cored on me but iso-contour plot seems to be working.

markcmiller86 commented 2 years ago

@corvette20 and @Magicat-0 once we get the file naming issue (above) worked out so we can properly order the images in a volue, one thing worth doing this week as you find some time to do so is to start VisIt on LLNL system, load the whole volume and then down-sample using VisIt's IndexSelect operator by say a factor of 5 in each dimension, export the resulting database to a new format and then it should be small enough to download to your laptops and play with there as well just to have something to work with that is small and local.

markcmiller86 commented 2 years ago

@wmondy I am wondering if you might have any insights regarding the image file naming pattern used in this data and how to properly order the images to form the volume? Is there maybe metadata buried within the .tif files that hint at this?

wmondy commented 2 years ago

Only thing I know is that the image# numbering sequence is automatically uploaded in order when using the CTan program.

Sent from my iPhone

On Jun 21, 2022, at 11:21 AM, markcmiller86 @.***> wrote:

 @wmondy I am wondering if you might have any insights regarding the image file naming pattern used in this data and how to properly order the images to form the volume? Is there maybe metadata buried within the .tif files that hint at this?

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

markcmiller86 commented 2 years ago

Ok I've updated the initial comment here to reflect most recent experiences and updated paths and filenames.

We have the image sequencing question resolved.

Two quick thigs for @corvette20 and @Magicat-0 is to use VisIt's IndexSelect operator using 5x5x5 downsample and the Export database to a format you can then copy to your laptops to use as a proxy there for a lot of your initial inquiry.

If questions or issues with proceeding with this, please ask.

markcmiller86 commented 2 years ago

Ok, it appears VisIt's IndexSelect operator has a bug in it. Using it is constantly crashing the engine. You can take another route (avoiding VisIt entirely for this step) using some image tools like ImageMagick (e.g. convert command).

Alternatively, I've played with 10x fewer slices (keeping only every 10th in visit_vol_10xfewer_slices.imgvol and done a contour plot of that on 8 nodes of pascal. I didn't think to grab a snapshot.