Open chrisroat opened 3 years ago
chrisroat
commented
3 years ago I see many of the following messages in the logs. I'm trying to run with fewer cores so I can disentangle where they are coming from.
munmap_chunk(): invalid pointer
Aborted (core dumped)With my dataset, it seems that using 5 cores or less avoids the invalid pointer / core dump issue.
chrisroat
commented
3 years ago I created a debug build and loaded a dumped core. The stack trace is below, where the deletion of the buf pointer is causing an issue. From the debugger I see that this is happening for the first loop through channels (c=1).
The relevant lines in malloc.c that are catching there error:
size = chunksize (p);
...
if (__builtin_expect ((uintptr_t) p > (uintptr_t) -size, 0)
|| __builtin_expect (misaligned_chunk (p), 0)) #4 0x00007f09e5715cac in _int_free (av=<optimized out>, p=<optimized out>, have_lock=0) at malloc.c:4173
4173 malloc_printerr ("free(): invalid pointer");
#5 0x0000563517c76dba in UnstitchedVolume::loadSubvolume_to_UINT8 (this=0x563519df97f0, V0=0, V1=512, H0=1024, H1=1536, D0=0, D1=64,
channels=0x7ffc840c5338, ret_type=0) at /home/croat/TeraStitcher/src/imagemanager/UnstitchedVolume.cpp:1159
1159 delete []buf;
#6 0x0000563517be93c0 in VolumeConverter::generateTilesVaa3DRawMC (this=0x7ffc840c5300, output_path="/workdir/stitched_tiff", ch_dir="",
resolutions=0x7ffc840c57c8, block_height=512, block_width=512, block_depth=256, method=1, isotropic=false, show_progress_bar=false,
saved_img_format=0x563517da05f5 "Tiff3D", saved_img_depth=16, frame_dir="", par_mode=true, fixed_tiling=true)
at /home/croat/TeraStitcher/src/utils/volumeconverter/VolumeConverter.cpp:3648
3648 ubuffer[0] = volume->loadSubvolume_to_UINT8(V0,V1,H0,H1,(int)z,(z+z_max_res <= D1) ? (int)(z+z_max_res) : D1,&channels,iim::NATIVE_RTYPE);
#7 0x0000563517bd3b1d in vcDriver (vPtr=0x0, src_root_dir="/workdir/stitched_tiff/xml_import.xml", dst_root_dir="/workdir/stitched_tiff",
src_format="TIFF (unstitched, 3D)", dst_format="TIFF (tiled, 4D)", img_depth=0, resolutions=0x7ffc840c57c8, chanlist="", ch_dir="", mdata_fname="",
slice_depth=256, slice_height=512, slice_width=512, downsamplingFactor=1, halving_method=1, libtiff_rowsPerStrip=1, libtiff_uncompressed=false,
libtiff_bigtiff=false, show_progress_bar=true, isotropic=false, V0=0, V1=512, H0=1024, H1=1536, D0=0, D1=251, timeseries=false, makeDirs=false,
metaData=false, parallel=true, fixed_tiling=true, outFmt="", nbits=0) at /home/croat/TeraStitcher/src/utils/volumeconverter/VolumeConverter.cpp:335
335 vc.generateTilesVaa3DRawMC(dst_root_dir.c_str(),ch_dir,resolutions,
#8 0x0000563517bcf293 in main (argc=17, argv=0x7ffc840c59c8) at /home/croat/TeraStitcher/src/utils/teraconverter/teraconverter.cpp:212
212 vcDriver(
[Edited after I realized the missing files weren't a problem -- the data were all zeros]
I have run terastitcher 1.11.11, and also compiled at HEAD, with --fixed_tiling to generated tiff stacks, but some data seems to be missing: a cross-shape of data is all zeros
Looking at the data itself, there is a missing swath of x starting at 1024 pixels from the left edge, and of y starting 1536 from the top.
The solution to the stitching looks reasonable. The relevant xml files are included in tstitch_missing_data.zip
The command I then use to do the conversion, which is converting from input tiffs that are 1024x1024: