search

Image-Py / sknw

build net work from skeleton image (2D-3D)
BSD 3-Clause "New" or "Revised" License
222 stars 47 forks source link

No bounds checking on fill/trace buf access #1

Open Alex-Richman opened 5 years ago

Alex-Richman commented 5 years ago

I'm having some trouble processing big 3d volumes using the library. The cause seems to be a lack of bounds checking on buf access in fill() and trace(), buf being statically sized. Due to the numba.jit usage it manifests as a segfault due to the out-of-bounds write.

Here's a PR to add that bounds checking, and allows caller specification of the fixed buffer size to build_sknw(): https://github.com/yxdragon/sknw/pull/2

I guess it could also be dynamically resized, but I'm not familiar with numba.jit flavoured python so I'll leave that.

yxdragon commented 5 years ago

I am sorry, so long time not watch this repo, your image is very large? are you sure the image is a skeleton image? I think 131072 is enough to contain a edge. maybe I should malloc 1/10 lenght of the 3d image. I think 1/10 size is enough for any edge.

numba cound not resize an array dynamically, we can only np.zeros(...) a longer one, and copy data in.

Alex-Richman commented 5 years ago

Yeah the images we're looking at are very big, up to 1024x1024x1024 with lots of connectivity in the skeleton.

I don't have the maximum edge lengths we acquired to hand, but I imagine some factor of the total image size would work well as an initialiser for the edge buffer.

Main thing I needed to fix locally though was the breaks preventing overflowing of that edge buffer, since the buffer overrun in numba causes a SEGFAULT not a clean python exception.

Thanks!

yxdragon commented 5 years ago

can you export your data as png sequence and email to me? I think 1024^3 is not large, 131042 = 1024 * 128. may be there is something wrong other where!

victor000000 commented 3 years ago

131042 8(int64) = 1M 131042 1024 * 8 = 1G

victor000000 commented 3 years ago

I found that in build_sknw(ske, multi=False, iso=True, ring=True) if ske is of type np.uint8, then if node num > 255 - 10, node_num will be 0, that is a bug image

so if node num > 255, we shoud make sure ske is of type np.int64