edt returning nan for large arrays (Windows)

[fbordignon]

I have a large 2D array that I want to use edt on, but it returns nan:

from edt import edt
import numpy as np
a[0,0] = 0
a = np.ones((2**16, 2))
edt(a)
array([[0.       , 1.       ],
       [1.       , 1.4142135],
       [2.       , 2.236068 ],
       ...,
       [      nan,       nan],
       [      nan,       nan],
       [      nan,       nan]], dtype=float32)

[william-silversmith]

Hi, I was unable to reproduce this issue. Can you provide some more information about your python version, platform, and edt version?

The a[0,0] = 0 was located above the definition of a in the example, so I moved it to after. This is what my experiments look like:

a = np.ones((2**16, 2)) edt(a) array([[inf, inf], [inf, inf], [inf, inf], ..., [inf, inf], [inf, inf], [inf, inf]], dtype=float32) a[0,0] =0 edt(a) array([[0.0000000e+00, 1.0000000e+00], [1.0000000e+00, 1.4142135e+00], [2.0000000e+00, 2.2360680e+00], ..., [6.5533000e+04, 6.5533000e+04], [6.5534000e+04, 6.5534000e+04], [6.5535000e+04, 6.5535000e+04]], dtype=float32)

On Mon, Jan 17, 2022 at 2:02 PM Fernando Bordignon @.***> wrote:

I have a large 2D array that I want to use edt on, but it returns nan:

from edt import edtimport numpy as npa[0,0] = 0a = np.ones((2**16, 2))edt(a)array([[0. , 1. ], [1. , 1.4142135], [2. , 2.236068 ], ..., [ nan, nan], [ nan, nan], [ nan, nan]], dtype=float32)

— Reply to this email directly, view it on GitHub https://github.com/seung-lab/euclidean-distance-transform-3d/issues/37, or unsubscribe https://github.com/notifications/unsubscribe-auth/AATGQSONOT2YJRGRBG4CROTUWRRSVANCNFSM5MFMK2LA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you are subscribed to this thread.Message ID: @.***>

[william-silversmith]

I do have -ffast-math enabled which could potentially affect something on your architecture. Maybe I should disable that. Would you mind checking if compiling from source without -ffast-math helps if there aren't any other clues?

[marcioweck]

Hi, I work in the same project as @fbordignon and I got the same problem on my machine, even with a smaller array:

a = np.ones((46342, 2))
a[0, 0] = 0
np.any(np.isnan(edt(a)))
RuntimeWarning: invalid value encountered in sqrt
True

edt(a)
RuntimeWarning: invalid value encountered in sqrt
array([[0.0000000e+00, 1.0000000e+00],
       [1.0000000e+00, 1.4142135e+00],
       [2.0000000e+00, 2.2360680e+00],
       ...,
       [4.6339000e+04, 4.6339000e+04],
       [4.6340000e+04, 4.6340000e+04],
       [          nan,           nan]], dtype=float32)

In [10]: platform.python_version()
Out[10]: '3.9.9'

In [8]: platform.platform()
Out[8]: 'Windows-10-10.0.19042-SP0'

In [11]: platform.processor()
Out[11]: 'AMD64 Family 23 Model 8 Stepping 2, AuthenticAMD'

edt 2.1.1

I compiled following the steps in https://github.com/seung-lab/euclidean-distance-transform-3d/tree/master/python#recompiling-edtpyx . But before compiling I removed --fast-math and reduced to O2:

extra_compile_args = [
  '-std=c++11', '-O2', '-pthread'
]

But I still got the same error.

[william-silversmith]

Interesting, looks like I can reproduce this on Windows but not MacOS. It's gotta be some compiler specific thing. I'll look into it. Thanks for reporting!

On Mon, Jan 17, 2022 at 7:50 PM Márcio Weck @.***> wrote:

Hi, I work in the same project as @fbordignon https://github.com/fbordignon and I got the same problem on my machine, even with a smaller array:

a = np.ones((46342, 2))a[0, 0] = 0np.any(np.isnan(edt(a)))RuntimeWarning: invalid value encountered in sqrtTrue

edt(a)RuntimeWarning: invalid value encountered in sqrtarray([[0.0000000e+00, 1.0000000e+00], [1.0000000e+00, 1.4142135e+00], [2.0000000e+00, 2.2360680e+00], ..., [4.6339000e+04, 4.6339000e+04], [4.6340000e+04, 4.6340000e+04], [ nan, nan]], dtype=float32)

In [10]: platform.python_version()Out[10]: '3.9.9' In [8]: platform.platform()Out[8]: 'Windows-10-10.0.19042-SP0' In [11]: platform.processor()Out[11]: 'AMD64 Family 23 Model 8 Stepping 2, AuthenticAMD' edt 2.1.1

I compiled following the steps in https://github.com/seung-lab/euclidean-distance-transform-3d/tree/master/python#recompiling-edtpyx . But before compiling I removed --fast-math and reduced to O2:

extra_compile_args = [ '-std=c++11', '-O2', '-pthread' ]

But I stil got the same error.

— Reply to this email directly, view it on GitHub https://github.com/seung-lab/euclidean-distance-transform-3d/issues/37#issuecomment-1014979309, or unsubscribe https://github.com/notifications/unsubscribe-auth/AATGQSKSZJ3NHBIQEC4ATHTUWS2OHANCNFSM5MFMK2LA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you commented.Message ID: @.*** com>

[william-silversmith]

Interesting, 46342 is almost exactly 15.5 bits to represent. I wonder if that means something... something getting rounded up to the next bit then going out of range.

[william-silversmith]

Hello, I think I've found the issue! On MVCC the operation sq(i - v[k]) was squaring the integers before casting to floats and overflowed. On clang/g++ they were cast to floats and then squared. I'm making a fix in #39

[fbordignon]

@william-silversmith thanks for the fix and sorry for the mix-up with the order of the statements and not specifying the platform. Regards