search

apache / mxnet

Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Scala, Go, Javascript and more
https://mxnet.apache.org
Apache License 2.0
20.77k stars 6.8k forks source link

Image transforms sometimes produce incorrect result #18911

Closed rogerdettloff closed 4 years ago

rogerdettloff commented 4 years ago

Description

I've noticed that image transforms like npx.image.random_flip_left_right and npx.image.random_flip_top_bottom sometimes produce an incorrect result--although, sometimes they work correctly. Please see my example below...

Error Message

No error message is produced.

To Reproduce

>>> from mxnet import np, npx
>>> npx.set_np()

>>> img=np.arange(11*9).reshape(11,9,1)

>>> img[:,:,0]  # have a look at our test image for reference.
array([[ 0.,  1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.],
       [ 9., 10., 11., 12., 13., 14., 15., 16., 17.],
       [18., 19., 20., 21., 22., 23., 24., 25., 26.],
       [27., 28., 29., 30., 31., 32., 33., 34., 35.],
       [36., 37., 38., 39., 40., 41., 42., 43., 44.],
       [45., 46., 47., 48., 49., 50., 51., 52., 53.],
       [54., 55., 56., 57., 58., 59., 60., 61., 62.],
       [63., 64., 65., 66., 67., 68., 69., 70., 71.],
       [72., 73., 74., 75., 76., 77., 78., 79., 80.],
       [81., 82., 83., 84., 85., 86., 87., 88., 89.],
       [90., 91., 92., 93., 94., 95., 96., 97., 98.]])

>>> img_flip_lr = npx.image.random_flip_left_right(img)
>>> img_flip_lr[:,:,0]  # correctly flipped image
array([[ 8.,  7.,  6.,  5.,  4.,  3.,  2.,  1.,  0.],
       [17., 16., 15., 14., 13., 12., 11., 10.,  9.],
       [26., 25., 24., 23., 22., 21., 20., 19., 18.],
       [35., 34., 33., 32., 31., 30., 29., 28., 27.],
       [44., 43., 42., 41., 40., 39., 38., 37., 36.],
       [53., 52., 51., 50., 49., 48., 47., 46., 45.],
       [62., 61., 60., 59., 58., 57., 56., 55., 54.],
       [71., 70., 69., 68., 67., 66., 65., 64., 63.],
       [80., 79., 78., 77., 76., 75., 74., 73., 72.],
       [89., 88., 87., 86., 85., 84., 83., 82., 81.],
       [98., 97., 96., 95., 94., 93., 92., 91., 90.]])

>>> img_flip_lr = npx.image.random_flip_left_right(img)  # try again...
>>> img_flip_lr[:,:,0]  # sometimes I get a messed-up result...
array([[ 8.0000000e+00,  7.0000000e+00,  6.0000000e+00,  5.0000000e+00,
         4.4816047e-38,  3.0000000e+00,  2.0000000e+00,  1.0000000e+00,
         0.0000000e+00],
       [ 1.7000000e+01,  1.6000000e+01,  1.5000000e+01,  1.4000000e+01,
         5.9144397e-38,  1.2000000e+01,  1.1000000e+01,  1.0000000e+01,
         9.0000000e+00],
       [ 2.6000000e+01,  2.5000000e+01,  2.4000000e+01,  2.3000000e+01,
         7.3440759e-34,  2.1000000e+01,  2.0000000e+01,  1.9000000e+01,
         1.8000000e+01],
       [ 3.5000000e+01,  3.4000000e+01,  3.3000000e+01,  3.2000000e+01,
         4.8542180e-32,  3.0000000e+01,  2.9000000e+01,  2.8000000e+01,
         2.7000000e+01],
       [ 4.4000000e+01,  4.3000000e+01,  4.2000000e+01,  4.1000000e+01,
        -5.2694353e-17,  3.9000000e+01,  3.8000000e+01,  3.7000000e+01,
         3.6000000e+01],
       [ 5.3000000e+01,  5.2000000e+01,  5.1000000e+01,  5.0000000e+01,
         9.3095263e-41,  4.8000000e+01,  4.7000000e+01,  4.6000000e+01,
         4.5000000e+01],
       [ 6.2000000e+01,  6.1000000e+01,  6.0000000e+01,  5.9000000e+01,
         4.6286172e-38,  5.7000000e+01,  5.6000000e+01,  5.5000000e+01,
         5.4000000e+01],
       [ 7.1000000e+01,  7.0000000e+01,  6.9000000e+01,  6.8000000e+01,
         1.9254146e-37,  6.6000000e+01,  6.5000000e+01,  6.4000000e+01,
         6.3000000e+01],
       [ 8.0000000e+01,  7.9000000e+01,  7.8000000e+01,  7.7000000e+01,
         1.8514722e-37,  7.5000000e+01,  7.4000000e+01,  7.3000000e+01,
         7.2000000e+01],
       [ 8.9000000e+01,  8.8000000e+01,  8.7000000e+01,  8.6000000e+01,
         1.0652919e-38,  8.4000000e+01,  8.3000000e+01,  8.2000000e+01,
         8.1000000e+01],
       [ 9.8000000e+01,  9.7000000e+01,  9.6000000e+01,  9.5000000e+01,
         2.3658196e-37,  9.3000000e+01,  9.2000000e+01,  9.1000000e+01,
         9.0000000e+01]])

The example above does not always produce messed-up results for me, but I found that it seems more frequently messed-up when I Compose several sequential transformations, like this:

>>> from mxnet.gluon.data.vision import transforms
>>> transformer = transforms.Compose([
...         transforms.RandomFlipLeftRight(),
...         transforms.RandomFlipTopBottom()
...     ])
>>> transformer(img)[:,:,0]
array([[8.0000000e+00, 7.0000000e+00, 6.0000000e+00, 5.0000000e+00,
        6.3878950e-04, 3.0000000e+00, 2.0000000e+00, 1.0000000e+00,
        0.0000000e+00],
       [1.7000000e+01, 1.6000000e+01, 1.5000000e+01, 1.4000000e+01,
        4.7425812e+30, 1.2000000e+01, 1.1000000e+01, 1.0000000e+01,
        9.0000000e+00],
       [2.6000000e+01, 2.5000000e+01, 2.4000000e+01, 2.3000000e+01,
        1.7657325e+22, 2.1000000e+01, 2.0000000e+01, 1.9000000e+01,
        1.8000000e+01],
       [3.5000000e+01, 3.4000000e+01, 3.3000000e+01, 3.2000000e+01,
        7.5839171e+31, 3.0000000e+01, 2.9000000e+01, 2.8000000e+01,
        2.7000000e+01],
       [4.4000000e+01, 4.3000000e+01, 4.2000000e+01, 4.1000000e+01,
        3.3809470e+03, 3.9000000e+01, 3.8000000e+01, 3.7000000e+01,
        3.6000000e+01],
       [5.3000000e+01, 5.2000000e+01, 5.1000000e+01, 5.0000000e+01,
        2.2267046e-15, 4.8000000e+01, 4.7000000e+01, 4.6000000e+01,
        4.5000000e+01],
       [6.2000000e+01, 6.1000000e+01, 6.0000000e+01, 5.9000000e+01,
        7.5033516e+28, 5.7000000e+01, 5.6000000e+01, 5.5000000e+01,
        5.4000000e+01],
       [7.1000000e+01, 7.0000000e+01, 6.9000000e+01, 6.8000000e+01,
        4.7427375e+30, 6.6000000e+01, 6.5000000e+01, 6.4000000e+01,
        6.3000000e+01],
       [8.0000000e+01, 7.9000000e+01, 7.8000000e+01, 7.7000000e+01,
        1.4589531e-19, 7.5000000e+01, 7.4000000e+01, 7.3000000e+01,
        7.2000000e+01],
       [8.9000000e+01, 8.8000000e+01, 8.7000000e+01, 8.6000000e+01,
        4.7427375e+30, 8.4000000e+01, 8.3000000e+01, 8.2000000e+01,
        8.1000000e+01],
       [9.8000000e+01, 9.7000000e+01, 9.6000000e+01, 9.5000000e+01,
        1.9209545e+31, 9.3000000e+01, 9.2000000e+01, 9.1000000e+01,
        9.0000000e+01]])

>>> transformer(img)[:,:,0]  # try again, and sometimes it works okay...
array([[90., 91., 92., 93., 94., 95., 96., 97., 98.],
       [81., 82., 83., 84., 85., 86., 87., 88., 89.],
       [72., 73., 74., 75., 76., 77., 78., 79., 80.],
       [63., 64., 65., 66., 67., 68., 69., 70., 71.],
       [54., 55., 56., 57., 58., 59., 60., 61., 62.],
       [45., 46., 47., 48., 49., 50., 51., 52., 53.],
       [36., 37., 38., 39., 40., 41., 42., 43., 44.],
       [27., 28., 29., 30., 31., 32., 33., 34., 35.],
       [18., 19., 20., 21., 22., 23., 24., 25., 26.],
       [ 9., 10., 11., 12., 13., 14., 15., 16., 17.],
       [ 0.,  1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.]])

Environment

----------Python Info----------
Version      : 3.7.6
Compiler     : MSC v.1916 64 bit (AMD64)
Build        : ('tags/v3.7.6:43364a7ae0', 'Dec 19 2019 00:42:30')
Arch         : ('64bit', 'WindowsPE')
------------Pip Info-----------
Version      : 20.2.1
Directory    : D:\Projects\Combinati\dPCR-analysis\env-mxnet\lib\site-packages\pip
----------MXNet Info-----------
Version      : 1.6.0
Directory    : D:\Projects\Combinati\dPCR-analysis\env-mxnet\lib\site-packages\mxnet
Num GPUs     : 0
Hashtag not found. Not installed from pre-built package.
----------System Info----------
Platform     : Windows-10-10.0.18362-SP0
system       : Windows
node         : roger-5577
release      : 10
version      : 10.0.18362
----------Hardware Info----------
machine      : AMD64
processor    : Intel64 Family 6 Model 158 Stepping 9, GenuineIntel
Name                                       
Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz  
----------Network Test----------
Setting timeout: 10
Timing for MXNet: https://github.com/apache/incubator-mxnet, DNS: 0.0080 sec, LOAD: 1.1030 sec.
Timing for GluonNLP GitHub: https://github.com/dmlc/gluon-nlp, DNS: 0.0010 sec, LOAD: 0.5180 sec.
Timing for GluonNLP: http://gluon-nlp.mxnet.io, DNS: 0.0861 sec, LOAD: 0.3459 sec.
Timing for D2L: http://d2l.ai, DNS: 0.0520 sec, LOAD: 0.2090 sec.
Timing for D2L (zh-cn): http://zh.d2l.ai, DNS: 0.0190 sec, LOAD: 0.2664 sec.
Timing for FashionMNIST: https://repo.mxnet.io/gluon/dataset/fashion-mnist/train-labels-idx1-ubyte.gz, DNS: 0.0870 sec, LOAD: 0.4820 sec.
Timing for PYPI: https://pypi.python.org/pypi/pip, DNS: 0.0100 sec, LOAD: 0.5760 sec.
Error open Conda: https://repo.continuum.io/pkgs/free/, HTTP Error 403: Forbidden, DNS finished in 0.006048440933227539 sec.
github-actions[bot] commented 4 years ago

Welcome to Apache MXNet (incubating)! We are on a mission to democratize AI, and we are glad that you are contributing to it by opening this issue. Please make sure to include all the relevant context, and one of the @apache/mxnet-committers will be here shortly. If you are interested in contributing to our project, let us know! Also, be sure to check out our guide on contributing to MXNet and our development guides wiki.

szha commented 4 years ago

Thanks for reporting this. @zhreshold mind taking a look?

zhreshold commented 4 years ago

confirmed, will update once I rooted the cause