search

google-research / pix2seq

Pix2Seq codebase: multi-tasks with generative modeling (autoregressive and diffusion)
Apache License 2.0
857 stars 71 forks source link

multitask object detection result is wrong! #41

Open jiejie1993 opened 1 year ago

jiejie1993 commented 1 year ago

when I run the 'pix2seq_inference_multitask.ipynb', the detection result is below, the detection result is almostly 640,which might be wrong, why the problem happen? `(<tf.Tensor: shape=(1, 640, 640, 3), dtype=float32, numpy= array([[[[0.03137255, 0.03529412, 0.04313725], [0.01960784, 0.02352941, 0.03137255], [0.00784314, 0.01176471, 0.01960784], ..., [0.3 , 0.3 , 0.3 ], [0.3 , 0.3 , 0.3 ], [0.3 , 0.3 , 0.3 ]],

    [[0.00784314, 0.01176471, 0.01960784],
     [0.00784314, 0.01176471, 0.01960784],
     [0.00784314, 0.01176471, 0.01960784],
     ...,
     [0.3       , 0.3       , 0.3       ],
     [0.3       , 0.3       , 0.3       ],
     [0.3       , 0.3       , 0.3       ]],

    [[0.00784314, 0.01176471, 0.01960784],
     [0.00784314, 0.01176471, 0.01960784],
     [0.01176471, 0.01568627, 0.02352941],
     ...,
     [0.3       , 0.3       , 0.3       ],
     [0.3       , 0.3       , 0.3       ],
     [0.3       , 0.3       , 0.3       ]],

    ...,

    [[0.50980395, 0.5137255 , 0.5294118 ],
     [0.5254902 , 0.5294118 , 0.54509807],
     [0.50980395, 0.50980395, 0.5176471 ],
     ...,
     [0.3       , 0.3       , 0.3       ],
     [0.3       , 0.3       , 0.3       ],
     [0.3       , 0.3       , 0.3       ]],

    [[0.5568628 , 0.5686275 , 0.5882353 ],
     [0.5294118 , 0.53333336, 0.54901963],
     [0.5294118 , 0.5294118 , 0.5372549 ],
     ...,
     [0.3       , 0.3       , 0.3       ],
     [0.3       , 0.3       , 0.3       ],
     [0.3       , 0.3       , 0.3       ]],

    [[0.5803922 , 0.5921569 , 0.6117647 ],
     [0.5254902 , 0.5294118 , 0.54509807],
     [0.5254902 , 0.5294118 , 0.5372549 ],
     ...,
     [0.3       , 0.3       , 0.3       ],
     [0.3       , 0.3       , 0.3       ],
     [0.3       , 0.3       , 0.3       ]]]], dtype=float32)>, <tf.Tensor: shape=(1,), dtype=int32, numpy=array([230983], dtype=int32)>, <tf.Tensor: shape=(1, 10, 4), dtype=float32, numpy=

array([[[1. , 1. , 1. , 1. ], [1. , 1. , 1. , 1. ], [1. , 1. , 1. , 1. ], [1. , 0.5105105, 1. , 1. ], [1. , 1. , 1. , 1. ], [1. , 1. , 1. , 1. ], [0.7327327, 1. , 1. , 1. ], [1. , 1. , 1. , 1. ], [1. , 1. , 1. , 1. ], [1. , 1. , 1. , 1. ]]], dtype=float32)>, <tf.Tensor: shape=(1, 10, 4), dtype=float32, numpy= array([[[640. , 640. , 640. , 640. ], [640. , 640. , 640. , 640. ], [640. , 640. , 640. , 640. ], [640. , 326.7267 , 640. , 640. ], [640. , 640. , 640. , 640. ], [640. , 640. , 640. , 640. ], [468.94894, 640. , 640. , 640. ], [640. , 640. , 640. , 640. ], [640. , 640. , 640. , 640. ], [640. , 640. , 640. , 640. ]]], dtype=float32)>, <tf.Tensor: shape=(1, 10), dtype=int64, numpy=array([[505, 802, 505, 505, 505, 505, 505, 776, 505, 505]])>, <tf.Tensor: shape=(1, 10), dtype=float32, numpy= array([[8.74843026e-05, 9.13504991e-05, 8.93203469e-05, 8.99704537e-05, 8.39981003e-05, 9.77213494e-05, 8.80038278e-05, 8.94075274e-05, 1.05320119e-04, 1.03460596e-04]], dtype=float32)>, <tf.Tensor: shape=(1, 10), dtype=int64, numpy=array([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]])>, <tf.Tensor: shape=(1, 10, 4), dtype=float32, numpy= array([[[0., 0., 0., 0.], [0., 0., 0., 0.], [0., 0., 0., 0.], [0., 0., 0., 0.], [0., 0., 0., 0.], [0., 0., 0., 0.], [0., 0., 0., 0.], [0., 0., 0., 0.], [0., 0., 0., 0.], [0., 0., 0., 0.]]], dtype=float32)>, <tf.Tensor: shape=(1, 10, 4), dtype=float32, numpy= array([[[0., 0., 0., 0.], [0., 0., 0., 0.], [0., 0., 0., 0.], [0., 0., 0., 0.], [0., 0., 0., 0.], [0., 0., 0., 0.], [0., 0., 0., 0.], [0., 0., 0., 0.], [0., 0., 0., 0.], [0., 0., 0., 0.]]], dtype=float32)>, <tf.Tensor: shape=(1, 10), dtype=float32, numpy=array([[0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]], dtype=float32)>, <tf.Tensor: shape=(1, 10), dtype=bool, numpy= array([[False, False, False, False, False, False, False, False, False, False]])>) `

52THANOS commented 1 year ago

can i see your pip list，i have environment conflict

Guzaiwang commented 1 year ago

Maybe you need to check whether the ckpt model has been loaded successfully.