facebookresearch / Detectron

FAIR's research platform for object detection research, implementing popular algorithms like Mask R-CNN and RetinaNet.
Apache License 2.0
26.22k stars 5.45k forks source link

SmoothL1LossTest fails when trying to call gc.CheckSimple() #924

Open triasoft opened 5 years ago

triasoft commented 5 years ago

Detailed steps to reproduce

I am using the docker image to train and test Detectron successfully so far. Currently, I am trying to get unit tests to run. Successfully so far. Except:

root@3d1f8558b797:/detectron/detectron/tests# /usr/bin/python2 -u /root/.pycharm_helpers/pycharm/_jb_unittest_runner.py --target test_loader.TestRoIDataLoader

Did not yield in success since the Caffe2 Gradient Check seems to fail.

Expected results

What did you expect to see?

OK 

Actual results

What did you observe instead?

Testing started at 15:49 ...
sudo+ssh://root@dev2.l.aces:12345/usr/bin/python2 -u /root/.pycharm_helpers/pycharm/_jb_unittest_runner.py --target test_smooth_l1_loss_op.SmoothL1LossTest
Launching unittests with arguments python -m unittest test_smooth_l1_loss_op.SmoothL1LossTest in /detectron/detectron/tests

No handlers could be found for logger "caffe2.python.net_drawer"
net_drawer will not run correctly. Please install the correct dependencies.
Error
Traceback (most recent call last):
  File "/usr/lib/python2.7/unittest/case.py", line 329, in run
    testMethod()
  File "/detectron/detectron/tests/test_smooth_l1_loss_op.py", line 57, in test_forward_and_gradient
    res, grad, grad_estimated = gc.CheckSimple(op, [Y_hat, Y, inside_weights, outside_weights], 0, [0])
  File "/usr/local/caffe2_build/caffe2/python/gradient_checker.py", line 269, in CheckSimple
    op, [s + '_grad' for s in op.output])
  File "/usr/local/caffe2_build/caffe2/python/core.py", line 1082, in GetGradientForOp
    format(op.type, e, str(op))
Exception: Exception when creating gradient for [SmoothL1Loss]:[enforce fail at pybind_state.cc:535] caffe2::GradientRegistry()->Has(def.type()).  .
Op: 
input: "Y_hat"
input: "Y"
input: "inside_weights"
input: "outside_weights"
output: "loss"
name: ""
type: "SmoothL1Loss"
arg {
  name: "beta"
  f: 0.632352948189
}
arg {
  name: "scale"
  f: 0.539091706276
}
device_option {
  device_type: 1
  cuda_gpu_id: 0
}

Assertion failed

Ran 1 test in 0.004s

FAILED (errors=1)

System information