Detectron Ami in Aws

[RomRoc]

It could be very useful if someone publish an Amazon Aws Ami where Detectron is already installed, similarly to what is available for other deep learning platform (i.e. Tensorflow). Currently in Aws Marketplace there isn't any Ami with Caffe2 and Detectron installed. In other case it could be useful just a single script or Jupyter notebook to execute all the steps to install Caffe2 and Detectron from a brand new Ami. Thanks

[rbgirshick]

@orionr this sounds like a suggestion worth looking into.

[mattifrind]

I can publish my AMI, but I need to remove some private files first. I think I can do that in the middle of the next week, but I have some issues and I'm not sure what my mistakes during the installation are. These are all the executed steps during my installation from a clean AWS machine. When I run the test_spatial_narrow_as_op.py I get this info:

ubuntu@ip-172-31-6-83:~$ python2 detectron/tests/test_spatial_narrow_as_op.py
No handlers could be found for logger "caffe2.python.net_drawer"
net_drawer will not run correctly. Please install the correct dependencies.
E0202 09:51:53.608631 63603 init_intrinsics_check.cc:54] CPU feature avx is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
E0202 09:51:53.608661 63603 init_intrinsics_check.cc:54] CPU feature avx2 is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
E0202 09:51:53.608675 63603 init_intrinsics_check.cc:54] CPU feature fma is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
...
----------------------------------------------------------------------
Ran 3 tests in 1.056s

OK

Have anyone an idea how to fix this?

[vobject]

+1 An AWS Ami would be great!

@mattifrind, pass "-DUSE_ARCH_NATIVE=ON" when configuring CMake for Caffe2. That step is missing in your post entirely under "Install Caffe2." mkdir build && cd build && cmake -DUSE_ARCH_NATIVE=ON .. It worked for my local build.

Ref: https://github.com/caffe2/caffe2/blob/master/CMakeLists.txt#L40

[partnercloudsupport]

Hi, I had this same error....saw this commend and ran cmake -DUSE_ARCH_NATIVE=ON .. my build ran but i got this error Please advise.

Thank you.

python //home/ubuntu/detectron/tests/test_spatial_narrow_as_op.py E0220 22:47:26.863031 14449 init_intrinsics_check.cc:59] CPU feature avx is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU. E0220 22:47:26.863085 14449 init_intrinsics_check.cc:59] CPU feature avx2 is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU. E0220 22:47:26.863097 14449 init_intrinsics_check.cc:59] CPU feature fma is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU. ...

Ran 3 tests in 1.275s

OK

[matsui528]

Hi there,

I wrote a single install script of Detectron on an AWS EC2 instance (p3.2xlarge) with a Deep Learning Base AMI. Hope it would be helpful :) https://gist.github.com/matsui528/6d223d17241842c84d5882a9afa0453a

[mattifrind]

@vobject thanks for your tip but as @partnercloudsupport commented we get the same error. Any ideas?

[partnercloudsupport]

Hi Matti Frind, I got a reply on a separate post. This is not an error...it is a expected response.

168 Hi @partnercloudsupport, that sounds great. Note that the lines you posted above are not an error. Namely, these are Caffe2 warnings saying that you have not compiled Caffe2 binary with avx support. This is relevant for running on the CPU and can be ignored in case of Detectron that runs on the GPU. The test now passes and you can proceed with the rest of the install/getting_started instructions.