Implement OpenCV 'VideoCapture' class in 'detector.cpp'

[jeff1evesque]

We need to implement the VideoCapture class in detector.cpp. This file will capture video from the default camera.

[jeff1evesque]

ef2220c: we are unable to compile an implementation of the VideoCapture class in detector.c:

vagrant@vagrant-ubuntu-trusty-64:/vagrant$ whereis gcc
gcc: /usr/bin/gcc /usr/lib/gcc /usr/bin/X11/gcc /usr/share/man/man1/gcc.1.gz
vagrant@vagrant-ubuntu-trusty-64:/vagrant$ ls
detector.c  manifests  README.md  Vagrantfile
vagrant@vagrant-ubuntu-trusty-64:/vagrant$ gcc detector.c
In file included from /usr/local/include/opencv2/opencv.hpp:46:0,
                 from detector.c:1:
/usr/local/include/opencv2/core.hpp:47:4: error: #error core.hpp header must be compiled as C++
 #  error core.hpp header must be compiled as C++
    ^
In file included from /usr/local/include/opencv2/core.hpp:52:0,
                 from /usr/local/include/opencv2/opencv.hpp:46,
                 from detector.c:1:
/usr/local/include/opencv2/core/base.hpp:48:4: error: #error base.hpp header must be compiled as C++
 #  error base.hpp header must be compiled as C++
    ^
/usr/local/include/opencv2/core/base.hpp:51:19: fatal error: climits: No such file or directory
 #include <climits>
                   ^
compilation terminated.

[jeff1evesque]

IRC #opencv (04/30/15 ~ 5:05pm EST):

jeffreylevesque, trying to compile opencv, and i get "base.hpp:48:4: error: #error base.hpp header must be compiled as C++" jeffreylevesque, I have compiled opencv from github using version '3.0.0-beta' jeffreylevesque, i tried implement http://docs.opencv.org/modules/highgui/doc/reading_and_writing_images_and_video.html#videocapture, as verbatim jeffreylevesque, and received the following errors - https://github.com/jeff1evesque/iScanner/issues/30#issuecomment-97953450 jeffreylevesque, I noticed someone with a similar problem - https://github.com/lazywei/go-opencv/issues/18 jeffreylevesque, curious as to itseez repository on github has resolved that issue?

gliese581h, jeffreylevesque: arre you sure, the file that was called from was a .cpp or .cxx one (in other words did it really call g++ in the end?)

jeffreylevesque, ohh

gliese581h, if it's called *.c, probably interpreted as plain c

jeffreylevesque, the file that's implementing the opencv example (verbatim), has the file extension *.c

gliese581h, try to rename it, see if there's any difference

jeffreylevesque, do I need to change the file extension to either .cpp, or .cxx, and recompile using gcc?

Hunts, yeah

jeffreylevesque, is there any difference between cpp or cxx?

gliese581h, but, to my knowledge, gcc or even g++ switch on file extension there

Hunts, jeffreylevesque: nope

[jeff1evesque]

1042664: when we rename detector.c to detector.cpp, we get the following error traceback:

vagrant@vagrant-ubuntu-trusty-64:/vagrant$ ls
detector.cpp  manifests  README.md  Vagrantfile
vagrant@vagrant-ubuntu-trusty-64:/vagrant$ gcc detector.cpp
detector.cpp: In function ‘int main(int, char**)’:
detector.cpp:17:32: error: ‘CV_BGR2GRAY’ was not declared in this scope
         cvtColor(frame, edges, CV_BGR2GRAY);
                                ^

[jeff1evesque]

c767934: when we rename detector.cpp to detector.cxx, we get the following error traceback:

vagrant@vagrant-ubuntu-trusty-64:/vagrant$ ls
detector.cxx  manifests  README.md  Vagrantfile
vagrant@vagrant-ubuntu-trusty-64:/vagrant$ gcc detector.cxx
detector.cxx: In function ‘int main(int, char**)’:
detector.cxx:17:32: error: ‘CV_BGR2GRAY’ was not declared in this scope
         cvtColor(frame, edges, CV_BGR2GRAY);
                                ^

[jeff1evesque]

The above attempted code was designed for opencv version 2.4.11.0. However, our build is 3.0.0-beta. So, we need to adjust the logic to the opencv version we are using, respectively.

[jeff1evesque]

88f3438: we used the opencv 3.0.0-dev syntax. However, we still received the following traceback error:

vagrant@vagrant-ubuntu-trusty-64:/vagrant$ ls
detector.cxx  manifests  README.md  Vagrantfile
vagrant@vagrant-ubuntu-trusty-64:/vagrant$ gcc detector.cxx
/tmp/cc21xK5O.o: In function `main':
detector.cxx:(.text+0x29): undefined reference to `cv::VideoCapture::VideoCapture(int)'
detector.cxx:(.text+0x38): undefined reference to `cv::VideoCapture::isOpened() const'
detector.cxx:(.text+0x7a): undefined reference to `cv::namedWindow(cv::String const&, int)'
detector.cxx:(.text+0x99): undefined reference to `vtable for cv::VideoCapture'
detector.cxx:(.text+0xfb): undefined reference to `cv::cvtColor(cv::_InputArray const&, cv::_OutputArray const&, int, int)'
detector.cxx:(.text+0x1b8): undefined reference to `cv::GaussianBlur(cv::_InputArray const&, cv::_OutputArray const&, cv::Size_<int>, double, double, int)'
detector.cxx:(.text+0x242): undefined reference to `cv::Canny(cv::_InputArray const&, cv::_OutputArray const&, double, double, int, bool)'
detector.cxx:(.text+0x2a6): undefined reference to `cv::imshow(cv::String const&, cv::_InputArray const&)'
detector.cxx:(.text+0x2ce): undefined reference to `cv::waitKey(int)'
detector.cxx:(.text+0x31d): undefined reference to `cv::VideoCapture::~VideoCapture()'
detector.cxx:(.text+0x400): undefined reference to `cv::VideoCapture::~VideoCapture()'
/tmp/cc21xK5O.o: In function `__static_initialization_and_destruction_0(int, int)':
detector.cxx:(.text+0x43c): undefined reference to `std::ios_base::Init::Init()'
detector.cxx:(.text+0x44b): undefined reference to `std::ios_base::Init::~Init()'
/tmp/cc21xK5O.o: In function `cv::String::String(char const*)':
detector.cxx:(.text._ZN2cv6StringC2EPKc[_ZN2cv6StringC5EPKc]+0x4f): undefined reference to `cv::String::allocate(unsigned long)'
/tmp/cc21xK5O.o: In function `cv::String::~String()':
detector.cxx:(.text._ZN2cv6StringD2Ev[_ZN2cv6StringD5Ev]+0x14): undefined reference to `cv::String::deallocate()'
/tmp/cc21xK5O.o: In function `cv::_InputArray::_InputArray()':
detector.cxx:(.text._ZN2cv11_InputArrayC2Ev[_ZN2cv11_InputArrayC5Ev]+0x13): undefined reference to `vtable for cv::_InputArray'
/tmp/cc21xK5O.o: In function `cv::_InputArray::_InputArray(cv::Mat const&)':
detector.cxx:(.text._ZN2cv11_InputArrayC2ERKNS_3MatE[_ZN2cv11_InputArrayC5ERKNS_3MatE]+0x17): undefined reference to `vtable for cv::_InputArray'
/tmp/cc21xK5O.o: In function `cv::_InputArray::~_InputArray()':
detector.cxx:(.text._ZN2cv11_InputArrayD2Ev[_ZN2cv11_InputArrayD5Ev]+0x13): undefined reference to `vtable for cv::_InputArray'
detector.cxx:(.text._ZN2cv11_InputArrayD2Ev[_ZN2cv11_InputArrayD5Ev]+0x28): undefined reference to `operator delete(void*)'
/tmp/cc21xK5O.o: In function `cv::_InputArray::~_InputArray()':
detector.cxx:(.text._ZN2cv11_InputArrayD0Ev[_ZN2cv11_InputArrayD0Ev]+0x20): undefined reference to `operator delete(void*)'
/tmp/cc21xK5O.o: In function `cv::_OutputArray::_OutputArray(cv::Mat&)':
detector.cxx:(.text._ZN2cv12_OutputArrayC2ERNS_3MatE[_ZN2cv12_OutputArrayC5ERNS_3MatE]+0x23): undefined reference to `vtable for cv::_OutputArray'
/tmp/cc21xK5O.o: In function `cv::_OutputArray::~_OutputArray()':
detector.cxx:(.text._ZN2cv12_OutputArrayD2Ev[_ZN2cv12_OutputArrayD5Ev]+0x13): undefined reference to `vtable for cv::_OutputArray'
detector.cxx:(.text._ZN2cv12_OutputArrayD2Ev[_ZN2cv12_OutputArrayD5Ev]+0x34): undefined reference to `operator delete(void*)'
/tmp/cc21xK5O.o: In function `cv::_OutputArray::~_OutputArray()':
detector.cxx:(.text._ZN2cv12_OutputArrayD0Ev[_ZN2cv12_OutputArrayD0Ev]+0x20): undefined reference to `operator delete(void*)'
/tmp/cc21xK5O.o: In function `cv::Mat::~Mat()':
detector.cxx:(.text._ZN2cv3MatD2Ev[_ZN2cv3MatD5Ev]+0x39): undefined reference to `cv::fastFree(void*)'
/tmp/cc21xK5O.o: In function `cv::Mat::release()':
detector.cxx:(.text._ZN2cv3Mat7releaseEv[_ZN2cv3Mat7releaseEv]+0x4b): undefined reference to `cv::Mat::deallocate()'
/tmp/cc21xK5O.o:(.eh_frame+0x20b): undefined reference to `__gxx_personality_v0'
collect2: error: ld returned 1 exit status

[jeff1evesque]

When we attempt to compile our detector.cxxusing the pkg-config --cflags --libs opencv flags, we get the following traceback:

vagrant@vagrant-ubuntu-trusty-64:/vagrant$ gcc detector.cxx `pkg-config --cflags --libs opencv`
/usr/bin/ld: /tmp/ccW1L4Fs.o: undefined reference to symbol '_ZNSt8ios_base4InitD1Ev@@GLIBCXX_3.4'
//usr/lib/x86_64-linux-gnu/libstdc++.so.6: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status

[jeff1evesque]

We were able to compile detector.cxx using the g++ compiler:

vagrant@vagrant-ubuntu-trusty-64:/vagrant$ ls
detector.cxx  manifests  README.md  Vagrantfile
vagrant@vagrant-ubuntu-trusty-64:/vagrant$ g++ detector.cxx `pkg-config --cflags --libs opencv`
vagrant@vagrant-ubuntu-trusty-64:/vagrant$ ls
a.out  detector.cxx  manifests  README.md  Vagrantfile

[jeff1evesque]

Though we were able to successfully implement g++ for compilation, we were not able to run the compiled detector.exe executable:

vagrant@vagrant-ubuntu-trusty-64:/vagrant$ ls
detector.cpp  manifests  README.md  Vagrantfile
vagrant@vagrant-ubuntu-trusty-64:/vagrant$ g++ -o detector.exe detector.cpp `pkg-config --cflags --libs opencv`
vagrant@vagrant-ubuntu-trusty-64:/vagrant$ ls
detector.cpp  detector.exe  manifests  README.md  Vagrantfile
vagrant@vagrant-ubuntu-trusty-64:/vagrant$ ./detector.exe
libdc1394 error: Failed to initialize libdc1394

[jeff1evesque]

To resolve the above error of Failed to initialize libdc1394, we need to enable the USB Controller:

To keep logic contained within automated process (Vagrant), instead of simply checking the box within VirtualBox GUI, we need to install Virtualbox Extension Pack, and add the --usb, and --usbehci definitions in the Vagrantfile:

box.vm.provider :virtualbox do |vb|
    vb.customize ["modifyvm", :id, "--usb", "on"]
    vb.customize ["modifyvm", :id, "--usbehci", "on"]
end

Then, we can see that we have enabled the USB Controller:

Finally, we need to define the usbfilter to allow VirtualBox to mount any USB device:

box.vm.provider :virtualbox do |vb|
    vb.customize ["usbfilter", "add", "0", 
        "--target", :id, 
        "--name", "Any mass storage", 
        "--manufacturer", "Generic",
        "--product", "Mass Storage Device"]
end

The above code needs to adhere to the usbfilter syntax:

  usbfilter                 add <index,0-N>
                            --target <uuid|vmname>|global
                            --name <string>
                            --action ignore|hold (global filters only)
                            [--active yes|no] (yes)
                            [--vendorid <XXXX>] (null)
                            [--productid <XXXX>] (null)
                            [--revision <IIFF>] (null)
                            [--manufacturer <string>] (null)
                            [--product <string>] (null)
                            [--remote yes|no] (null, VM filters only)
                            [--serialnumber <string>] (null)
                            [--maskedinterfaces <XXXXXXXX>]

Now, we have USB Device Filters defined:

Note: we can obtain information a list of available usb devices on the host (if needed):

$ sudo VBoxManage list usbhost
[sudo] password for jeffrey: 
Host USB Devices:

UUID:               xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
VendorId:           0x064e (064E)
ProductId:          0xc107 (C107)
Revision:           2.49 (0249)
Port:               4
USB version/speed:  2/2
Manufacturer:       SuYin
Product:            xx Webcam
SerialNumber:       xxxxx-xxxx-xxxx-xx-R02.43.01
Address:            sysfs:/sys/devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.5//device:/dev/vboxusb/002/003
Current State:      Busy

UUID:               xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
VendorId:           0x0781 (0781)
ProductId:          0x5575 (5575)
Revision:           1.39 (0139)
Port:               0
USB version/speed:  2/2
Manufacturer:       SanDisk
Product:            Cruzer Glide
SerialNumber:       xxxxxxxxxxxxxxxxxxxx
Address:            sysfs:/sys/devices/pci0000:00/0000:00:1a.0/usb1/1-1/1-1.1//device:/dev/vboxusb/001/011
Current State:      Busy

[jeff1evesque]

Though, the above commits resolve the Failed to initialize libdc1394 error, it does not resolve the issue of being able to mount USB drives from the host machine to the Vagrant guest.

Also, running ./detector.exe after compilation does have any desired results:

vagrant@vagrant-ubuntu-trusty-64:/vagrant$ ls
detector.cpp  detector.exe  manifests  README.md  Vagrantfile
vagrant@vagrant-ubuntu-trusty-64:/vagrant$ ./detector.exe
vagrant@vagrant-ubuntu-trusty-64:/vagrant$

The executable simply runs, and exits immediately, without any desired outcomes.

[jeff1evesque]

When we check the exit code for the last executed command, we get 255:

vagrant@vagrant-ubuntu-trusty-64:/vagrant$ ls
detector.cpp  detector.exe  puppet  README.md  Vagrantfile
vagrant@vagrant-ubuntu-trusty-64:/vagrant$ pico detector.cpp
vagrant@vagrant-ubuntu-trusty-64:/vagrant$ echo $?
0
vagrant@vagrant-ubuntu-trusty-64:/vagrant$ ./detector.exe
vagrant@vagrant-ubuntu-trusty-64:/vagrant$ echo $?
255

This likely means that default camera on the host machine failed to initialize. Specifically, the following from detector.cpp failed:

#include "opencv2/opencv.hpp"

using namespace cv;

int main(int, char**)
{
    VideoCapture cap(0); // open the default camera
    if(!cap.isOpened())  // check if we succeeded
        return -1;
...

[jeff1evesque]

OpenCV requires the implementation of ffmpeg. However, we have decided to forgo the installation of ffmpeg, since libav has replaced the package within most recent linux (i.e. Ubuntu) package managers (i.e. apt-get). Instead, we have installed the following:

...
sudo apt-get libavcodec-dev
sudo apt-get libavformat-dev
...

[jeff1evesque]

We were able to open the Ubuntu cheese application, and it worked as expected.

[jeff1evesque]

The above VBoxManage command shows that the default camera is a USB device:

$ sudo VBoxManage list usbhost
[sudo] password for jeffrey: 
Host USB Devices:

UUID:               xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
VendorId:           0x064e (064E)
ProductId:          0xc107 (C107)
Revision:           2.49 (0249)
Port:               4
USB version/speed:  2/2
Manufacturer:       SuYin
Product:            xx Webcam
SerialNumber:       xxxxx-xxxx-xxxx-xx-R02.43.01
Address:            sysfs:/sys/devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.5//device:/dev/vboxusb/002/003
Current State:      Busy
...

This means we need to define a USB Device Filter within the VirtualBox GUI.

[jeff1evesque]

After we enabled the default camera device via the USB Device Filter on the VirtualBox GUI:

We were still unable to bypass the first if statement from the compiled detector.cpp:

#include "opencv2/opencv.hpp"

using namespace cv;
using namespace std;

int main(int, char**)
{
    VideoCapture cap(0);  // open the default camera
    if(!cap.isOpened())   // check if we succeeded
    {
        cout << "Cannot open the default webcam" << endl;
        return -1;
    }
...

Again, we received the following output within the terminal console:

vagrant@vagrant-ubuntu-trusty-64:/vagrant$ ls
detector.cpp  detector.exe  puppet  README.md  Vagrantfile
vagrant@vagrant-ubuntu-trusty-64:/vagrant$ ./detector.exe
Cannot open the default webcam
vagrant@vagrant-ubuntu-trusty-64:/vagrant$ echo $?
255

Therefore, further logic is unable to execute since the default camera (USB) is not properly initialized with the VideoCapture class.

[jeff1evesque]

According to the opencv 3.0 installation documentation, we can choose to either install ffmpeg, or install the libav development packages:

• libavcodec-dev
• libavformat-dev
• libswscale-dev

[jeff1evesque]

9964568, 6a09fe6: we include gstreamer to cmake, and include an additional 300s for this process. If this does not work, we will attempt the following changes in install_opencv.pp:

• change WITH_V4L=ON to USE_V4L=ON to cmake
• install ffmpeg via PPA, and add WITH_FFMPEG=ON to cmake
  • keep USE_ prefix for V4L, and FFMPEG
  • change USE_ prefix to WITH_ for V4L, and FFMPEG

[jeff1evesque]

We attempted the following cases with the cmake command in install_opencv.pp:

• USE_GStreamer=ON, WITH_V4L=ON
• USE_GStreamer=ON, and USE_V4L=ON
• USE_GStreamer=ON, USE_V4L=ON, and WITH_FFMPEG=OFF

[jeff1evesque]

We created a separate virtual environment instance, consisting of Ubuntu Server 14.04, along with the following packages:

• Guest Additions: sudo apt-get install virtualbox-guest-x11
• Ubuntu GUI: sudo apt-get install xorg gnome-core gnome-system-tools gnome-app-install
• Cheese: sudo apt-get install cheese

Then, we also ensured that we Enable USB Controller on the Ubuntu Settings GUI, and defined any USB Device Filters present, without inserting any external USB devices into the host machine.

However, after sudo reboot, we notice the No device found message on the Cheese application from the guest virtual machine:

[jeff1evesque]

We need to take a look at the Virtualbox Webcam Passthrough documentation. The documentation section, discusses the possibility of the guest machine accessing the host webcam.

The following shows how the VBoxManage can be used to list the current available webcam devices, and then to attach the webcam from the host, to the guest machine:

jeffreys-Air:iScanner jeffrey$ VBoxManage list webcams
Video Input Devices: 1
.1 "FaceTime HD Camera"
FKH32D01XGKF9FSC7
jeffreys-Air:iScanner jeffrey$ VBoxManage list vms
"Ubuntu14.04" {ed46s6f9-cb22-4355-5555-f1bd7215ffec}
"iScanner_default_1431262003442_79804" {3fea3a26-e6aa-4324-be3f1-5frs26e6032c1}
jeffreys-Air:iScanner jeffrey$ VBoxManage controlvm "Ubuntu14.04" webcam attach FKH32D01XGKF9FSC7

Note: in windows, we can install VBoxManage by the following:

$ cd /c/Program Files/Oracle/VirtualBox
$ VBoxManage.exe list hdds

In the case of an OSX, no USB Device Filters needed to be defined. However, the above VBoxManage command needs to be executed each time the virtual machine starts up.

Now, the Cheese application on the guest VM, was able to access the host webcam:

[jeff1evesque]

This time, we connect the webcam to the target virtual machine:

jeffreys-Air:iScanner jeffrey$ VBoxManage list webcams
Video Input Devices: 1
.1 "FaceTime HD Camera"
FKH32D01XGKF9FSC7
jeffreys-Air:iScanner jeffrey$ VBoxManage list vms
"Ubuntu14.04" {ed46s6f9-cb22-4355-5555-f1bd7215ffec}
"iScanner_default_1431262003442_79804" {3fea3a26-e6aa-4324-be3f1-5frs26e6032c1}
jeffreys-Air:iScanner jeffrey$ VBoxManage controlvm "iScanner_default_1431262003442_79804" webcam attach FKH32D01XGKF9FSC7

However, upon vagrant ssh, we still cannot connect to the webcam:

jeffreys-Air:iScanner jeffrey$ cd /Users/jeffrey/github/iScanner/
jeffreys-Air:iScanner jeffrey$ ls
README.md   Vagrantfile detector.cpp    detector.exe    puppet
jeffreys-Air:iScanner jeffrey$ vagrant up
jeffreys-Air:iScanner jeffrey$ vagrant ssh
Welcome to Ubuntu 14.04.2 LTS (GNU/Linux 3.13.0-52-generic x86_64)

 * Documentation:  https://help.ubuntu.com/

  System information as of Mon May 11 11:03:33 UTC 2015

  System load:  0.82               Processes:           85
  Usage of /:   11.5% of 39.34GB   Users logged in:     0
  Memory usage: 24%                IP address for eth0: xx.x.x.xx
  Swap usage:   0%

  Graph this data and manage this system at:
    https://landscape.canonical.com/

  Get cloud support with Ubuntu Advantage Cloud Guest:
    http://www.ubuntu.com/business/services/cloud

Last login: Sun May 10 13:16:04 2015 from xx.x.x.x
vagrant@vagrant-ubuntu-trusty-64:~$ cd /vagrant
vagrant@vagrant-ubuntu-trusty-64:/vagrant$ g++ -o detector.exe detector.cpp `pkg-config --cflags --libs opencv`
vagrant@vagrant-ubuntu-trusty-64:/vagrant$ ls
detector.cpp  detector.exe  puppet  README.md  Vagrantfile
vagrant@vagrant-ubuntu-trusty-64:/vagrant$ ./detector.exe
Cannot open the default webcam