Dashboard on Pepper's tablet does not show anything in the Conversation area (and Raspberry camera issue)

[takaomoriyama]

On Pepper, dashboard is now shown on it's tablet. Video monitor is shown in the top left area, spectrum graph of input voice is shown in the top right area, and text input is show below the graph. However nothing is shown in the Conversation area (bottom half of the screen), where we expect dialog bubbles appear.

[nddao]

I also experience this issue on Raspberry

[rlyle]

I wonder if the end-point is hard-coded into the javascript.. so it's not connecting to the websocket.

[takaomoriyama]

I think this depends on the Web browser functionality. Pepper's dashboard successfully shows Conversation on the Chrome browser on remote machine. @nddao Can you check with local or remote Chrome browser ?

[nddao]

I see the messages on Conversation area after configuring the microphone correctly. However, still get warning message WARN][OpenCVCamera] Failed to open camera even thought camera is connected

[takaomoriyama]

@nddao Could you check whether your camera is working properly without running Intu ? I have my camera connected to my Raspi, but it does not properly.

First. Please check if the camera recognized by the kernel by the following command.

$ vcgencmd get_camera
supported=1 detected=0

If it is connected and recognized properly, the result must be detected=1.

If the connection is OK, then confirm if the camera is capturing correct image data. Please note that the output below is bad case.

$ raspistill -t 100 -n -o test.jpg
mmal: Cannot read camera info, keeping the defaults for OV5647
mmal: mmal_vc_component_create: failed to create component 'vc.ril.camera' (1:ENOMEM)
mmal: mmal_component_create_core: could not create component 'vc.ril.camera' (1)
mmal: Failed to create camera component
mmal: main: Failed to create camera component
mmal: Camera is not detected. Please check carefully the camera module is installed correctly

[nddao]

@takaomoriyama : pi@raspberrypi:~ $ vcgencmd get_camera supported=1 detected=1 My camera is a Noir Camera 8 mega pixel V2.1, and I can take pictures and video. On the SelfInstance.log, there are lines for camera as below: [SensorManager] Added sensor Camera, Id: e85cb1a8-8f0e-35b0-6770-f91ad3b8efe6 [SensorManager] Added sensor DepthCamera, Id: 783a8643-0301-24a9-949f-33731ef21006 .......... [OpenCVCamera] Failed to open camera

---

I tested with 2 different cameras, and got the same result. Not sure what are the difference between sensor Camera and sensor DepthCamera. For your issue, I think that the camera cable was not installed correctly. the side that has words on it faces toward the usb ports.

[takaomoriyama]

On Raspi, Camera is accessed through OpenCV, so ou can ignore Camera and DepthCamera lines in the log file. The last line shows that Intu could not access the camera through OpenCV. plugins/opencv/sensors/OpenCVCamera.cpp.

        if (m_CameraStream.size() > 0)
            m_VideoCapture = new cv::VideoCapture(m_CameraStream); 
        else
            m_VideoCapture = new cv::VideoCapture(m_CameraDevice); 

        if ( m_VideoCapture->isOpened() )                                                                                     
        {
            m_spWaitTimer = TimerPool::Instance()->StartTimer(VOID_DELEGATE(OpenCVCamera, OnCaptureImage, this), (1.0f / m_fFramesPerSec), false, true);     
            Log::Status("OpenCVCamera", "Camera has started"); 
        }
        else
            Log::Warning("OpenCVCamera", "Failed to open camera");

Something wrong around OpenCV in your environment. Please let me check this once I get my camera working.

[takaomoriyama]

@nddao Let's figure out whether problem is in Camera device or OpenCV.

We can feed photo file or movie file instead of Camera input. Can you follow the steps below ?

1. Take a video. I used Photo Booth application on my Mac.
2. Transfer the .mov file to your Raspberry Pi, and place it under the directory where Intu binary is contained.
3. Edit etc/shared/bootstrap.json, and specify the movie file name to "m_CameraStream" item. Or you can do this through Intu dashboard.
4. If you edited bootstrap.json, remove db/knowledge.db
5. Start Intu

Each frame of the movie file will be used as camera input, and will be shown on the Intu dashboard. If this does not work, the problem is around OpenCV library, otherwise in your Camera device.

[nddao]

@takaomoriyama . With a .mov or .mp4 in the m_CameraStream, I can see a frame in the camera box, and in the log, I can see [OpenCVCamera] Camera has started

[takaomoriyama]

OK. OpenCV is properly loaded and working well. Then the problem is in camera side. Please be patient until I get new camera.

[nddao]

Found solution to fix this issue on Raspberry. The main reason that we do not see video stream in the dash board is the Raspberry does not load the library bcm2835-v4l2 at boot time. Solution: sudo nano /etc/modules add bcm2835-v4l2 in to this file to load bcm2835-v4l2 at boot time

sudo nano /etc/modules bcm2835-v4l2

Then save file

[nddao]

However, when the stream of video displays on Intu Dashboard, look like it streams to Watson continuously and causes problem on Raspberry. There are lots of messages like below 11/09/17 18:38:01.584][ERRO][ImageClassifier] Invalid response from IVisualRecognition: { "status" : "ERROR", "statusInfo" : "daily-transaction-limit-exceeded" }

[11/09/17 18:38:02.912][ERRO][PersonClassifer] Invalid response from VisualRecognition: { "status" : "ERROR", "statusInfo" : "daily-transaction-limit-exceeded" }

So for testing the Conversation service, I have to stop loading the bcm2835-v4l2 module at boot time.

[takaomoriyama]

@nddao After replacing my camera with new one, linux kernel started to recognize the camera, and raspistill comand started to work well. Intu is now recognizing the camera after adding bcm2835-v4l2 to /etc/modules according to your suggestion. Thanks!