Not able to run mycroft face wake up

[patilaum]

I did the setup .....but when I run the main.py file it only comes till this and doesn't run further ALSA lib pcm.c:2266:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear ALSA lib pcm.c:2266:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe ALSA lib pcm.c:2266:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side ALSA lib pcm_route.c:867:(find_matching_chmap) Found no matching channel map

may be im not able to connect to same host as mycroft core please help me with it

[ChristopherRogers1991]

Is this running on a desktop computer, a Mark 1, or a Raspberry Pi using the pycroft image?

Can you run pip freeze and paste the output? Be sure to activate your virtualenv first, if you used one.

[patilaum]

I am running it on desktop. It doesn't run further than this. I'm not using virtualenv.

[ChristopherRogers1991]

@patilaum,

Can you run pip freeze, and paste the output here?

Then, please run https://github.com/ChristopherRogers1991/mycroft-face-wake/blob/master/simple_audio_recorder/simple_audio_recorder.py. If you run that file, it should record 5 seconds of audio, and save it as a file called test.wav. Let me know if you're able to successfully record anything.

Next, please run https://github.com/ChristopherRogers1991/mycroft-face-wake/blob/master/object_watcher/object_watcher.py. That should run for 10 seconds, and should print a line each time you look at your camera, and then away from the camera. Let me know if that works.

Each of these can be run with python /path/to/file.py, so from the top level of the git repo,

python simple_audio_recorder/simple_audio_recorder.py

and

python object_watcher/object_watcher.py

-Chris

[patilaum]

OUTPUT OF pip3 freeze absl-py==0.2.2 alabaster==0.7.10 apturl==0.5.2 astor==0.6.2 astroid==1.6.2 Babel==2.5.3 beautifulsoup4==4.4.1 bleach==1.5.0 blinker==1.3 Brlapi==0.6.4 certifi==2018.1.18 chardet==3.0.4 checkbox-support==0.22 cloudpickle==0.5.2 command-not-found==0.3 cryptography==1.2.3 cycler==0.9.0 decorator==4.2.1 defer==1.0.6 dlib==19.13.1 docutils==0.14 entrypoints==0.2.3 feedparser==5.1.3 gast==0.2.0 grpcio==1.12.0 guacamole==0.9.2 html5lib==0.9999999 httplib2==0.9.1 idna==2.6 imagesize==1.0.0 imutils==0.4.6 ipykernel==4.8.2 ipython==6.2.1 ipython-genutils==0.2.0 isort==4.3.4 jedi==0.11.1 Jinja2==2.10 jsonschema==2.6.0 jupyter-client==5.2.3 jupyter-core==4.4.0 language-selector==0.1 lazy-object-proxy==1.3.1 louis==2.6.4 lxml==3.5.0 Mako==1.0.3 Markdown==2.6.11 MarkupSafe==1.0 matplotlib==1.5.1 mccabe==0.6.1 mistune==0.8.3 mycroft-face-wake==0.0.0 nbconvert==5.3.1 nbformat==4.4.0 numpy==1.14.4 numpydoc==0.7.0 oauthlib==1.0.3 onboard==1.2.0 opencv-python==3.4.1.15 packaging==17.1 padme==1.1.1 pandocfilters==1.4.2 parso==0.1.1 pep8==1.7.0 pexpect==4.4.0 pickleshare==0.7.4 Pillow==3.1.2 plainbox==0.25 prompt-toolkit==1.0.15 protobuf==3.5.2.post1 psutil==5.4.3 ptyprocess==0.5.2 pyasn1==0.1.9 PyAudio==0.2.11 pycodestyle==2.3.1 pycups==1.9.73 pycurl==7.43.0 pyflakes==1.6.0 Pygments==2.2.0 pygobject==3.20.0 PyJWT==1.3.0 pylint==1.8.3 PyOpenGL==3.1.0 pyparsing==2.2.0 PyQt5==5.9.2 python-apt==1.1.0b1+ubuntu0.16.4.1 python-dateutil==2.7.0 python-debian==0.1.27 python-systemd==231 pytz==2018.3 pyxdg==0.25 pyzmq==17.0.0 QtAwesome==0.4.4 qtconsole==4.3.1 QtPy==1.4.0 reportlab==3.3.0 requests==2.18.4 rope==0.10.7 rubikscubetracker==1.0.0 scikit-learn==0.19.1 scipy==1.1.0 screen-resolution-extra==0.0.0 sessioninstaller==0.0.0 simplegeneric==0.8.1 sip==4.19.8 six==1.11.0 snowballstemmer==1.2.1 SpeechRecognition==3.8.1 Sphinx==1.7.2 sphinxcontrib-websupport==1.0.1 spyder==3.2.8 system-service==0.3 tensorboard==1.8.0 tensorflow-gpu==1.8.0 termcolor==1.1.0 testpath==0.3.1 tornado==5.0.1 traitlets==4.3.2 ubuntu-drivers-common==0.0.0 ufw==0.35 unattended-upgrades==0.1 unity-scope-calculator==0.1 unity-scope-chromiumbookmarks==0.1 unity-scope-colourlovers==0.1 unity-scope-devhelp==0.1 unity-scope-firefoxbookmarks==0.1 unity-scope-gdrive==0.7 unity-scope-manpages==0.1 unity-scope-openclipart==0.1 unity-scope-texdoc==0.1 unity-scope-tomboy==0.1 unity-scope-virtualbox==0.1 unity-scope-yelp==0.1 unity-scope-zotero==0.1 urllib3==1.22 usb-creator==0.3.0 virtualenv==15.2.0 wcwidth==0.1.7 webencodings==0.5.1 websocket-client==0.48.0 Werkzeug==0.14.1 wrapt==1.10.11 xdiagnose==3.8.4.1 xkit==0.0.0 XlsxWriter==0.7.3

simple_audio_recoorder.py is running quite well but object_watcher.py runs for 10 seconds but does not print any line. I am using a usb webcam. thank you

[ChristopherRogers1991]

Does your webcam have a light? If so, does it turn on after you start object_watcher.py? Is there any output of any kind (e.g. error messages)? Also, what os and version are you running?

-Chris

[patilaum]

I dont have flash on my webcam. There is no output. It just runs for 10 sec and thats it. I am using Ubuntu 16.04.

[ChristopherRogers1991]

Ok, 16.04 is the same as me.

If you use a different application, such as Cheese, does the webcam work?

-Chris

[patilaum]

Yes it does.

[ChristopherRogers1991]

If you run:

import cv2

cv2.namedWindow("preview")
vc = cv2.VideoCapture(0)

if vc.isOpened(): # try to get the first frame
    rval, frame = vc.read()
else:
    rval = False

while rval:
    cv2.imshow("preview", frame)
    rval, frame = vc.read()
    key = cv2.waitKey(20)
    if key == 27: # exit on ESC
        break

cv2.destroyWindow("preview")
vc.release()

Does it bring up a window, and display video from your webcam?

(the code above is copied from https://stackoverflow.com/a/11449901/1406997)

[patilaum]

yes it brings a windows and displays video

[ChristopherRogers1991]

Ok, so not an issue with the camera. Can you running this?

import dlib # dlib for accurate face detection
import cv2 # opencv
import imutils # helper functions from pyimagesearch.com

# Grab video from your webcam
stream = cv2.VideoCapture(0)

# Face detector
detector = dlib.get_frontal_face_detector()

# Fancy box drawing function by Dan Masek
def draw_border(img, pt1, pt2, color, thickness, r, d):
    x1, y1 = pt1
    x2, y2 = pt2

    # Top left drawing
    cv2.line(img, (x1 + r, y1), (x1 + r + d, y1), color, thickness)
    cv2.line(img, (x1, y1 + r), (x1, y1 + r + d), color, thickness)
    cv2.ellipse(img, (x1 + r, y1 + r), (r, r), 180, 0, 90, color, thickness)

    # Top right drawing
    cv2.line(img, (x2 - r, y1), (x2 - r - d, y1), color, thickness)
    cv2.line(img, (x2, y1 + r), (x2, y1 + r + d), color, thickness)
    cv2.ellipse(img, (x2 - r, y1 + r), (r, r), 270, 0, 90, color, thickness)

    # Bottom left drawing
    cv2.line(img, (x1 + r, y2), (x1 + r + d, y2), color, thickness)
    cv2.line(img, (x1, y2 - r), (x1, y2 - r - d), color, thickness)
    cv2.ellipse(img, (x1 + r, y2 - r), (r, r), 90, 0, 90, color, thickness)

    # Bottom right drawing
    cv2.line(img, (x2 - r, y2), (x2 - r - d, y2), color, thickness)
    cv2.line(img, (x2, y2 - r), (x2, y2 - r - d), color, thickness)
    cv2.ellipse(img, (x2 - r, y2 - r), (r, r), 0, 0, 90, color, thickness)

while True:
    # read frames from live web cam stream
    (grabbed, frame) = stream.read()

    # resize the frames to be smaller and switch to gray scale
    frame = imutils.resize(frame, width=700)
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

    # Make copies of the frame for transparency processing
    overlay = frame.copy()
    output = frame.copy()

    # set transparency value
    alpha  = 0.5

    # detect faces in the gray scale frame
    face_rects = detector(gray, 0)

    # loop over the face detections
    for i, d in enumerate(face_rects):
        x1, y1, x2, y2, w, h = d.left(), d.top(), d.right() + 1, d.bottom() + 1, d.width(), d.height()

        # draw a fancy border around the faces
        draw_border(overlay, (x1, y1), (x2, y2), (162, 255, 0), 2, 10, 10)

    # make semi-transparent bounding box
    cv2.addWeighted(overlay, alpha, output, 1 - alpha, 0, output)

    # show the frame
    cv2.imshow("Face Detection", output)
    key = cv2.waitKey(1) & 0xFF

    # press q to break out of the loop
    if key == ord("q"):
        break

# cleanup
cv2.destroyAllWindows()
stream.stop()

That's copied from https://www.codemade.io/fast-and-accurate-face-tracking-in-live-video-with-python/, which was one of the tutorials I used as a reference when I wrote face_wake. If it works, it should display the video from your camera, and put a box around your face.

If that does not work, you could try https://github.com/shantnu/Webcam-Face-Detect. It should do the same thing (display video from your camera, and put a box around your face), but it uses an older method of facial detection (haar cascades). If this works, and the other does not, you can swap in the haar cascade method for the dlib method (dlib is default because it is generally more efficient and more accurate, but if the haar cascade works, you would only need to change https://github.com/ChristopherRogers1991/mycroft-face-wake/blob/master/main.py#L42 and https://github.com/ChristopherRogers1991/mycroft-face-wake/blob/master/main.py#L3 to use the haar cascade detector (https://github.com/ChristopherRogers1991/mycroft-face-wake/blob/master/object_watcher/object_watcher.py#L156))

If either or both of those don't work, you might check the angle of your camera, and your lighting. It's possible that one or both are contributing to difficulty identifying your face.

[patilaum]

yeah Its working fine I think there's no issue with my webcam. the issue lies in sendmessage function of your main.py

[ChristopherRogers1991]

You said no output was produced by object_watcher/object_watcher.py, correct? If that's not printing lines as you move in and out of frame, it's not recognizing your face. Also, in main.py, it [prints a message] (https://github.com/ChristopherRogers1991/mycroft-face-wake/blob/master/main.py) when it has detected a face and begins recording, then
transcribe_and_send prints the message it's going to send to Mycroft, so if you're not seeing that output, it's not even getting to the send_message function, so I do not think that is the issue.

One more thing you could try - comment out lines 33 and 34 in https://github.com/ChristopherRogers1991/mycroft-face-wake/blob/master/object_watcher/object_watcher.py. Those lines reduce the frame size of the video being captured, which was done for efficiency, but may also make it more difficult for your face to be detected.

[patilaum]

Yeah it worked ......object_watcher.py is showing output now... but foolowing error occured when I ran main.py

recording! done recording! Traceback (most recent call last): File "main.py", line 32, in transcribe_and_send text = transcriber.transcribe(path_to_source) File "/home/aum/mycroft-face-wake-master/stt/simple_stt.py", line 12, in transcribe return self.recognizer.recognize_google(audio) File "/home/aum/.local/lib/python3.5/site-packages/speech_recognition/init.py", line 858, in recognize_google if not isinstance(actual_result, dict) or len(actual_result.get("alternative", [])) == 0: raise UnknownValueError() speech_recognition.UnknownValueError

[ChristopherRogers1991]

Excellent! Now we're making progress.

That error happens if it's unable to transcribe the audio to text (which includes if you didn't say anything at all).

Try moving into frame, speaking a command, and then moving out of frame.

[patilaum]

this occurred now ...

Traceback (most recent call last): File "main.py", line 34, in transcribe_and_send send_message(text) File "main.py", line 25, in send_message ws = create_connection(url) File "/home/aum/.local/lib/python3.5/site-packages/websocket/_core.py", line 494, in create_connection websock.connect(url, **options) File "/home/aum/.local/lib/python3.5/site-packages/websocket/_core.py", line 217, in connect options.pop('socket', None)) File "/home/aum/.local/lib/python3.5/site-packages/websocket/_http.py", line 120, in connect sock = _open_socket(addrinfo_list, options.sockopt, options.timeout) File "/home/aum/.local/lib/python3.5/site-packages/websocket/_http.py", line 190, in _open_socket raise err File "/home/aum/.local/lib/python3.5/site-packages/websocket/_http.py", line 170, in _open_socket sock.connect(address) ConnectionRefusedError: [Errno 111] Connection refused

[ChristopherRogers1991]

Ok, now we're to the point where it's actually trying to send the message to Mycroft.

Is face_wake running on the same machine as Mycroft? Or are they running on separate machines (e.g. face_wake on your desktop, and Mycroft on on a mark1)?

[patilaum]

They are running on same machine...is it require to run mycroft while using face wake up

[ChristopherRogers1991]

Yes, both need to be running. After transcribing the speech to text, face_wake is just sending the text to Mycroft, and Mycroft handles everything from there.

On Wed, Jun 20, 2018, 8:53 AM patilaum notifications@github.com wrote:

They are running on same machine...is it require to run mycroft while using face wake up

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ChristopherRogers1991/mycroft-face-wake/issues/6#issuecomment-398738067, or mute the thread https://github.com/notifications/unsubscribe-auth/AINZv9Mi1EYeDpZUvy-eeyz9bU6Zi5UJks5t-kXkgaJpZM4UhaU_ .

[ChristopherRogers1991]

Hey @patilaum,

Just wanted to check in and see if you have any updates. With both mycroft and face_wake running, were you able to get this to work?

-Chris

[ChristopherRogers1991]

I'm going to go ahead and close this, since I haven't heard back. Feel free to reopen if this if still an issue.

-Chris

[patilaum]

sorry for delay......Its working now

[patilaum]

Thank you very much