search

rhysmorgan134 / react-carplay

MIT License
381 stars 58 forks source link

Microphone only works after "hey siri" #28

Closed seth602 closed 8 months ago

seth602 commented 1 year ago

Audio works great, but the microphone is only listening after "hey siri". Even to confirm a message it stops listening. On phone calls the mic is not listening. I had a mic plugged into the USB, but I realized its not listening through it when i unplugged it and got the same behavior. I know some of the newer dongles have a mic, and/or it uses the phone's mic.

v3.0.1 Raspberry pi 4 on Bullseye Carlink model: Model CPC200-CCPA

rhysmorgan134 commented 1 year ago

Thanks Seth,

Phonecall integration needs to be completed still, however it should work fine with siri and responding to messaged. Does it work if you activate siri by holding down on the screen?

seth602 commented 1 year ago

good to know about the phonecall integration. let me know whatever testing you want done for that. Even when holding down the screen to activate siri it still doesn't listen. It only listens when i say "hey siri" and give it a command. it doesn't listen for follow up

rhysmorgan134 commented 1 year ago

@seth602 are you running this on a raspberry pi? If so could you post the output of arecord please?

seth602 commented 1 year ago

Apologies for the delay, I had to re-image the SD card for my raspberry pi.

When I run arecord by itself after testing out Siri on Carplay it gives me this:

Usage: arecord [OPTION]... [FILE]...

-h, --help              help
    --version           print current version
-l, --list-devices      list all soundcards and digital audio devices
-L, --list-pcms         list device names
-D, --device=NAME       select PCM by name
-q, --quiet             quiet mode
-t, --file-type TYPE    file type (voc, wav, raw or au)
-c, --channels=#        channels
-f, --format=FORMAT     sample format (case insensitive)
-r, --rate=#            sample rate
-d, --duration=#        interrupt after # seconds
-s, --samples=#         interrupt after # samples per channel
-M, --mmap              mmap stream
-N, --nonblock          nonblocking mode
-F, --period-time=#     distance between interrupts is # microseconds
-B, --buffer-time=#     buffer duration is # microseconds
    --period-size=#     distance between interrupts is # frames
    --buffer-size=#     buffer duration is # frames
-A, --avail-min=#       min available space for wakeup is # microseconds
-R, --start-delay=#     delay for automatic PCM start is # microseconds 
                        (relative to buffer size if <= 0)
-T, --stop-delay=#      delay for automatic PCM stop is # microseconds from xrun
-v, --verbose           show PCM structure and setup (accumulative)
-V, --vumeter=TYPE      enable VU meter (TYPE: mono or stereo)
-I, --separate-channels one file for each channel
-i, --interactive       allow interactive operation from stdin
-m, --chmap=ch1,ch2,..  Give the channel map to override or follow
    --disable-resample  disable automatic rate resample
    --disable-channels  disable automatic channel conversions
    --disable-format    disable automatic format conversions
    --disable-softvol   disable software volume control (softvol)
    --test-position     test ring buffer position
    --test-coef=#       test coefficient for ring buffer position (default 8)
                        expression for validation is: coef * (buffer_size / 2)
    --test-nowait       do not wait for ring buffer - eats whole CPU
    --max-file-time=#   start another output file when the old file has recorded
                        for this many seconds
    --process-id-file   write the process ID here
    --use-strftime      apply the strftime facility to the output file name
    --dump-hw-params    dump hw_params of the device
    --fatal-errors      treat all errors as fatal
Recognized sample formats are: S8 U8 S16_LE S16_BE U16_LE U16_BE S24_LE S24_BE U24_LE U24_BE S32_LE S32_BE U32_LE U32_BE FLOAT_LE FLOAT_BE FLOAT64_LE FLOAT64_BE IEC958_SUBFRAME_LE IEC958_SUBFRAME_BE MU_LAW A_LAW IMA_ADPCM MPEG GSM S20_LE S20_BE U20_LE U20_BE SPECIAL S24_3LE S24_3BE U24_3LE U24_3BE S20_3LE S20_3BE U20_3LE U20_3BE S18_3LE S18_3BE U18_3LE U18_3BE G723_24 G723_24_1B G723_40 G723_40_1B DSD_U8 DSD_U16_LE DSD_U32_LE DSD_U16_BE DSD_U32_BE
Some of these may not be available on selected hardware
The available format shortcuts are:
-f cd (16 bit little endian, 44100, stereo)
-f cdr (16 bit big endian, 44100, stereo)
-f dat (16 bit little endian, 48000, stereo)

When I run ./ReactCarplay-3.0.1-armv7l.AppImage --no-sandbox arecord and test out using Siri I get:

/home/raspberry/.config/ReactCarplay
spawning carplay {
  dpi: 240,
  nightMode: 0,
  hand: 0,
  boxName: 'nodePlay',
  width: 1920,
  height: 1080,
  fps: 30
}
directory found
sending dpi
device connected and ready
unkown type:  24  with data::  <Buffer aa 55 aa 55 71 00 00 00 18 00 00 00 e7 ff ff ff>
unkown type:  38  with data::  <Buffer aa 55 aa 55 00 00 00 00 26 00 00 00 d9 ff ff ff>
unkown type:  1  with data::  <Buffer aa 55 aa 55 00 00 00 00 01 00 00 00 fe ff ff ff>
parsing type:  8  for length:  4 <Buffer aa 55 aa 55 04 00 00 00 08 00 00 00 f7 ff ff ff>
parsing 8 <Buffer f2 03 00 00>
Carplay message:  undefined
test message r
starting projection
parsing type:  8  for length:  4 <Buffer aa 55 aa 55 04 00 00 00 08 00 00 00 f7 ff ff ff>
parsing 8 <Buffer e8 03 00 00>
Carplay message:  Support Wifi
unkown type:  13  with data::  <Buffer aa 55 aa 55 0d 00 00 00 0d 00 00 00 f2 ff ff ff>
unkown type:  14  with data::  <Buffer aa 55 aa 55 0d 00 00 00 0e 00 00 00 f1 ff ff ff>
unkown type:  18  with data::  <Buffer aa 55 aa 55 1e 00 00 00 12 00 00 00 ed ff ff ff>
parsing type:  8  for length:  4 <Buffer aa 55 aa 55 04 00 00 00 08 00 00 00 f7 ff ff ff>
parsing 8 <Buffer e9 03 00 00>
Carplay message:  undefined
test message i
unkown type:  24  with data::  <Buffer aa 55 aa 55 71 00 00 00 18 00 00 00 e7 ff ff ff>
parsing type:  8  for length:  4 <Buffer aa 55 aa 55 04 00 00 00 08 00 00 00 f7 ff ff ff>
parsing 8 <Buffer 07 00 00 00>
Carplay message:  undefined
test message 
parsing type:  204  for length:  32 <Buffer aa 55 aa 55 20 00 00 00 cc 00 00 00 33 ff ff ff>
parsing 204 <Buffer 32 30 32 32 2e 30 34 2e 32 35 2e 31 33 31 37 43 41 59 00 00 00 00 00 00 00 00 00 00 00 00 00 00>
version number:  2022.04.25.1317CAY
unkown type:  25  with data::  <Buffer aa 55 aa 55 7a 00 00 00 19 00 00 00 e6 ff ff ff>
unkown type:  1  with data::  <Buffer aa 55 aa 55 1c 00 00 00 01 00 00 00 fe ff ff ff>
[2394:0210/125821.416746:ERROR:gbm_wrapper.cc(255)] Failed to export buffer to dma_buf: No such file or directory (2)
[2394:0210/125821.417603:ERROR:gbm_wrapper.cc(255)] Failed to export buffer to dma_buf: No such file or directory (2)
[2394:0210/125821.418158:ERROR:gbm_wrapper.cc(255)] Failed to export buffer to dma_buf: No such file or directory (2)
[2394:0210/125821.418694:ERROR:gbm_wrapper.cc(255)] Failed to export buffer to dma_buf: No such file or directory (2)
[2394:0210/125821.419321:ERROR:gbm_wrapper.cc(255)] Failed to export buffer to dma_buf: No such file or directory (2)
[2394:0210/125821.419865:ERROR:gbm_wrapper.cc(255)] Failed to export buffer to dma_buf: No such file or directory (2)
[2394:0210/125821.420387:ERROR:gbm_wrapper.cc(255)] Failed to export buffer to dma_buf: No such file or directory (2)
[2394:0210/125821.420911:ERROR:gbm_wrapper.cc(255)] Failed to export buffer to dma_buf: No such file or directory (2)
[2394:0210/125821.421420:ERROR:gbm_wrapper.cc(255)] Failed to export buffer to dma_buf: No such file or directory (2)
[2394:0210/125821.421927:ERROR:gbm_wrapper.cc(255)] Failed to export buffer to dma_buf: No such file or directory (2)
[2394:0210/125821.422431:ERROR:gbm_wrapper.cc(255)] Failed to export buffer to dma_buf: No such file or directory (2)
[2394:0210/125821.422928:ERROR:gbm_wrapper.cc(255)] Failed to export buffer to dma_buf: No such file or directory (2)
[2394:0210/125821.423479:ERROR:gbm_wrapper.cc(255)] Failed to export buffer to dma_buf: No such file or directory (2)
[2394:0210/125821.424065:ERROR:gbm_wrapper.cc(255)] Failed to export buffer to dma_buf: No such file or directory (2)
[2394:0210/125821.427103:ERROR:gbm_wrapper.cc(255)] Failed to export buffer to dma_buf: No such file or directory (2)
[2394:0210/125821.427705:ERROR:gbm_wrapper.cc(255)] Failed to export buffer to dma_buf: No such file or directory (2)
[2394:0210/125821.428229:ERROR:gbm_wrapper.cc(255)] Failed to export buffer to dma_buf: No such file or directory (2)
[2394:0210/125821.428740:ERROR:gbm_wrapper.cc(255)] Failed to export buffer to dma_buf: No such file or directory (2)
[2394:0210/125821.429326:ERROR:gbm_wrapper.cc(255)] Failed to export buffer to dma_buf: No such file or directory (2)
[2394:0210/125821.429888:ERROR:gbm_wrapper.cc(255)] Failed to export buffer to dma_buf: No such file or directory (2)
[2394:0210/125821.430430:ERROR:gbm_wrapper.cc(255)] Failed to export buffer to dma_buf: No such file or directory (2)
enabling wifi
auto connecting
parsing type:  8  for length:  4 <Buffer aa 55 aa 55 04 00 00 00 08 00 00 00 f7 ff ff ff>
parsing 8 <Buffer f2 03 00 00>
Carplay message:  undefined
test message r
parsing type:  2  for length:  4 <Buffer aa 55 aa 55 04 00 00 00 02 00 00 00 fd ff ff ff>
parsing 2 <Buffer 03 00 00 00>
no wifi avail, phone type:  3
parsing type:  8  for length:  24 <Buffer aa 55 aa 55 18 00 00 00 08 00 00 00 f7 ff ff ff>
parsing 8 <Buffer 04 00 00 00 46 43 3a 41 41 3a 38 31 3a 45 46 3a 31 36 3a 38 46 33 31 2e>
Carplay message:  undefined
test message FC:AA:81:EF:16:8F31.
unkown type:  3  with data::  <Buffer aa 55 aa 55 04 00 00 00 03 00 00 00 fc ff ff ff>
[2394:0210/125826.491860:ERROR:gl_utils.cc(315)] [.RenderCompositor-0x3d22d100] GL_INVALID_FRAMEBUFFER_OPERATION: Framebuffer is incomplete: Attachment is not renderable.
[2394:0210/125826.504453:ERROR:gl_utils.cc(315)] [.RenderCompositor-0x3d22d100] GL_INVALID_FRAMEBUFFER_OPERATION: Framebuffer is incomplete: Attachment is not renderable.
[2394:0210/125826.509164:ERROR:gl_utils.cc(315)] [.RenderCompositor-0x3d22d100] GL_INVALID_FRAMEBUFFER_OPERATION: Framebuffer is incomplete: Attachment is not renderable.
[2394:0210/125826.584187:ERROR:gl_utils.cc(315)] [.RenderCompositor-0x3d22d100] GL_INVALID_FRAMEBUFFER_OPERATION: Framebuffer is incomplete: Attachment is not renderable.
[2394:0210/125826.594845:ERROR:gl_utils.cc(315)] [.RenderCompositor-0x3d22d100] GL_INVALID_FRAMEBUFFER_OPERATION: Framebuffer is incomplete: Attachment is not renderable.
[2394:0210/125826.597982:ERROR:gl_utils.cc(315)] [.RenderCompositor-0x3d22d100] GL_INVALID_FRAMEBUFFER_OPERATION: Framebuffer is incomplete: Attachment is not renderable.
14 0.0453125 0.9388888888888889
non-audio found
unknown audio type:  3 false false
parsing type:  8  for length:  4 <Buffer aa 55 aa 55 04 00 00 00 08 00 00 00 f7 ff ff ff>
parsing 8 <Buffer 01 00 00 00>
Carplay message:  undefined
test message 
received 0 event
undefined
received 0 {}
parsing type:  0  for length:  0 <Buffer aa 55 aa 55 00 00 00 00 00 00 00 00 ff ff ff ff>
non-audio found
starting mic
non-audio found
unknown audio type:  1 false false
<Buffer 61 72 65 63 6f 72 64 3a 20 6d 61 69 6e 3a 38 33 30 3a 20>
<Buffer 61 75 64 69 6f 20 6f 70 65 6e 20 65 72 72 6f 72 3a 20 4e 6f 20 73 75 63 68 20 66 69 6c 65 20 6f 72 20 64 69 72 65 63 74 6f 72 79 0a>
16 0.0453125 0.9388888888888889
parsing type:  8  for length:  4 <Buffer aa 55 aa 55 04 00 00 00 08 00 00 00 f7 ff ff ff>
parsing 8 <Buffer 02 00 00 00>
Carplay message:  undefined
test message 
received 0 event
undefined
received 0 {}
parsing type:  0  for length:  0 <Buffer aa 55 aa 55 00 00 00 00 00 00 00 00 ff ff ff ff>
non-audio found
non-audio found
unknown audio type:  2 false false
non-audio found
unknown audio type:  10 false false
non-audio found
unknown audio type:  1 false false
non-audio found
unknown audio type:  11 false false
non-audio found
unknown audio type:  2 false false
non-audio found
unknown audio type:  3 false false
parsing type:  8  for length:  4 <Buffer aa 55 aa 55 04 00 00 00 08 00 00 00 f7 ff ff ff>
parsing 8 <Buffer 01 00 00 00>
Carplay message:  undefined
test message 
received 0 event
undefined
received 0 {}
parsing type:  0  for length:  0 <Buffer aa 55 aa 55 00 00 00 00 00 00 00 00 ff ff ff ff>
non-audio found
starting mic
non-audio found
unknown audio type:  1 false false
<Buffer 61 72 65 63 6f 72 64 3a 20 6d 61 69 6e 3a 38 33 30 3a 20 61 75 64 69 6f 20 6f 70 65 6e 20 65 72 72 6f 72 3a 20 4e 6f 20 73 75 63 68 20 66 69 6c 65 20 ... 13 more bytes>
parsing type:  8  for length:  4 <Buffer aa 55 aa 55 04 00 00 00 08 00 00 00 f7 ff ff ff>
parsing 8 <Buffer 02 00 00 00>
Carplay message:  undefined
test message 
received 0 event
undefined
received 0 {}
parsing type:  0  for length:  0 <Buffer aa 55 aa 55 00 00 00 00 00 00 00 00 ff ff ff ff>
non-audio found
non-audio found
unknown audio type:  2 false false
seth602 commented 1 year ago

Also, see output below from listing available recording devices:

raspberry@raspberrypi:~/Documents/Carplay $ arecord -l
**** List of CAPTURE Hardware Devices ****
card 3: Device [Plugable USB Audio Device], device 0: USB Audio [USB Audio]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
rhysmorgan134 commented 1 year ago

Can you try without —no-sandbox? I doubt it will make a difference, but I tested last night on Mac OS, and seems to be working fine

rhysmorgan134 commented 1 year ago

If not I will get a fresh pi imaged

seth602 commented 1 year ago

no difference without --no-sandbox. I tested the mic to make sure it works on rpi and all is fine. im going to try updating the /home/pi configuration to make sure it's using the right card and device numbers