Minimal demo - Githubissues

DylanVanAssche commented 2 years ago

Hi!

Thanks a lot for working on this! Your demo is really nice!

I have some questions about this on how to run it:

Is kernel 5.13 just fine without doing anything in https://github.com/tomasz-grobelny/AACS/blob/master/doc/INSTALL.md ? Are there specific kernel configs which must be applied to get this to work (as there a kernel config in the repo)?
Do you need Anbox/OsmAnd/AnyKeyboard to get this to work? Is there a more minimal demo for this? Would be great if this can just be external display.
As long there's OTG support, it is fine?
I don't need a phone with Android Auto running? This is fully standalone right?

I mostly want to try this to see if the hardware (Allwinner A64 quad core 1.1 GHz, 3GB RAM) is capable of doing Android Auto

rahulsangwn commented 2 years ago

Hi Dylan, I try to give answers based on my experience

No patching need for kernel 5.10+ (i'm using kernel 5.10 on raspberry pi 4 & it is working fine)
Optional Anbox/OsmAnd/AnyKeyboard, no need. You can use snowmix to inject custom video stream to head unit
Yes, currently running on pi 4, soon switch to pi zero W
If you are using AAClient (like proxying the android auto traffic) then it needs phone

DylanVanAssche commented 2 years ago

@rahulsangwn Thanks for your reply!

So it boils down to:

Build & setup AAServer for my device
Connect my device to a headunit with Android Auto support
AAServer automatically connects to the headunit and Android Auto starts
Do something with Snowmix to inject custom video

The last part seem vague, do you have some more information about that on how to use it in combination with the AAServer? Does it show up as yet another display for Linux?

rahulsangwn commented 2 years ago

I haven't tried snowmix, but I think you will get some idea by looking into the scripts folder.

DylanVanAssche commented 2 years ago

I compiled the various dependencies and AACS for my distro and I could run the AAServer.

However, I have 2 issues:

GStreamer seems to fail with (AAServer:7037): GStreamer-CRITICAL **: 20:53:07.074: gst_poll_remove_fd: assertion 'fd->fd >= 0' failed, Android Auto doesn't seem to show up in Crankshaft (an OpenAuto based headunit). I added some additional debug statements to locate the exact line where GStreamer seems to crash, but I couldn't. Did I forget anything here?
Yesterday (when I compiled it and could launch the AAServer), the AAServer didn't crash on a cert error. Today it does. I changed the date of the headunit and the phone to yesterday and it could authenticate again. No idea why it happens, the cert is valid until 27 Oct 2021.

Logs

ptr-8aby7wgvws40fjdoa6a:/etc/ssl/certs# AAServer 
Library init
init OK
Gadget::Gadget() usbg_create_gadget_vid_pid()  VID=4817 PID=4222 NAME=initial_state
Gadget constructor OK
Gadget::setStrings: Set manufacturer to TAG
Gadget::setStrings: Set product to AAServer
Gadget::setStrings: Set serial number to TAGAAS
setStrings OK
MassStorageFunction: usbg_create_function
MassStorageFunction: usbg_create_function OK
FfsFunction create function
FfsFunction OK
usbg_create_config for NAME=c0
CONFIG OK
addFunction for NAME=massstorage_initial
addFunction OK
addFunction for NAME=loopback_initial
addFunction OK
Enabling gadget
Enabling OK
pre read
post read: 12
pre hSM

post hSM: 12
pre read
post read: 12
pre hSM

post hSM: 12
pre read
post read: 12
pre hSM
Got 51, write=2
post hSM: 12
pre read
post read: 12
pre hSM
Got some info: 0=
post hSM: 12
pre read
post read: 8
pre hSM
post hSM: 8
pre read
post read: 12
pre hSM
Got some info: 1=post hSM: 12
pre read
post read: 13
pre hSM
Android Auto
post hSM: 13
pre read
post read: 12
pre hSM
Got some info: 2=post hSM: 12
pre read
post read: 13
pre hSM
Android Auto
post hSM: 13
pre read
post read: 12
pre hSM
Got some info: 3=
post hSM: 12
pre read
post read: 6
pre hSM
post hSM: 6
pre read
post read: 12
pre hSM
Got some info: 4=post hSM: 12
pre read
post read: 22
pre hSM
https://f1xstudio.com
post hSM: 22
pre read
post read: 12
pre hSM
Got some info: 5=post hSM: 12
pre read
post read: 13
pre hSM
HU-AAAAAA001
post hSM: 13
pre read
post read: 12
pre hSM
Got 53, exit
post hSM: 12
pre read
post read: -1
pre close fd
post close fd
~Gadget destroy
~Gadget destroy OK
DefaultChannelHandler: 0
dumpfile: 
Gadget::Gadget() usbg_create_gadget_vid_pid()  VID=6353 PID=11520 NAME=main_state
Gadget constructor OK
Gadget::setStrings: Set manufacturer to TAG
Gadget::setStrings: Set product to AAServer
Gadget::setStrings: Set serial number to TAGAAS
setStrings OK
FfsFunction create function
FfsFunction OK
usbg_create_config for NAME=c0
CONFIG OK
addFunction for NAME=ffs_main
addFunction OK
Enabling gadget
ep0 event 0 
Enabling OK
ep0 event 2 
got version request
version negotiation ok
auth complete
got service discovery response
channels {
  channel_id: 7
  media_input_channel {
    stream_type: Audio
    audio_config {
      sample_rate: 16000
      bits_per_sample: 16
      channel_count: 1
    }
  }
}
channels {
  channel_id: 4
  media_channel {
    media_type: Audio
    audio_type: Media
    audio_configs {
      sample_rate: 48000
      bits_per_sample: 16
      channel_count: 2
    }
    5: 1
  }
}
channels {
  channel_id: 5
  media_channel {
    media_type: Audio
    audio_type: Speech
    audio_configs {
      sample_rate: 16000
      bits_per_sample: 16
      channel_count: 1
    }
    5: 1
  }
}
channels {
  channel_id: 6
  media_channel {
    media_type: Audio
    audio_type: System
    audio_configs {
      sample_rate: 16000
      bits_per_sample: 16
      channel_count: 1
    }
    5: 1
  }
}
channels {
  channel_id: 2
  sensor_channel {
    sensors {
      type: DrivingStatus
    }
    sensors {
      type: Location
    }
    sensors {
      type: NightData
    }
  }
}
channels {
  channel_id: 3
  media_channel {
    media_type: Video
    video_configs {
      video_resolution: H720
      video_fps: F30
      margin_width: 0
      margin_height: 0
      dpi: 140
    }
    5: 1
  }
}
channels {
  channel_id: 1
  input_channel {
    screen_config {
      width: 1920
      height: 1080
    }
  }
}
channels {
  channel_id: 14
  16 {
    1: "CRANKSHAFT-NG"
  }
}
2: "Crankshaft-NG"
3: "Universal"
4: "2018"
5: "20180301"
6: 1
7: "f1x"
8: "Crankshaft-NG Autoapp"
9: "1"
10: "1.0"
11: 0
12: 0

DefaultChannelHandler: 7
DefaultChannelHandler: 4
DefaultChannelHandler: 5
DefaultChannelHandler: 6
DefaultChannelHandler: 2
********************************************
VideoChannelHandler: 3
pipeline created
app_sink created
queue OK
videoconvert OK
videoscale OK
videorate OK
x264enc OK
h264caps OK
capsfilter OK
rawcaps OK
capsfilter_pre OK
shmsrc OK
queue_snowmix OK
snowmixcaps OK
capsfilter_snowmix OK
gst_bin_add_many OK
gst_element_link_many OK
bus OK
add signal OK
signal connect OK

(AAServer:7037): GStreamer-CRITICAL **: 20:53:07.074: gst_poll_remove_fd: assertion 'fd->fd >= 0' failed
set_state OK
********************************************
InputChannelHandler: 1
DefaultChannelHandler: 14

paulhothersall commented 2 years ago

Hi Dylan, I try to give answers based on my experience

No patching need for kernel 5.10+ (i'm using kernel 5.10 on raspberry pi 4 & it is working fine)

Optional Anbox/OsmAnd/AnyKeyboard, no need. You can use snowmix to inject custom video stream to head unit

Yes, currently running on pi 4, soon switch to pi zero W

If you are using AAClient (like proxying the android auto traffic) then it needs phone

interested in how you got it going on the Pi4 , and plans for the pi Zero W.

I am having an issue building AACS on Raspbian, and whilst I can build in Ubuntu 21.04 64bit I can't get it the USB ORG working!!

Interested in collaborating on this?

rahulsangwn commented 2 years ago

Put dtoverlay=dwc2 in \boot\config.txt and run sudo modprobe libcomposite after boot.

tomasz-grobelny commented 2 years ago

The "gst_poll_remove_fd: assertion 'fd->fd >= 0' failed" seems to suggest issues with shmsrc GStreamer plugin and this one is used for communication with snowmix - do you have snowmix configured correctly and running?

DylanVanAssche commented 2 years ago

@tomasz-grobelny I only compiled and installed snowmix, I didn't do any configuration. I thought it would 'just work', but apparently not ;)

Is this the config file I need? https://github.com/tomasz-grobelny/AACS/blob/master/scripts/base.ini

Anything else?

tomasz-grobelny commented 2 years ago

Yes, just starting snowmix within this config file should be enough. But in any such cases its good to refer to startup scripts, in this case scripts/start:14-19.

DylanVanAssche commented 2 years ago

@tomasz-grobelny Finally had some time to try this. The GStreamer error goes away but Snowmix seems to fail:

phone:/home/dylan/snowmix# snowmix base.ini 
Warning. Environment variable SNOWMIX_PREFIX was not set. Using SNOWMIX_PREFIX=.snowmix
Snowmix version 0.5.1.1.
Copyright Peter Maersk-Moller 2012-2015.
Search Path : /home/dylan/snowmix, /root/.snowmix, /usr/lib/Snowmix-0.5.1.1/
Controller opened file /home/dylan/snowmix/base.ini for reading.
MSG: Invalid number of parameters: "feed add 1 Camera 1 "
MSG: Invalid feed no: 1 in feed_set_geometry
MSG: Invalid number of parameters: "feed geometry 1 800 480 "
MSG: Invalid feed no: 1 in feed_set_live
MSG: Invalid number of parameters: "feed live 1 "
MSG: Invalid feed no: 1 in feed_set_idle
MSG: Invalid number of parameters: "feed idle 1 30 dead_frame.bgra "
MSG: Invalid feed no: 1 in feed_set_socket
MSG: Invalid number of parameters: "feed socket 1 /tmp/aacs_feed1 "
MSG: Invalid number of parameters: "virtual feed source feed 1 1 "
Initial ini file read

I used https://github.com/tomasz-grobelny/AACS/blob/master/scripts/base.ini though and checked these commands in the Snowmix docs, they look fine.

DylanVanAssche commented 2 years ago

@tomasz-grobelny I had to patch Snowmix command parser for musl, the way Snowmix parses some commands wasn't valid for more strict libc libraries like musl. That part works now fine (no errors anymore), but still nothing in Crankshaft NG. I only see the 'Lock' icon in left upper corner when AAServer is ready.

If no video stream is feed into snowmix, it should give the dead_frame.bgra right to the headunit?

tomasz-grobelny commented 2 years ago

Not sure why you needed to patch snowmix... anyway, good to hear you managed to get rid of the error. Yes, it should show dead_frame.bgra - did you create the file (scripts/setup:21)?

DylanVanAssche commented 2 years ago

@tomasz-grobelny

Not sure why you needed to patch snowmix...

It's how they parse several commands, glibc is more forgiving than musl. I patched it and now the commands are properly read with musl systems as well.

Yes, it should show dead_frame.bgra - did you create the file (scripts/setup:21)?

Yes, it's in /home/dylan/snowmix/. I would debug myself a little but I don't find any errors, logs or anything else so I'm bit in the dark with Snowmix. Any suggestions on how this part can be debugged?

tomasz-grobelny commented 2 years ago

Any suggestions on how this part can be debugged?

First I would check the snowmix part by writing a gstreamer pipeline to cast snowmix output to screen (autovideosink). You should be ale to find examples in snowmix documentation.

Second thing I noticed is that your crankshaft uses h720 resolution, whereas default for aacs is h480. Can you check if crankshaft can be at least temporarily reconfigured for 480 lines resolution?

DylanVanAssche commented 2 years ago

@tomasz-grobelny

I tried to use output2screen as mentioned in the Snowmix docs but it crashes the device's UI. Complains about some connection to X (XServer problably), but this is wayland-only. Not sure if this whole setup even works under Wayland...

I had a look at GStreamer 's pipelines but that's way above my current capabilities.

Second thing I noticed is that your crankshaft uses h720 resolution, whereas default for aacs is h480. Can you check if crankshaft can be at least temporarily reconfigured for 480 lines resolution?

No difference :/ I also looked into the logs of OpenAuto on the Crankshaft side and that shows the same: setup and connections are there. It's just that Crankshaft doesn't show anything as video output.

tomasz-grobelny commented 2 years ago

@DylanVanAssche, GStreamer is quite an important part here... certainly there is a way to display some video on your device using gstreamer, even if it is wayland only. And even if not you could try streaming the snowmix output over network and display on another device. Some GStreamer knowledge will be needed here as I don't have ready solutions unfortunately.

Video compatibility seems to be an issue, not only in this case. If I don't have any logs from decoders on receiving side it is hard to tell anything. Do you know what actually decodes video stream? Eg. if it is gstreamer then maybe it is possible to set some env variables to increase output verbosity. Same may be true for other libs.

tomasz-grobelny / AACS

Minimal demo #16