gartnera / headunit

Headunit for Android Auto
GNU Affero General Public License v3.0
347 stars 90 forks source link

Refactor AA protocol code to use protocol buffers #16

Open lmagder opened 8 years ago

lmagder commented 8 years ago

I did some preliminary investigation into writing proto files instead of manually constructing the wire protocol, but only with some messages sent by the headunit to the phone so far.

For example in the function I added to send a button message

int hu_fill_button_message(uint8_t* buffer, uint64_t timeStamp, HU_INPUT_BUTTON button, int isPress)
{
    int buffCount = 0;
    buffer[buffCount++] = 0x80;
    buffer[buffCount++] = 0x01;
    buffer[buffCount++] = 0x08;

    buffCount += varint_encode(timeStamp, buffer + buffCount, 0);

    buffer[buffCount++] = 0x22;
    buffer[buffCount++] = 0x0A;
    buffer[buffCount++] = 0x0A;
    buffer[buffCount++] = 0x08;
    buffer[buffCount++] = 0x08;
    buffer[buffCount++] = (uint8_t)button;
    buffer[buffCount++] = 0x10;
    buffer[buffCount++] = isPress ? 0x01 : 0x00;
    buffer[buffCount++] = 0x18;
    buffer[buffCount++] = 0x00;
    buffer[buffCount++] = 0x20;
    buffer[buffCount++] = 0x00;
    return buffCount;
}

I saved this out with timestamp = 15, button = 32, isPress = true as buttonMessage.bin. Running this through protoc --decode_raw < buttonMessage.bin yields

1: 1
1: 15
4 {
  1 {
    1: 32
    2: 1
    3: 0
    4: 0
  }
}

so actually you can see there are two messages concatenated here (since there can't be two field #1s). Assuming all the fields are there the first field is pretty simple, something like

message Header
{
    enum PacketType
    {
        INPUT_EVENT = 1;
        //More here probably
    }
    required PacketType type = 1;
}

so I chopped it off to get

protoc --decode_raw < buttonMessageNoHeader.bin 
1: 15
4 {
  1 {
    1: 32
    2: 1
    3: 0
    4: 0
  }
}

so just making up names assuming what the code does and inferring types from the wire format since unfortunately protoc does not print them, despite knowing them.

message ButtonInfo
{
    required uint32 scanCode = 1;
    required bool pressed = 2;
    required uint32 unknown1 = 3 [default = 0];
    required uint32 unknown2 = 4 [default = 0];
}

message ButtonInfoWrapper
{
    required ButtonInfo button = 1;
}

message InputEvent
{
    required uint64 timeStamp = 1;
    optional ButtonInfoWrapper button = 4;
}

seems to work:

protoc test.proto --decode HU.InputEvent < buttonMessageNoHeader.bin 
timeStamp: 15
button {
  button {
    scanCode: 32
    pressed: true
    unknown1: 0
    unknown2: 0
  }
}

I did a few more like the touch event and the day/night sensor notification and you can start to see patterns. The first packet is always a single enum which is the type of second packet and then all the input events are one packet type with optional sub-structs, same with the sensor event. So it seems you can send all or any subset of the sensors at once. I attached the files.

This would definitely clean up code a lot (especially the sd_buf stuff in hu_app.c) and it shouldn't add runtime requirements since all the protoc stuff generates code you compile and link against. The only potential problem is that protoc generates C++ code by default, which is probably nicer and would work since input_filter uses modern C++, but would be a drastic change. There is a unofficial protoc-c project though testFiles.zip

lmagder commented 7 years ago

Got farther in decoding more of the init message into proto code. Turns out the Linux Google desktop head unit binary is not stripped, so if you look at the symbols and know the structure of the code protoc generates you can get info:

syntax = "proto2";

package HU;

message Header
{
    enum PacketType
    {
        INPUT_EVENT = 1;
        SENSOR_EVENT = 3;
    }
    required PacketType type = 1;
}

message ButtonInfo
{
    required uint32 scanCode = 1;
    required bool isPressed = 2;
    required uint32 meta = 3;
    required bool longPress = 4;
}

message ButtonInfoWrapper
{
    repeated ButtonInfo button = 1;
}

message TouchInfo
{
    enum TOUCH_ACTION
    {
        RELEASE = 0;
        PRESS = 1;
        DRAG = 2;
    }
    message Location
    {
        required uint32 x = 1;
        required uint32 y = 2;
        required uint32 pointerId = 3;
    }
    repeated Location location = 1;
    required uint32 actionIndex = 2;
    required TOUCH_ACTION action = 3;
}

message InputEvent
{
    required uint64 timeStamp = 1;
    optional int32 dispChannel = 2;
    optional TouchInfo touch = 3;
    optional ButtonInfoWrapper button = 4;
}

message SensorEvent
{
    message NightMode
    {
        required bool isNight = 1;
    }
    repeated NightMode nightMode = 10;
}

enum CHANNEL_TYPE
{
    AA_CH_CTR = 0;    
    AA_CH_SEN = 1;
    AA_CH_VID = 2;
    AA_CH_TOU = 3;
    AA_CH_AUD = 4;
    AA_CH_AU1 = 5;
    AA_CH_AU2 = 6;
    AA_CH_MIC = 7;
}

enum SENSOR_TYPE
{
    SENSOR_TYPE_DRIVING_STATUS = 11;    
    SENSOR_TYPE_NIGHT_DATA = 10;
    SENSOR_TYPE_RPM = 3;
    SENSOR_TYPE_DIAGNOSTICS = 8;
    SENSOR_TYPE_GEAR = 7;
    SENSOR_TYPE_COMPASS = 1;
    SENSOR_TYPE_LOCATION = 9;
}

enum AUDIO_TYPE
{
    SPEECH = 1;
    SYSTEM = 2;
    MEDIA = 3;
}

enum STREAM_TYPE
{
    AUDIO = 1;
    VIDEO = 3;   
}

message AudioCofig
{
    required uint32 sampleRate = 1;
    required uint32 bitDepth = 2;
    required uint32 channelCount = 3;
}

message ChannelDescriptor
{
    required uint32 channelId = 1;
    message SensorChannel
    {
        message Sensor
        {
            required SENSOR_TYPE type = 1;
        }
        repeated Sensor sensorList = 1;
    }
    optional SensorChannel sensorChannel = 2;

    message OutputStreamChannel
    {
        required STREAM_TYPE type = 1;
        optional AUDIO_TYPE audioType = 2;
        repeated AudioCofig audioConfigs = 3;

        message VideoConfig
        {
            enum VIDEO_RESOLUTION
            {
                VR_800x480 = 1;
                VR_1280x720 = 2;
                VR_1920x1080 = 3;
            }

            enum VIDEO_FPS
            {
                VFPS_30 = 1;
                VFPS_60 = 2;
            }
            required VIDEO_RESOLUTION resolution = 1;
            required VIDEO_FPS frameRate = 2;
            required uint32 marginWidth = 3;
            required uint32 marginHeight = 4;
            required uint32 dpi = 5;
            optional uint32 additionalDepth = 6;
        }
        repeated VideoConfig videoConfigs = 4;
        optional bool availableWhileInCall = 5;
    }
    optional OutputStreamChannel outputStreamChannel = 3;

    message InputEventChannel
    {
        message TouchScreenConfig
        {
            required uint32 width = 1;
            required uint32 height = 2;
        }
        repeated uint32 keycodesSupported = 1;
        optional TouchScreenConfig touchScreenConfig = 2;
        optional TouchScreenConfig touchPadConfig = 3;
    }

    optional InputEventChannel inputEventChannel = 4;

    message InputStreamChannel
    {
        required STREAM_TYPE type = 1;
        required AudioCofig audioConfig = 2;
        optional bool availableWhileInCall = 3;
    }

    optional InputStreamChannel inputStreamChannel = 5;

    //bt service == 6
    //radio == 7
    //nav==8
    //mediaplayback == 9
    //phonestatus = 10
    //mediabrowser=11
    //vendor extension==12
    //genericnotification==13
}

message CarInfo
{
    repeated ChannelDescriptor channels = 1;
    required string HeadUnitName = 2;
    required string CarModel = 3;
    required string CarYear = 4;
    required string CarSerial = 5;
    required bool DriverPos = 6;
    required string HeadUnitMake = 7;
    required string HeadUnitModel = 8;
    required string SWBuild = 9;
    required string SWVersion = 10;
    required bool canPlayNativeMediaDuringVr = 11;
    required bool hideClock = 12;
}
izacus commented 7 years ago

This looks really really promising for less headaches in the source.

(I'd archive the Linux binary before Google notices and removes it though :P )

borconi commented 7 years ago

Yes it does look good, the only problem is for the love of my life I was never able to understand the protobuf thing, I'm not saying that I know too much about programing or so, but with the original code I kind'a can understand what I'm doing, and how to manipulate / construct the message, all through still having some difficulties figuring out everything.

Here is some more information I managed to figure out, if GPS sensor data can be fully decoded it means the phone can take advantage of the car's GPS rather than relying on the phone GPS. Also injecting a "speed" of 0.001 as below brings up the unlimited browsing in AA. For example GPS sensor data:

        byte[] data = new byte[17];
        data[0]=(byte)8;
        data[1]=(byte)0x80;
        data[2]=(byte)0x03;
        data[3]=(byte)0x0a;
        data[4]=(byte)12;
        data[5]=(byte)0x00;
        hu_uti.varint_encode (86400000000000L, data,  5);
        data[13]=(byte)0x30;
        data[14]=(byte)0x01;
        data[15]=(byte)0x20;
        data[16]=(byte)0x01;

data[3]=(byte)0x0a - Is indicating the Location sensor: data[4]=(byte)12 - The total size sent 1day in future after that,

This translates to velocity of 0.001 (aka we are NOT moving)

        data[13]=(byte)0x30;
        data[14]=(byte)0x01;

This on the other hand translates to a suspicious accuracy of 0.001 (i use to inject this so AA doesn't take the location received from the car in consideration, otherwise it will put you in the middle of the ocean at Lat: 0, Long: 0)

    data[15]=(byte)0x20;
    data[16]=(byte)0x01;

I figured out the the (byte)0x30 and (byte)0x20 are following a pattern, every time increment of 8. Hexa values below: 08 - Not sure what it is 10 - Latitude? 18 - Longitude?

borconi commented 7 years ago

For aa version 1.6 drive status 0 was enough, since aa 2.0 it does check on the GPS data sent over by the car..... Grrrrrrr.... If no data sent over the speed is considered null which means car not parked, what a crazy logic.

Yeah I know what's the concept of protobuf but somehow they are not designed for my brain.... You know when you just cannot make sense of something no matter what.... Or maybe my brain is to limited to understand them..... I fully get the concept but was never able to get the examples running form Google site so I gave up.

Byte buffer it is then, trail and fail and more fail till I succeed, but that's the wrong way i know your approach is much better.

Take a look on the byte buffer for the Bluetooth hands free as well, I think you will be able to work out the naming there as well, I have added it as comment to issue number 2, sorry typing from phone and lazy to insert links

lmagder commented 7 years ago

Yeah I could see it adds another layer of obfuscation, but basically what it's doing is you write a text file which describes the structs then protoc generates you source code to convert the structs to and from the bytestream (with Parse and Serialize methods on C++ version)

Since the bytestreams you are writing are originally protobuf messages they have data in them for protoc to understand the format, but just no names of the fields. So you can use the decode_raw mode to print out the structure of a dump. So what I was doing was looking at that then adding the field names myself based on the existing code and/or guessing. You can test it by decoding the dump on the command line with protoc and the proto file.

So like for example for the CarInfo struct, this writes the same data as the sd_buf:


//  extern int wifi_direct;// = 0;//1;//0;
  int aa_pro_ctr_a05 (int chan, byte * buf, int len) {                  // Service Discovery Request
    if (len < 4 || buf [2] != 0x0a)
      loge ("Service Discovery Request: %x", buf [2]);
    else
      logd ("Service Discovery Request");                               // S 0 CTR b src: HU  lft:   113  msg_type:     6 Service Discovery Response    S 0 CTR b 00000000 0a 08 08 01 12 04 0a 02 08 0b 0a 13 08 02 1a 0f

    HU::CarInfo carInfo;
    carInfo.set_head_unit_name("Mazda Connect");
    carInfo.set_car_model("Mazda");
    carInfo.set_car_year("2016");
    carInfo.set_car_serial("0001");
    carInfo.set_driver_pos(true);
    carInfo.set_headunit_make("Mazda");
    carInfo.set_headunit_model("Connect");
    carInfo.set_sw_build("SWB1");
    carInfo.set_sw_version("SWV1");
    carInfo.set_can_play_native_media_during_vr(false);
    carInfo.set_hide_clock(false);

    carInfo.mutable_channels()->Reserve(AA_CH_MAX);

    HU::ChannelDescriptor* sensorChannel = carInfo.add_channels();
    sensorChannel->set_channel_id(AA_CH_SEN);
    {
      auto inner = sensorChannel->mutable_sensor_channel();
      inner->add_sensor_list()->set_type(HU::SENSOR_TYPE_DRIVING_STATUS);
      inner->add_sensor_list()->set_type(HU::SENSOR_TYPE_NIGHT_DATA);
    }

    HU::ChannelDescriptor* videoChannel = carInfo.add_channels();
    videoChannel->set_channel_id(AA_CH_VID);
    {
      auto inner = videoChannel->mutable_output_stream_channel();
      inner->set_type(HU::STREAM_TYPE_VIDEO);
      auto videoConfig = inner->add_video_configs();
      videoConfig->set_resolution(HU::ChannelDescriptor::OutputStreamChannel::VideoConfig::VIDEO_RESOLUTION_800x480);
      videoConfig->set_frame_rate(HU::ChannelDescriptor::OutputStreamChannel::VideoConfig::VIDEO_FPS_30);
      videoConfig->set_margin_width(0);
      videoConfig->set_margin_height(0);
      videoConfig->set_dpi(160);
      inner->set_available_while_in_call(true);
    }

    HU::ChannelDescriptor* inputChannel = carInfo.add_channels();
    inputChannel->set_channel_id(AA_CH_TOU);
    {
      auto inner = inputChannel->mutable_input_event_channel();
      auto tsConfig = inner->mutable_touch_screen_config();
      tsConfig->set_width(800);
      tsConfig->set_height(480);
    }

    HU::ChannelDescriptor* micChannel = carInfo.add_channels();
    micChannel->set_channel_id(AA_CH_MIC);
    {
      auto inner = micChannel->mutable_input_stream_channel();
      inner->set_type(HU::STREAM_TYPE_AUDIO);
      auto audioConfig = inner->mutable_audio_config();
      audioConfig->set_sample_rate(16000);
      audioConfig->set_bit_depth(16);
      audioConfig->set_channel_count(1);
    }

    HU::ChannelDescriptor* audioChannel0 = carInfo.add_channels();
    audioChannel0->set_channel_id(AA_CH_AUD);
    {
      auto inner = audioChannel0->mutable_output_stream_channel();
      inner->set_type(HU::STREAM_TYPE_AUDIO);
      inner->set_audio_type(HU::AUDIO_TYPE_MEDIA);
      auto audioConfig = inner->add_audio_configs();
      audioConfig->set_sample_rate(48000);
      audioConfig->set_bit_depth(16);
      audioConfig->set_channel_count(2);
    }

    HU::ChannelDescriptor* audioChannel1 = carInfo.add_channels();
    audioChannel1->set_channel_id(AA_CH_AU1);
    {
      auto inner = audioChannel1->mutable_output_stream_channel();
      inner->set_type(HU::STREAM_TYPE_AUDIO);
      inner->set_audio_type(HU::AUDIO_TYPE_SPEECH);
      auto audioConfig = inner->add_audio_configs();
      audioConfig->set_sample_rate(16000);
      audioConfig->set_bit_depth(16);
      audioConfig->set_channel_count(1);
    }

    std::ofstream old("old.bin", std::ostream::binary);
    old.write((const char*)sd_buf, sizeof (sd_buf));

    std::ofstream newbin("new.bin", std::ostream::binary);
    char header0 = 0;
    char header1 = 6;
    newbin.write(&header0, 1);
    newbin.write(&header1, 1);
    carInfo.SerializeToOstream(&newbin);

I used SerializeToOstream since I'm lazy ๐Ÿ˜„ but there is Serialize and Parse to raw buffers too.

For the GPS stuff, I'm not sure that's the right way to go it might cause problems with maps and other stuff. What about the SENSOR_TYPE_DRIVING_STATUS? It seems like it's a bool which sets this setting directly.

EDIT: Reposting under right account

lmagder commented 7 years ago

Argh, that's super annoying. I would assume it would use the phone's GPS if the HU doesn't report a GPS sensor. Still can't mess with the map when driving but at least it would match a real HU. The car has a GPS so maybe we need to figure out how to get that data from the CMU dbus.

Yeah I will take a look at the bluetooth stuff when I have some time. Some of the other stuff I would hope to add is hooking up more stuff to the real CMU parts. Like the Mazda OS has access to the hardware light sensor though a proc file, I want to hook that up to night/day mode instead of the time. It also seems like there is way to expose the XM radio as a AA music source based on the desktop headunit code. That would be awesome.

borconi commented 7 years ago

It doesn't mess up the GPS, I'm using it like that for the last 2 days, initially it set you to 0,0 (near Africa) but than the phone GPS picks up and it all works fine.

borconi commented 7 years ago

@lmagder it looks like you have way more knowledge than me, so maybe you have an idea on this. This is how the Sensor data should look (in Java):

public CarSensorEvent(int paramInt1, int paramInt2, long paramLong, float[] paramArrayOfFloat, byte[] paramArrayOfByte)
  {
    this.a = paramInt1;
    this.b = paramInt2;
    this.c = paramLong;
    this.d = paramArrayOfFloat;
    this.e = paramArrayOfByte;
  }

Now, I know that the following bytearray is working:

byte[] data = new byte[17];
        data[0]=(byte)8;
        data[1]=(byte)0x80;
        data[2]=(byte)0x03;
        data[3]=(byte)0x0a;
        data[4]=(byte)12;
        data[5]=(byte)0x00;
        hu_uti.varint_encode (86400000000000L, data,  5);
    data[13]=(byte)0x30;
    data[14]=(byte)0x01;
    data[15]=(byte)0x20;
    data[16]=(byte)0x01;

I also know that data[13] in the input byte array is translated to d[5] in Java, if instead of (byte)0x30 I use byte(0x08) that is translated to d[1] in Java, which looks to me that I'm only passing the: float[] paramArrayOfFloat to Java, any idea how a proto will pass the byte[] paramArrayOfByte, AA will use that byte array to calculate the Latitude and Longitude, with the following formula:

 if ((i & 0x1) != 0)
    {
      if (((CarSensorEvent)localObject3).a >= 2) {
        ((Location)localObject1).setLatitude(CarSensorEvent.a(((CarSensorEvent)localObject3).e, 1) * 1.0E-7D);
      }
    }
    else
    {
      if ((i & 0x2) != 0)
      {
        if (((CarSensorEvent)localObject3).a < 2) {
          break label845;
        }
        ((Location)localObject1).setLongitude(CarSensorEvent.a(((CarSensorEvent)localObject3).e, 5) * 1.0E-7D);
      }

Now to my unknowing head this says that the first 4 bytes of the byte[] paramArrayOfByte referenced above will be the latitude and the next 4 will be the longitude, what I cannot figure out how to wire that byte[] paramArrayOfByte into the byte array.

Any ideas?

izacus commented 7 years ago

Can you paste CarSensorEvent.a(byte[], int) (the method, not the field) so we see how they transform the number?

borconi commented 7 years ago

Here is the whole class:

package com.google.android.gms.car;

import android.os.Parcel;
import android.os.Parcelable.Creator;
import com.google.android.gms.common.internal.safeparcel.AbstractSafeParcelable;
import fge;
import imf;

public class CarSensorEvent
  extends AbstractSafeParcelable
{
  public static final Parcelable.Creator CREATOR = new fge();
  public final int a;
  public int b;
  public long c;
  public final float[] d;
  public final byte[] e;

  public CarSensorEvent(int paramInt1, int paramInt2, long paramLong, float[] paramArrayOfFloat, byte[] paramArrayOfByte)
  {
    this.a = paramInt1;
    this.b = paramInt2;
    this.c = paramLong;
    this.d = paramArrayOfFloat;
    this.e = paramArrayOfByte;
  }

  public CarSensorEvent(int paramInt1, long paramLong, int paramInt2, int paramInt3)
  {
    this.a = 3;
    this.b = paramInt1;
    this.c = paramLong;
    this.d = new float[paramInt2];
    this.e = new byte[paramInt3];
  }

  public static int a(byte[] paramArrayOfByte, int paramInt)
  {
    return paramArrayOfByte[paramInt] & 0xFF | paramArrayOfByte[(paramInt + 1)] << 8 & 0xFF00 | paramArrayOfByte[(paramInt + 2)] << 16 & 0xFF0000 | paramArrayOfByte[(paramInt + 3)] << 24 & 0xFF000000;
  }

  public static void a(byte[] paramArrayOfByte, int paramInt1, int paramInt2)
  {
    paramArrayOfByte[paramInt1] = ((byte)paramInt2);
    paramArrayOfByte[(paramInt1 + 1)] = ((byte)(paramInt2 >> 8));
    paramArrayOfByte[(paramInt1 + 2)] = ((byte)(paramInt2 >> 16));
    paramArrayOfByte[(paramInt1 + 3)] = ((byte)(paramInt2 >>> 24));
  }

  public final void a(int paramInt)
  {
    if (this.b == paramInt) {
      return;
    }
    throw new UnsupportedOperationException(String.format("Invalid sensor type: expected %d, got %d", new Object[] { Integer.valueOf(paramInt), Integer.valueOf(this.b) }));
  }

  public String toString()
  {
    StringBuilder localStringBuilder = new StringBuilder();
    localStringBuilder.append(String.valueOf(getClass().getName()).concat("["));
    Object localObject = String.valueOf(Integer.toHexString(this.b));
    if (((String)localObject).length() != 0) {}
    int j;
    int i;
    for (localObject = "type:".concat((String)localObject);; localObject = new String("type:"))
    {
      localStringBuilder.append((String)localObject);
      if ((this.d == null) || (this.d.length <= 0)) {
        break;
      }
      localStringBuilder.append(" float values:");
      localObject = this.d;
      j = localObject.length;
      i = 0;
      while (i < j)
      {
        float f = localObject[i];
        localStringBuilder.append(16 + " " + f);
        i += 1;
      }
    }
    if ((this.e != null) && (this.e.length > 0))
    {
      localStringBuilder.append(" byte values:");
      localObject = this.e;
      j = localObject.length;
      i = 0;
      while (i < j)
      {
        int k = localObject[i];
        localStringBuilder.append(5 + " " + k);
        i += 1;
      }
    }
    localStringBuilder.append("]");
    return localStringBuilder.toString();
  }

  public void writeToParcel(Parcel paramParcel, int paramInt)
  {
    paramInt = imf.a(paramParcel, 20293);
    imf.b(paramParcel, 1, this.b);
    imf.a(paramParcel, 2, this.c);
    imf.a(paramParcel, 3, this.d, false);
    imf.a(paramParcel, 4, this.e, false);
    imf.b(paramParcel, 1000, this.a);
    imf.b(paramParcel, paramInt);
  }
}
anod commented 7 years ago

@borconi

LocationData proto definiton, hope that will help you:

    message LocationData
    {
        optional uint64 timestamp = 1;
        optional int32 latitude = 2;
        optional int32 longitude = 3;
        optional uint32 accuracy = 4;
        optional int32 altitude = 5;
        optional int32 speed = 6;
        optional int32 bearing = 7;
    }
borconi commented 7 years ago

Thanks @anod the only problem is I wasn't yet able to construct the proper bytearray for this, I think I need to sit down and chew over those protobuf documentations till I finally able to master them.

anod commented 7 years ago

I think example will help you,

Example of nightmode message:

        Protocol.SensorBatch sensorBatch = new Protocol.SensorBatch();
        sensorBatch.nightMode = new Protocol.SensorBatch.NightMode[1];
        sensorBatch.nightMode[0] = new Protocol.SensorBatch.NightMode();
        sensorBatch.nightMode[0].isNight = true;

        // allocate reuqired size + 2 bytes for message type
        byte[] ba = new byte[sensorBatch.getSerializedSize() + 2];
        // Add message type header 'Sensor event'
        ba[0] = (byte) 0x80;
        ba[1] = 0x03;
        // serialize object into byte array
        MessageNano.toByteArray(sensorBatch, ba, 2, sensorBatch.getSerializedSize());

Result: 80 03 52 02 08 01 where 52 02 08 01 is the message. You can find more details about format: https://developers.google.com/protocol-buffers/docs/encoding

To recreate location data message:

        Protocol.SensorBatch sensorBatch = new Protocol.SensorBatch();
        sensorBatch.locationData = new Protocol.SensorBatch.LocationData[1];
        sensorBatch.locationData[0] = new Protocol.SensorBatch.LocationData();
        sensorBatch.locationData[0].timestamp = ...
        sensorBatch.locationData[0].latitude = ...
        sensorBatch.locationData[0].longitude = ...
        sensorBatch.locationData[0].accuracy = ...
        sensorBatch.locationData[0].speed = ...
        sensorBatch.locationData[0].bearing = ...

        byte[] ba = new byte[sensorBatch.getSerializedSize() + 2];
        // Add message type header 'Sensor event'
        ba[0] = (byte) 0x80;
        ba[1] = 0x03;
        MessageNano.toByteArray(sensorBatch, ba, 2, sensorBatch.getSerializedSize());
borconi commented 7 years ago

@anod - You're a legend, inserted a location successfully, based on your protobuf now I need to fully implement it into the code, rather than converting it on the command line. This is huge (and great), it means we can offload all the GPS to the headunit (tablet) and rely on the external GPS antenna of the headunit rather than the small built in GPS of the phone, beside if we offload the GPS that should reduce power needed by phone, meaning it can charger slightly better when plugged in.

izacus commented 7 years ago

Also, some additions about location data:

latitude, longtitude seem to be multiplied by 1e7 and then converted to int. Accuracy is multiplied by 1e3, altitude 1e2, speed 1e3 and bearing 1e6. This is probably because all those payloads are ints in protocol, but have decimals in actual data.

izacus commented 7 years ago

So, basically, if decimal latitude from GPS is 46.0552778, the actual number encoded into message should be 460552778. If current speed is 112.12 km/h, the number sent should be 112120, etc. etc.

borconi commented 7 years ago

Yes that's correct, I already got it working from Android.

P.s. set the bearing and speed to 0 ignoring the real value and you will have access to unlimited browsing all the time.

Kind regards. Emil Borconi-Szedressy Sent from my mobile

On 3 Dec 2016 3:16 p.m., "Jernej Virag" notifications@github.com wrote:

So, basically, if decimal latitude from GPS is 46.0552778, the actual number encoded into message should be 460552778. If current speed is 112.12 km/h, the number sent should be 112120, etc. etc.

โ€” You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/gartnera/headunit/issues/16#issuecomment-264644983, or mute the thread https://github.com/notifications/unsubscribe-auth/AAo8PMkXl-mCDneVih3y5IIRA-IIb-y-ks5rEYe5gaJpZM4J6Irb .

izacus commented 7 years ago

Also, NanoPB http://koti.kapsi.fi/~jpa/nanopb/ seems to be a simpler more lightweight option for a C PB lib :)

lmagder commented 7 years ago

Ah too late though :) already converted it to use the standard one as a test:

https://github.com/lmagder/headunit/tree/protobuf-refactor

Haven't tried on my car yet, and too big a change to be a serious pull request but it's there if anyone's interested. The ubuntu version works at least and the mazda one compiles.

izacus commented 7 years ago

Ahh, nice, the proto definitions make code significantly more bareable :D

lmagder commented 7 years ago

Yeah, hopefully it will be easy to integrate the other fixes you guys discovered while I was doing this. Just tried in my car and it runs! I was worried the C++ conversion would add runtime requirements, but it seems using static libc++ avoids this. Audio broke for some reason but not mic, everything else works. Time to start debugging... :) The audio issue was intermittent before on 0.94 and was usually fixed by rebooting the CMU, so maybe not related.

lmagder commented 7 years ago

Ok false alarm. The audio actually works. I just had the phone connected to the car through Bluetooth audio by accident so it was routing the audio through there. So I would say the code at https://github.com/lmagder/headunit/tree/protobuf-refactor has the same functionality as 0.94 currently.

borconi commented 7 years ago

Thanks to @anod here is a quite complex protobuf with some additional info included. Going to post here when more is decoded. Most of the Sensors are not operational yet, but as I have time I will add more here so on the day AA releases the version which supports all this sensors we can quickly implement it.

syntax = "proto2";

package HU;

enum MessageTypeControl
{
    MSG_TYPE_MEDIADATA0 = 0x00;
    MSG_TYPE_CODECDATA1 = 0x01;
    MSG_TYPE_VERSIONRESPONSE = 0x02;
    MSG_TYPE_SSLHANDSHAKE = 0x03;
    MSG_TYPE_SERVICEDISCOVERYREQUEST = 0x05;
    MSG_TYPE_SERVICEDISCOVERYRESPONSE = 0x06;
    MSG_TYPE_CHANNELOPENREQUEST = 0x07;
    MSG_TYPE_CHANNELOPENRESPONSE = 0x08;
    MSG_TYPE_PINGREQUEST = 0x0B;
    MSG_TYPE_PINGRESPONSE = 0x0C;
    MSG_TYPE_NAVFOCUSREQUESTNOTIFICATION = 0x0D;
    MSG_TYPE_NAVFOCUSRNOTIFICATION = 0x0E;
    MSG_TYPE_BYEYEREQUEST = 0x0F;
    MSG_TYPE_SHUTDOWNRESPONSE = 0x10;
    MSG_TYPE_VOICESESSIONNOTIFICATION = 0x11;
    MSG_TYPE_AUDIOFOCUSREQUESTNOTFICATION = 0x12;
    MSG_TYPE_AUDIOFOCUSNOTFICATION = 0x13;
};                                                                                                                           // If video data, put on queue

enum MessageTypeMedia
{
    MSG_TYPE_MEDIASETUPREQUEST = 0x8000;
    MSG_TYPE_MEDIASTARTREQUEST = 0x8001;
    MSG_TYPE_MEDIASTOPREQUEST = 0x8002;
    MSG_TYPE_MEDIASETUPRESPONSE = 0x8003;
    MSG_TYPE_ACK = 0x8004;
    MSG_TYPE_MICREQUEST = 0x8005;
    MSG_TYPE_MICREPONSE = 0x8006;
    MSG_TYPE_VIDEOFOCUSREQUESTNOTIFICATION = 0x8007;
    MSG_TYPE_VIDEOFOCUSNOTIFICATION = 0x8008;
};

enum MessageTypeSensor
{
    MSG_TYPE_SENSORSTARTREQUEST = 0x8001;
    MSG_TYPE_SENSORSTARTRESPONSE = 0x8002;
    MSG_TYPE_SENSOREVENT = 0x8003;
};

enum MessageTypeInput
{
    MSG_TYPE_INPUTEVENT = 0x8001;
    MSG_TYPE_INPUTBINDINGREQUEST = 0x8002;
    MSG_TYPE_INPUTBINDINGRESPONSE = 0x8003;
};

enum MessageStatus
{
    STATUS_OK = 0;
}

message Key
{
    required uint32 keycode = 1;
    required bool down = 2;
    required uint32 metastate = 3;
    required bool longpress = 4;
}

message KeyEvent
{
    repeated Key keys = 1;
}

message TouchEvent
{
    enum PointerAction
    {
        RELEASE = 0;
        PRESS = 1;
        DRAG = 2;
        // 0x6
    }
    message Pointer
    {
        optional uint32 x = 1;
        optional uint32 y = 2;
        optional uint32 pointer_id = 3;
    }
    repeated Pointer pointer_data = 1;
    optional uint32 action_index = 2;
    optional PointerAction action = 3;
}

message InputReport
{
    optional uint64 timestamp = 1;
    optional int32 disp_channel_id = 2;
    optional TouchEvent touch_event = 3;
    optional KeyEvent key_event = 4;
//    optional AbsoluteEvent absolute_event = 5;
//    optional RelativeEvent relative_event = 6;
//    optional TouchEvent touchpad_event = 7;
}

message KeyBindingRequest
{
    repeated int32 keycodes = 1;
}

message BindingResponse
{
    required MessageStatus status = 1;
}

enum SensorType
{
    SENSOR_TYPE_DRIVING_STATUS = 11;
    SENSOR_TYPE_NIGHT_DATA = 10;
    SENSOR_TYPE_RPM = 3;
    SENSOR_TYPE_DIAGNOSTICS = 8;
    SENSOR_TYPE_GEAR = 7;
    SENSOR_TYPE_COMPASS = 1;
    SENSOR_TYPE_LOCATION = 9;
}

message SensorBatch
{
    message LocationData
    {
        optional uint64 timestamp = 1;
        optional int32 latitude = 2;
        optional int32 longitude = 3;
        optional uint32 accuracy = 4;
        optional int32 altitude = 5;
        optional int32 speed = 6;
        optional int32 bearing = 7;
    }
    message NightMode
    {
        required bool is_night = 1;
    }
    message RPM
    {
        required int32 rpm = 1;
    }
    message FuelLevel
    {
        required int32 fuellevel = 1;
        optional int32 range = 2;
        optional bool lowfuel = 3;
    } 
    message DrivingStatus
    {
        enum Status
        {
            DRIVING_STATUS_PARKED = 0;
            DRIVING_STATUS_MOOVING = 1;
        }
        required int32 status = 1;
    }
    message DeadReckoning
    {
        optional int32 steering_angel = 1;
        optional int32 wheel_speed = 2;
    }

    repeated LocationData location_data = 1;
    //repeated CompassData compass_data = 2;
    //repeated Speed = 3;
    repeated RPM rpm = 4;
    //repeated Odometer = 5;
    repeated FuelLevel fuel_data = 6;
    //repeated ParkingBreak = 7;
    //repeated GearData = 8;
    //repeated Diagnostics = 9;
    repeated NightMode night_mode = 10;
    //repeated Environment = 11;
    //repeated HVAC = 12;
    repeated DrivingStatus driving_status = 13;
    repeated DeadReckoning dead_reckoning = 14;
    //repeated Passenger = 15;
    //repeated Door = 16;
    //repeated Light = 17;
    //repeated Tire = 18;
    //repeated Accel = 19;
    //repeated Gyro = 20;
    //repeated GPS = 21;
}

enum AudioStreamType
{
    AUDIO_TYPE_SPEECH = 1;
    AUDIO_TYPE_SYSTEM = 2;
    AUDIO_TYPE_MEDIA = 3;
    AUDIO_TYPE_ALARM = 4;
}

enum MediaCodecType
{
    MEDIA_CODEC_AUDIO = 1;
    MEDIA_CODEC_VIDEO = 3;
}

message AudioConfiguration
{
    optional uint32 sample_rate = 1;
    required uint32 number_of_bits = 2;
    required uint32 number_of_channels = 3;
}

message Service
{
    optional uint32 id = 1;
    message SensorSourceService
    {
        message Sensor
        {
            required SensorType type = 1;
        }
        repeated Sensor sensors = 1;
    }
    optional SensorSourceService sensor_source_service = 2;

    message MediaSinkService
    {
        optional MediaCodecType available_type = 1;
        optional AudioStreamType audio_type = 2;
        repeated AudioConfiguration audio_configs = 3;

        message VideoConfiguration
        {
            enum VideoCodecResolutionType
            {
                VIDEO_RESOLUTION_800x480 = 1;
                VIDEO_RESOLUTION_1280x720 = 2;
                VIDEO_RESOLUTION_1920x1080 = 3;
            }

            enum VideoFrameRateType
            {
                VIDEO_FPS_30 = 1;
                VIDEO_FPS_60 = 2;
            }
            required VideoCodecResolutionType codec_resolution = 1;
            required VideoFrameRateType frame_rate = 2;
            required uint32 margin_width = 3;
            required uint32 margin_height = 4;
            required uint32 density = 5;
            optional uint32 decoder_additional_depth = 6;
        }
        repeated VideoConfiguration video_configs = 4;
        optional bool available_while_in_call = 5;
    }
    optional MediaSinkService media_sink_service = 3;

    message InputSourceService
    {
        message TouchConfig
        {
            required uint32 width = 1;
            required uint32 height = 2;
        }
        repeated uint32 keycodes_supported = 1;
        optional TouchConfig touchscreen = 2;
        optional TouchConfig touchpad = 3;
    }

    optional InputSourceService input_source_service = 4;

    message MediaSourceService
    {
        required MediaCodecType type = 1;
        required AudioConfiguration audio_config = 2;
        optional bool available_while_in_call = 3;
    }

    optional MediaSourceService media_source_service = 5;

    message BluetoothService {
        enum BluetoothPairingMethod
        {
            BLUETOOTH_PARING_METHOD_1 = 1;
            BLUETOOTH_PARING_METHOD_2 = 2;
            BLUETOOTH_PARING_METHOD_3 = 3;
            BLUETOOTH_PARING_METHOD_4 = 4;
        }
        required string car_address = 1;
        repeated BluetoothPairingMethod supported_pairing_methods = 2;
    }
    optional BluetoothService bluetooth_service = 6;

    message NavigationStatusService {
        message ImageOptions
        {
            required int32 width = 1;
            required int32 height = 2;
            required int32 colour_deth_bits = 3;
        }

        required uint32 minimum_interval_ms = 1;
        required uint32 type = 2;
        optional ImageOptions image_options = 3;
    }
    optional NavigationStatusService navigation_status_service = 8;

    //radio_service = 7
    //media_playback_service == 9
    //phone_status_service = 10
    //media_browser_service=11
    //vendor_extension_service==12
    //generic_notification_service==13
}

message ServiceDiscoveryRequest
{
    optional string phone_name = 4;
}

message ServiceDiscoveryResponse
{
    repeated Service services = 1;
    optional string make = 2;
    optional string model = 3;
    optional string year = 4;
    optional string vehicle_id = 5;
    optional bool driver_position = 6;
    optional string head_unit_make = 7;
    optional string head_unit_model = 8;
    optional string head_unit_software_build = 9;
    optional string head_unit_software_version = 10;
    optional bool can_play_native_media_during_vr = 11;
    optional bool hide_projected_clock = 12;
}

message ChannelOpenRequest
{
    optional int32 priority = 1;
    optional int32 service_id = 2;
}

message ChannelOpenResponse
{
    required MessageStatus status = 1;
}

message PingRequest
{
    optional int64 timestamp = 1;
    optional int32 bug_report = 2;
}

message PingResponse
{
    optional int64 timestamp = 1;
}

message ByeByeRequest
{
    enum ByeByeReason
    {
        REASON_QUIT = 1;
    }
    optional ByeByeReason reason = 1;
}

message MediaSetupRequest
{
    optional uint32 type = 1; //Enum?
}

message Config
{
    enum ConfigStatus
    {
        CONFIG_STATUS_1 = 1;
        CONFIG_STATUS_2 = 2;
    }
    required ConfigStatus status = 1;
    required uint32 max_unacked = 2;
    repeated uint32 configuration_indices = 3;
}

message Start
{
    optional int32 session_id = 1;
    optional uint32 configuration_index = 2;
}

message Ack
{
    optional int32 session_id = 1;
    optional uint32 ack = 2;
}

message MicrophoneRequest
{
    required bool open = 1;
    optional bool anc_enabled = 2;
    optional bool ec_enabled = 3;
    required int32 max_unacked = 4;
}

message MicrophoneResponse
{
    optional int32 status = 1;
    optional uint32 session_id = 2;
}

enum VideoFocusMode
{
    VIDEO_FOCUS_MODE_1 = 1;
    VIDEO_FOCUS_MODE_2 = 2;
}

message VideoFocusRequestNotification
{
    enum VideoFocusReason
    {
        VIDEO_FOCUS_REASON_1 = 1;
        VIDEO_FOCUS_REASON_2 = 2;
    }

    optional int32 disp_channel_id = 1;
    optional VideoFocusMode mode = 2; //Enum?
    optional VideoFocusReason reason = 3; //Enum?
}

message VideoFocusNotification
{
    optional VideoFocusMode mode = 1;
    optional bool unsolicited = 2;
}

message SensorRequest
{
    optional SensorType type = 1;
    optional int64 min_update_period = 2;
}

message SensorResponse
{
    required MessageStatus status = 1;
}

enum NavFocusType
{
    NAV_FOCUS_1 = 1;
    NAV_FOCUS_2 = 2;
}

message NavFocusRequestNotification
{
    optional NavFocusType focus_type = 1;
}

message NavFocusNotification
{
    optional NavFocusType focus_type = 1;
}

message VoiceSessionNotification
{
    enum VoiceSessionStatus
    {
        VOICE_STATUS_START = 1;
        VOICE_STATUS_STOP = 2;
    }
    optional VoiceSessionStatus status = 1;
}

message AudioFocusRequestNotification
{
    enum AudioFocusRequestType
    {
        AUDIO_FOCUS_GAIN = 1;
        AUDIO_FOCUS_GAIN_TRANSIENT = 2;
        AUDIO_FOCUS_UNKNOWN = 3;
        AUDIO_FOCUS_RELEASE = 4;
    }
    optional AudioFocusRequestType request = 1;
}

message AudioFocusNotification
{
    enum AudioFocusStateType
    {
        AUDIO_FOCUS_STATE_GAIN = 1;
        AUDIO_FOCUS_STATE_GAIN_TRANSIENT = 2;
        AUDIO_FOCUS_STATE_LOSS = 3;
        AUDIO_FOCUS_STATE_LOSS_TRANSIENT_CAN_DUCK = 4;
        AUDIO_FOCUS_STATE_LOSS_TRANSIENT = 5;
        AUDIO_FOCUS_STATE_GAIN_MEDIA_ONLY = 6;
        AUDIO_FOCUS_STATE_GAIN_TRANSIENT_GUIDANCE_ONLY = 7;
    }
    optional AudioFocusStateType focus_state = 1;
    optional bool unsolicited = 2;
}
izacus commented 7 years ago

@lmagder can you do a quick test on your branch? Set all input (audio) and the video channel property "available_in_call" to "false" and try if phone calls still get kicked back to the phone?

lmagder commented 7 years ago

@izacus sure, I will try and report back. If you want to mess with it my branch should be self-contained to build too if you get the submodules also.

@borconi awesome! Thanks. Looks like we picked different names for some the same fields. Was bound to happen, but slight difficult to merge :)

borconi commented 7 years ago

It's not my merit, it's @anod who did 95% of the work, I just completed a few more after he gave me a hint on how to get them, so the credit belongs to him not to me.

Kind regards. Emil Borconi-Szedressy Sent from my mobile

On 4 Dec 2016 8:55 p.m., "Lucas Magder" notifications@github.com wrote:

@izacus https://github.com/izacus sure, I will try and report back. If you want to mess with it my branch should be self-contained to build too if you get the submodules also.

@borconi https://github.com/borconi awesome! Thanks. Looks like we picked different names for some the same fields. Was bound to happen, but slight difficult to merge :)

โ€” You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/gartnera/headunit/issues/16#issuecomment-264730413, or mute the thread https://github.com/notifications/unsubscribe-auth/AAo8PN_pCIFE1IKEc6b3kJfUWLZXXwgYks5rEyilgaJpZM4J6Irb .

izacus commented 7 years ago

@lmagder hmm, your build refuses to recognise my Pixel XL on USB, kinda just dies with "MISMATCH".

Got gdk_screen_get_monitor_scale_factor() == 2.000000
iusb_vendor_get vendor: 0x05ac  device: 0x226e4f0 
iusb_vendor_get vendor: 0x18d1  device: 0x226c070 
iusb_vendor_get vendor: 0x1d6b  device: 0x226bf90 
iusb_vendor_get vendor: 0x05ac  device: 0x226d610 
iusb_vendor_get vendor: 0x0a5c  device: 0x226d160 
iusb_vendor_get vendor: 0x05ac  device: 0x226c2d0 
iusb_vendor_get vendor: 0x1d6b  device: 0x226c1f0 
Device found iusb_best_vendor: 0x18d1  iusb_best_device: 0x226c070  iusb_best_man: ""  iusb_best_pro: "" 
20:04:09 2016 W: hu_usb:: MISMATCH Done endpoint search iusb_ep_in: 0x81  iusb_ep_out: 0x01  ep_in_addr: 0xfe  ep_out_addr: 0xfe
SHAI1 iusb_best_product id 20194 
20:04:09 2016 E: hu_usb:: Done dir: recv  len: 4  bytes_xfrd: 0 usb_err: -4 (LIBUSB_ERROR_NO_DEVICE)  errno: 0 (Success)
20:04:09 2016 E: hu_aap:: ihu_tra_recv() error so stop Transport & AAP  ret: -1
20:04:09 2016 E: hu_usb:: Done libusb_release_interface usb_err: -4 (LIBUSB_ERROR_NO_DEVICE)
20:04:09 2016 E: hu_aap:: Recv have_len: -1
Phone switched to accessory mode. Attempting once more.
iusb_vendor_get vendor: 0x05ac  device: 0x226d980 
iusb_vendor_get vendor: 0x18d1  device: 0x226d3b0 
iusb_vendor_get vendor: 0x1d6b  device: 0x226dc30 
iusb_vendor_get vendor: 0x05ac  device: 0x226d2f0 
iusb_vendor_get vendor: 0x0a5c  device: 0x226c930 
iusb_vendor_get vendor: 0x05ac  device: 0x226c870 
iusb_vendor_get vendor: 0x1d6b  device: 0x226c7b0 
Device found iusb_best_vendor: 0x18d1  iusb_best_device: 0x226d3b0  iusb_best_man: ""  iusb_best_pro: "" 
20:04:10 2016 W: hu_usb:: MISMATCH Done endpoint search iusb_ep_in: 0x81  iusb_ep_out: 0x02  ep_in_addr: 0xfe  ep_out_addr: 0xfe
SHAI1 iusb_best_product id 11521 
20:04:10 2016 E: hu_usb:: Done dir: recv  len: 4  bytes_xfrd: 0 usb_err: -1 (LIBUSB_ERROR_IO)  errno: 11 (Resource temporarily unavailable)
20:04:10 2016 E: hu_aap:: ihu_tra_recv() error so stop Transport & AAP  ret: -1
20:04:11 2016 E: hu_aap:: Recv have_len: -1
STATUS:Phone switched to accessory mode. Restart to enter AA mode.

What were the changes there? (Also code doesn't compile without NDEBUG at all, which makes debugging now extremely hard.)

(Whatever it is, it was broken in https://github.com/lmagder/headunit/commit/adef161840fa545e42417ba9d23fbcca4290271a?diff=unified )

lmagder commented 7 years ago

Argh, sorry, you might want to try this branch https://github.com/lmagder/headunit/tree/protobuf-refactor it's strictly 0.94 with the protobuf changes. My main branch has a bunch of refactoring I'm not done yet so might be kind broken. It works on my PC with a 6P, but on my laptop I get the same MISMATCH error as you so at least I can debug.

lmagder commented 7 years ago

Also you are right I broke NDEBUG, but I think you can turn on the logging ints in the hu_uti.cpp file and it will print. Sorry it's kind of rough.

izacus commented 7 years ago

Yeah, no problem, just wanted to rebase my sensors stuff so we don't duplicate work :) The issue seems to be in bulk transfer code, for some reason the first RECV fails.

I'll just grab the protobuf branch as the base.

izacus commented 7 years ago

There, managed to port a bit of things to protobufs.

Interesting thing though - GMaps stopped showing the input field and the rotary keyboard. For the life of me I can't figure out what changed (and for some reason protoc refuses to decode the sd_buf fields). Any tips on how to decode the old sd_buf contents with protoc (Only getting Failed to parse input) ?

borconi commented 7 years ago

The keyboard disappeared because you activated the rotary input. When you are in rotary mode you don't have on-screen keyboard they are exclusive to each other. In gmaps use the rotary to enter address, just turn it left or right. I'm not in front of my pc now but I think it was the first line of sd_buf (0x01) which defined the rotary. Hope it makes sense.

Kind regards. Emil Borconi-Szedressy Sent from my mobile

On 11 Dec 2016 10:59 p.m., "Jernej Virag" notifications@github.com wrote:

There, managed to port a bit of things to protobufs.

Interesting thing though - GMaps stopped showing the input field and the rotary keyboard. For the life of me I can't figure out what changed (and for some reason protoc refuses to decode the sd_buf fields). Any idea on where to look for reasons for disappeared keyboard?

โ€” You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/gartnera/headunit/issues/16#issuecomment-266315066, or mute the thread https://github.com/notifications/unsubscribe-auth/AAo8PC72p4H7FJKd8V-7lzNTVXsiMI20ks5rHIBZgaJpZM4J6Irb .

lmagder commented 7 years ago

Awesome! If you are parsing with protoc make sure you are removing the 2 byte message I'd. It's a big endian uint16 and not part of the proto message.

As for rotary input. I'm not 100% sure there is actually a different mode. That just might be a quirk of the Google desktop head unit code. It seems as long as you declare the key codes in the car info along with the touchscreen it seems to switch the indicator based on the last input event. I noticed the the last undocumented key code we know 0x1 seems to do nothing but switch into key state (versus switching into key state and also doing another action). Maybe there are more such events to toggle?

izacus commented 7 years ago

@borconi the branch that came from your sd_buf patch showed a rotary input keyboard, zoom buttons and search bar on Google maps. Now the search bar, keyboard and the zoom buttons are gone again :)

Hmm, no idea really. I've hooked a debugger to DHU code and it sends 0x01, 0x02, 0x04, 0x13 ... 0x17, 0x54 and 0x10000. Google Maps then shows the rotary keyboard input. If you set only for touchscreen, it sends only 0x54 in the keycode list. Google Maps then shows an on-screen keyboard.

For us on the protobuf branch, it doesn't show either :/ Might be also connected to driving status or some other configuration?

anod commented 7 years ago

@izacus

Happened for me several times: when something doesn't work for me with protobuf, solution was to change an optional value to be required

borconi commented 7 years ago

@izacus - I know sound silly but try to force stop Android Auto and Google Play service and connect again. I had all kind of weird issues when I was playing with the sd_buf... Sometimes if you change some values and re-connect your phone it just doesn't work for some reason. Also you can try using each line of sd_buf separately against the protobuf and it should decode correctly (before knowing about protobuf I tried to re-arrange and put them in more readable way, so just use a line of code from https://github.com/gartnera/headunit/issues/28 and try to decode that one).

izacus commented 7 years ago

Hmm, I think there's something with the driving detection - since I can't scroll through items even if the car is stopped. Maybe it's ignoring / using wrong driving status sensor.

borconi commented 7 years ago

You need to use all 3 (1,10,13) sensors, otherwise it won't work sd_buf should contain this:

// SENSOR CHANNEL
0x0A, 0x10, 0x08, 0x02, 0x12, 0x0C, 0x0A, 0x02, 0x08, 0x01, 0x0A, 0x02, 0x08, 0x0A, 0x0A, 0x02, 0x08, 0x0D,
izacus commented 7 years ago

Yeah, found it - it seems that in protobuf, DRIVING_STATUS sensor had ID 11, not 13. Hence it didn't listen to it. Changing SENSOR_TYPE_DRIVING_STATUS to 13 in hu.proto made things appear again. Good news is that it also makes the full (instead of rotary) screen keyboard appear in maps.

Of course, this makes me wonder if other sensors have correct types now :/ (What's sensor 1? ) ....

EDIT: Yup, found another wrong one - SENSOR_TYPE_LOCATION is 1. Enabling additional logging will dump CAR.SENSOR lines in logcat on the device.

....

Hmm, changing driving sensor to 13 means that now I get "sensor not registered" when trying to send a driving event. Hmhm./

borconi commented 7 years ago

I think the confusion is caused by the numbers.... For example, in the protobuf (sd_buf) we define sensor ID:11, however when you force log from Android Auto, the ADB logs will refer to it as sensor 10 - Day/night, so yes the ID of the sensor is 11, but Android Audio calls it 10 and so on, which makes your head spin after a while and you keep losing track of which is which.... ๐Ÿ˜ธ

borconi commented 7 years ago

Guys I find it almost embarrassing to ask but somehow my brain cannot compute this:

When does the above return 1? Ok morning fresh head, so to the best of my understanding, it needs both conditions to be true in order to return true, in other words rax has to be bigger than 102, but then what's the purpose of the first condition? Or is it just one of those awkward decompiler things which makes no sense to the human?

P.S. this is the the validator for the enum of GearData, but you cannot really have so many different values....

 if ((rax > 0x6) && (rax > 0x66)) {
                    rax = 0x0;
            }
            else {
                    rax = 0x1;
            }

Thanks.

borconi commented 7 years ago

Ok, I have put some more of the protobuf data together, although most of it has no use at the moment because AA doesn't support it YET, but I'm confident it will eventually.

syntax = "proto2";

package HU;

enum MessageTypeControl
{
    MSG_TYPE_MEDIADATA0 = 0x00;
    MSG_TYPE_CODECDATA1 = 0x01;
    MSG_TYPE_VERSIONRESPONSE = 0x02;
    MSG_TYPE_SSLHANDSHAKE = 0x03;
    MSG_TYPE_SERVICEDISCOVERYREQUEST = 0x05;
    MSG_TYPE_SERVICEDISCOVERYRESPONSE = 0x06;
    MSG_TYPE_CHANNELOPENREQUEST = 0x07;
    MSG_TYPE_CHANNELOPENRESPONSE = 0x08;
    MSG_TYPE_PINGREQUEST = 0x0B;
    MSG_TYPE_PINGRESPONSE = 0x0C;
    MSG_TYPE_NAVFOCUSREQUESTNOTIFICATION = 0x0D;
    MSG_TYPE_NAVFOCUSRNOTIFICATION = 0x0E;
    MSG_TYPE_BYEYEREQUEST = 0x0F;
    MSG_TYPE_SHUTDOWNRESPONSE = 0x10;
    MSG_TYPE_VOICESESSIONNOTIFICATION = 0x11;
    MSG_TYPE_AUDIOFOCUSREQUESTNOTFICATION = 0x12;
    MSG_TYPE_AUDIOFOCUSNOTFICATION = 0x13;
};                                                                                                                           // If video data, put on queue

enum MessageTypeMedia
{
    MSG_TYPE_MEDIASETUPREQUEST = 0x8000;
    MSG_TYPE_MEDIASTARTREQUEST = 0x8001;
    MSG_TYPE_MEDIASTOPREQUEST = 0x8002;
    MSG_TYPE_MEDIASETUPRESPONSE = 0x8003;
    MSG_TYPE_ACK = 0x8004;
    MSG_TYPE_MICREQUEST = 0x8005;
    MSG_TYPE_MICREPONSE = 0x8006;
    MSG_TYPE_VIDEOFOCUSREQUESTNOTIFICATION = 0x8007;
    MSG_TYPE_VIDEOFOCUSNOTIFICATION = 0x8008;
};

enum MessageTypeSensor
{
    MSG_TYPE_SENSORSTARTREQUEST = 0x8001;
    MSG_TYPE_SENSORSTARTRESPONSE = 0x8002;
    MSG_TYPE_SENSOREVENT = 0x8003;
};

enum MessageTypeInput
{
    MSG_TYPE_INPUTEVENT = 0x8001;
    MSG_TYPE_INPUTBINDINGREQUEST = 0x8002;
    MSG_TYPE_INPUTBINDINGRESPONSE = 0x8003;
};

enum MessageStatus
{
    STATUS_OK = 0;
}

message Key
{
    required uint32 keycode = 1;
    required bool down = 2;
    required uint32 metastate = 3;
    required bool longpress = 4;
}

message KeyEvent
{
    repeated Key keys = 1;
}

message TouchEvent
{
    enum PointerAction
    {
        RELEASE = 0;
        PRESS = 1;
        DRAG = 2;
        // 0x6
    }
    message Pointer
    {
        optional uint32 x = 1;
        optional uint32 y = 2;
        optional uint32 pointer_id = 3;
    }
    repeated Pointer pointer_data = 1;
    optional uint32 action_index = 2;
    optional PointerAction action = 3;
}

message InputReport
{
    optional uint64 timestamp = 1;
    optional int32 disp_channel_id = 2;
    optional TouchEvent touch_event = 3;
    optional KeyEvent key_event = 4;
//    optional AbsoluteEvent absolute_event = 5;
//    optional RelativeEvent relative_event = 6;
//    optional TouchEvent touchpad_event = 7;
}

message KeyBindingRequest
{
    repeated int32 keycodes = 1;
}

message BindingResponse
{
    required MessageStatus status = 1;
}

enum SensorType
{
    SENSOR_TYPE_DRIVING_STATUS = 11;
    SENSOR_TYPE_NIGHT_DATA = 10;
    SENSOR_TYPE_RPM = 3;
    SENSOR_TYPE_DIAGNOSTICS = 8;
    SENSOR_TYPE_GEAR = 7;
    SENSOR_TYPE_COMPASS = 1;
    SENSOR_TYPE_LOCATION = 9;
}

message SensorBatch
{
    message LocationData
    {
        optional uint64 timestamp = 1;
        optional int32 latitude = 2;
        optional int32 longitude = 3;
        optional uint32 accuracy = 4;
        optional int32 altitude = 5;
        optional int32 speed = 6;
        optional int32 bearing = 7;
    }
    message NightMode
    {
        required bool is_night = 1;
    }
    message RPM
    {
        required int32 rpm = 1;
    }
    message FuelLevel
    {
        required int32 fuellevel = 1;
        optional int32 range = 2;
        optional bool lowfuel = 3;
    } 
    message DrivingStatus
    {
        enum Status
        {
    DRIVE_STATUS_FULLY_RESTRICTED = 31; // 0x1f
    DRIVE_STATUS_LIMIT_MESSAGE_LEN = 16; // 0x10
    DRIVE_STATUS_NO_CONFIG = 8; // 0x8
    DRIVE_STATUS_NO_KEYBOARD_INPUT = 2; // 0x2
    DRIVE_STATUS_NO_VIDEO = 1; // 0x1
    DRIVE_STATUS_NO_VOICE_INPUT = 4; // 0x4
    DRIVE_STATUS_UNRESTRICTED = 0; // 0x0
        }
        required int32 status = 1;
    }
    message DeadReckoning
    {
        optional int32 steering_angel = 1;
        optional int32 wheel_speed = 2;
    }
    message CompassData {
        optional int32 bearing_e6 =1;
        optional int32 pitch_e6 =2;
        optional int32 roll_e6 =3;
        }
    message SpeedData {
        optional int32 speed_e6 =1;
        optional bool cruise_engaged=2;
        optional bool cruise_set_speed=3;
    }
    message OdometerData {
        optional int32 kms_el = 1;
        optional int32 trip_kms_el = 2;
    }
    message ParkingBreak {
        optional bool parking_breake =1;
    }
    message Passenger {
        optional bool passenger_present =1;
    }
    message Diagnostics {
        optional bytes diagnostics_byte = 1;
        }
    message Environment {
        optional int32 temperature_e3 =1;
        optional int32 pressure_e3 = 2;
        optional int32 rain = 3;
    }
    message HVAC {
        optional int32 target_temperature_e3 = 1;
        optional int32 current_temperature_e3 = 2;
    }
    message Accel {
        optional int32 acceleration_x_e3 = 1;
        optional int32 acceleration_y_e3 = 2;
        optional int32 acceleration_z_e3 = 3;
        }
    message Gyro {
        optional int32 rotation_speed_x_e3 = 1;
        optional int32 rotation_speed_y_e3 = 2;
        optional int32 rotation_speed_z_e3 = 3;
        }   
    message Door {
        optional bool hood_open=1;
        optional bool boot_open=2;
        repeated bool door_open=3;
    }
    message Light {
        enum headlight_state {
            headlight_state_0 = 0;
            headlight_state_1 = 1;
            headlight_state_2 = 2;
            headlight_state_3 = 3;
        }
        enum turn_indicator_state {
            trun_indicator_state_0 = 0;
            trun_indicator_state_1 = 1;
            trun_indicator_state_2 = 2;
            trun_indicator_state_3 = 3;
        }
        optional headlight_state headlight=1;
        optional turn_indicator_state turn_indicator=2;
        optional bool hazard_light_on=3;

    }
    message GearData {
      enum selected_gear {
    GEAR_DRIVE = 100; // 0x64
    GEAR_EIGHTH = 8; // 0x8
    GEAR_FIFTH = 5; // 0x5
    GEAR_FIRST = 1; // 0x1
    GEAR_FOURTH = 4; // 0x4
    GEAR_NEUTRAL = 0; // 0x0
    GEAR_NINTH = 9; // 0x9
    GEAR_PARK = 101; // 0x65
    GEAR_REVERSE = 102; // 0x66
    GEAR_SECOND = 2; // 0x2
    GEAR_SEVENTH = 7; // 0x7
    GEAR_SIXTH = 6; // 0x6
    GEAR_TENTH = 10; // 0xa
   GEAR_THIRD = 3; // 0x3
    }
   required selected_gear gear =1
   }

    repeated LocationData location_data = 1; // Working
    repeated CompassData compass_data = 2; // Sensor not yet supported
    repeated SpeedData speed_data = 3; // Working - Aka data is received, but has no effect
    repeated RPM rpm = 4; //Sensor not yet supported
    repeated OdometerData odometer_data = 5; //Sensor not yet supported
    repeated FuelLevel fuel_data = 6; //Sensor not yet supported
    repeated ParkingBreak parkingbrake_data = 7; // Working
    repeated GearData gear_data = 8; //Working but need to do the enum definition, and has no effect
    repeated Diagnostics diagnostics_data = 9; //Sensor not yet supported
    repeated NightMode night_mode = 10; // Working
    repeated Environment enviorment_data = 11; //Sensor not yet supported
    repeated HVAC hvac_data = 12; //Sensor not yet supported
    repeated DrivingStatus driving_status = 13; //Working
    repeated DeadReckoning dead_reckoning = 14; //Sensor not yet supported
    repeated Passenger passenger_data = 15; //Sensor not yet supported
    repeated Door door_data= 16; //Sensor not yet supported
    repeated Light light_data= 17;  //Sensor not yet supported
    //repeated Tire = 18;
    repeated Accel accel_data = 19; //Sensor not yet supported
    repeated Gyro gyro_data = 20; //Sensor not yet supported
    //repeated GPS = 21;
}

enum AudioStreamType
{
    AUDIO_TYPE_SPEECH = 1;
    AUDIO_TYPE_SYSTEM = 2;
    AUDIO_TYPE_MEDIA = 3;
    AUDIO_TYPE_ALARM = 4;
}

enum MediaCodecType
{
    MEDIA_CODEC_AUDIO = 1;
    MEDIA_CODEC_VIDEO = 3;
}

message AudioConfiguration
{
    optional uint32 sample_rate = 1;
    required uint32 number_of_bits = 2;
    required uint32 number_of_channels = 3;
}

message Service
{
    optional uint32 id = 1;
    message SensorSourceService
    {
        message Sensor
        {
            required SensorType type = 1;
        }
        repeated Sensor sensors = 1;
    }
    optional SensorSourceService sensor_source_service = 2;

    message MediaSinkService
    {
        optional MediaCodecType available_type = 1;
        optional AudioStreamType audio_type = 2;
        repeated AudioConfiguration audio_configs = 3;

        message VideoConfiguration
        {
            enum VideoCodecResolutionType
            {
                VIDEO_RESOLUTION_800x480 = 1;
                VIDEO_RESOLUTION_1280x720 = 2;
                VIDEO_RESOLUTION_1920x1080 = 3;
            }

            enum VideoFrameRateType
            {
                VIDEO_FPS_30 = 1;
                VIDEO_FPS_60 = 2;
            }
            required VideoCodecResolutionType codec_resolution = 1;
            required VideoFrameRateType frame_rate = 2;
            required uint32 margin_width = 3;
            required uint32 margin_height = 4;
            required uint32 density = 5;
            optional uint32 decoder_additional_depth = 6;
        }
        repeated VideoConfiguration video_configs = 4;
        optional bool available_while_in_call = 5;
    }
    optional MediaSinkService media_sink_service = 3;

    message InputSourceService
    {
        message TouchConfig
        {
            required uint32 width = 1;
            required uint32 height = 2;
        }
        repeated uint32 keycodes_supported = 1;
        optional TouchConfig touchscreen = 2;
        optional TouchConfig touchpad = 3;
    }

    optional InputSourceService input_source_service = 4;

    message MediaSourceService
    {
        required MediaCodecType type = 1;
        required AudioConfiguration audio_config = 2;
        optional bool available_while_in_call = 3;
    }

    optional MediaSourceService media_source_service = 5;

    message BluetoothService {
        enum BluetoothPairingMethod
        {
            BLUETOOTH_PARING_METHOD_1 = 1;
            BLUETOOTH_PARING_METHOD_2 = 2;
            BLUETOOTH_PARING_METHOD_3 = 3;
            BLUETOOTH_PARING_METHOD_4 = 4;
        }
        required string car_address = 1;
        repeated BluetoothPairingMethod supported_pairing_methods = 2;
    }
    optional BluetoothService bluetooth_service = 6;

    message NavigationStatusService {
        message ImageOptions
        {
            required int32 width = 1;
            required int32 height = 2;
            required int32 colour_deth_bits = 3;
        }

        required uint32 minimum_interval_ms = 1;
        required uint32 type = 2;
        optional ImageOptions image_options = 3;
    }
    optional NavigationStatusService navigation_status_service = 8;

    //radio_service = 7
    //media_playback_service == 9
    //phone_status_service = 10
    //media_browser_service=11
    //vendor_extension_service==12
    // generic_notification_service = 13;

}

message ServiceDiscoveryRequest
{
    optional string phone_name = 4;
}

message ServiceDiscoveryResponse
{
    repeated Service services = 1;
    optional string make = 2;
    optional string model = 3;
    optional string year = 4;
    optional string vehicle_id = 5;
    optional bool driver_position = 6;
    optional string head_unit_make = 7;
    optional string head_unit_model = 8;
    optional string head_unit_software_build = 9;
    optional string head_unit_software_version = 10;
    optional bool can_play_native_media_during_vr = 11;
    optional bool hide_projected_clock = 12;
}

message ChannelOpenRequest
{
    optional int32 priority = 1;
    optional int32 service_id = 2;
}

message ChannelOpenResponse
{
    required MessageStatus status = 1;
}

message PingRequest
{
    optional int64 timestamp = 1;
    optional int32 bug_report = 2;
}

message PingResponse
{
    optional int64 timestamp = 1;
}

message ByeByeRequest
{
    enum ByeByeReason
    {
        REASON_QUIT = 1;
    }
    optional ByeByeReason reason = 1;
}

message MediaSetupRequest
{
    optional uint32 type = 1; //Enum?
}

message Config
{
    enum ConfigStatus
    {
        CONFIG_STATUS_1 = 1;
        CONFIG_STATUS_2 = 2;
    }
    required ConfigStatus status = 1;
    required uint32 max_unacked = 2;
    repeated uint32 configuration_indices = 3;
}

message Start
{
    optional int32 session_id = 1;
    optional uint32 configuration_index = 2;
}

message Ack
{
    optional int32 session_id = 1;
    optional uint32 ack = 2;
}

message MicrophoneRequest
{
    required bool open = 1;
    optional bool anc_enabled = 2;
    optional bool ec_enabled = 3;
    required int32 max_unacked = 4;
}

message MicrophoneResponse
{
    optional int32 status = 1;
    optional uint32 session_id = 2;
}

enum VideoFocusMode
{
    VIDEO_FOCUS_MODE_1 = 1;
    VIDEO_FOCUS_MODE_2 = 2;
}

message VideoFocusRequestNotification
{
    enum VideoFocusReason
    {
        VIDEO_FOCUS_REASON_1 = 1;
        VIDEO_FOCUS_REASON_2 = 2;
    }

    optional int32 disp_channel_id = 1;
    optional VideoFocusMode mode = 2; //Enum?
    optional VideoFocusReason reason = 3; //Enum?
}

message VideoFocusNotification
{
    optional VideoFocusMode mode = 1;
    optional bool unsolicited = 2;
}

message SensorRequest
{
    optional SensorType type = 1;
    optional int64 min_update_period = 2;
}

message SensorResponse
{
    required MessageStatus status = 1;
}

enum NavFocusType
{
    NAV_FOCUS_1 = 1;
    NAV_FOCUS_2 = 2;
}

message NavFocusRequestNotification
{
    optional NavFocusType focus_type = 1;
}

message NavFocusNotification
{
    optional NavFocusType focus_type = 1;
}

message VoiceSessionNotification
{
    enum VoiceSessionStatus
    {
        VOICE_STATUS_START = 1;
        VOICE_STATUS_STOP = 2;
    }
    optional VoiceSessionStatus status = 1;
}

message AudioFocusRequestNotification
{
    enum AudioFocusRequestType
    {
        AUDIO_FOCUS_GAIN = 1;
        AUDIO_FOCUS_GAIN_TRANSIENT = 2;
        AUDIO_FOCUS_UNKNOWN = 3;
        AUDIO_FOCUS_RELEASE = 4;
    }
    optional AudioFocusRequestType request = 1;
}

message AudioFocusNotification
{
    enum AudioFocusStateType
    {
        AUDIO_FOCUS_STATE_GAIN = 1;
        AUDIO_FOCUS_STATE_GAIN_TRANSIENT = 2;
        AUDIO_FOCUS_STATE_LOSS = 3;
        AUDIO_FOCUS_STATE_LOSS_TRANSIENT_CAN_DUCK = 4;
        AUDIO_FOCUS_STATE_LOSS_TRANSIENT = 5;
        AUDIO_FOCUS_STATE_GAIN_MEDIA_ONLY = 6;
        AUDIO_FOCUS_STATE_GAIN_TRANSIENT_GUIDANCE_ONLY = 7;
    }
    optional AudioFocusStateType focus_state = 1;
    optional bool unsolicited = 2;
}
lmagder commented 7 years ago

Great, this is useful. Unfortunately some of the stuff is renamed, probably the correct name according to the AA designers, but different than our current code, which makes it not a super clean merge. I guess we should eventually convert over to share the proto definitions more easily.

I dunno about the Gear enum. That does seem crazy. Maybe there is a non-contiguous range of valid values? Are you sure your disassembly is correct?

borconi commented 7 years ago

Yeah pretty positive it's correct. Thanks to @anod again. For sharing now I have the answer to that as well, just need to work it all out.

Knock you self out with this: https://android.googlesource.com/platform/packages/services/Car/+/nougat-release/

It will help a lot with understanding the whole AA protocol. I did saw all the possible values for gear data in one of the files I just need to add them to the protp

Kind regards. Emil Borconi-Szedressy Sent from my mobile

On 19 Dec 2016 2:16 a.m., "Lucas Magder" notifications@github.com wrote:

Great, this is useful. Unfortunately some of the stuff is renamed, probably the correct name according to the AA designers, but different than our current code, which makes it not a super clean merge. I guess we should eventually convert over to share the proto definitions more easily.

I dunno about the Gear enum. That does seem crazy. Maybe there is a non-contiguous range of valid values? Are you sure your disassembly is correct?

โ€” You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/gartnera/headunit/issues/16#issuecomment-267867260, or mute the thread https://github.com/notifications/unsubscribe-auth/AAo8PKut2IPH74tDhFPeXUGFwX0Fo6J3ks5rJejvgaJpZM4J6Irb .

borconi commented 7 years ago

Ok, update the gear enum as well with correct/possible values.... this project is getting really exciting... I'm very very intrigued now to figure out how can AA communicate with the car cluster, looks like nougat will be able to do that... assuming we have a class which can implement that on the car side....

Update: @anod @izacus @lmagder - I have just update/corrected the driving status enum, please see above post. If you have different names just take the values... ๐Ÿ˜„

borconi commented 7 years ago

Alos one very interesting - @anod you might like this - (and possibly important thing I have discovered and it is fully undocumented at the moment) is an additional service. So where we have this as of definition:

    //radio_service = 7
    //media_playback_service == 9
    //phone_status_service = 10
    //media_browser_service=11
    //vendor_extension_service==12
    // generic_notification_service = 13;

We should have:

//radio_service = 7
    //media_playback_service == 9
    //phone_status_service = 10
    //media_browser_service=11
    //vendor_extension_service==12
    // generic_notification_service = 13;
   // Wifi_projection_service = 14
izacus commented 7 years ago

Ohh, the driving status enum now makes a lot more sense - and kinda explains what's going on with the keyboard.

anod commented 7 years ago

@borconi Nice, it's logical that AA will get a wireless projection in the future

The driving status names don't make any sense for me, maybe only unrestricted :)

But, what I can't understand is the values for sensor type, in the sources they are different that we use (SENSOR_TYPE_DRIVING_STATUS = 13; SENSOR_TYPE_NIGHT_DATA = 10)

and in the source code their values are 11 and 9 (also in auto apk 1.0 they were referred like that) https://android.googlesource.com/platform/packages/services/Car/+/nougat-release/car-lib/src/android/car/hardware/CarSensorManager.java

According to this driving status is actually driving restrictions

borconi commented 7 years ago

@anod - Yeah the naming and coding it a total mind f!, have a look here: https://android.googlesource.com/platform/packages/services/Car/+/nougat-release/car-support-lib/api/current.txt

It is VERY confusing trying to puzzle it together I lost track so many times already and every time I have to sit down and start chewing from start what is what....

Basically when we define the connection, the driving status is the 13rd defined element, hence we use to think it is sensor 13, BUT actually in Android Auto that is sensor number 11 only.... so while we define it as 13 in the protobuf when we send data, we are sending data like 80036a020801 (message 8003, defined item 13, status 1 - this will be mapped in android auto to Sensor 11 satus 1) hope it makes sense, I know it is VERY VERY confusing....

anod commented 7 years ago

@borconi

Yes I understand that, cannot find the place where conversion is happening. should be where ServiceDiscoveryResponse is being processed