search

silklabs / silk

Build IoT using JavaScript. Silk Open Source
MIT License
164 stars 29 forks source link

initial commit to enable Victara support #4

Open angelpipe opened 8 years ago

angelpipe commented 8 years ago

Hey, I'm intending to enable Victara(Moto X 2nd generation) support. I have given some steps but i want to make sure I'm pointing in the right direction, and I still have some doubts. Please forgive me if i do very basic questions (I didn't know anything about AOSP building a week ago).

The approach

I want to use the most i can from AOSP because porting is easier as your current patches are written for it and mako board is a good starting point. Taking that in account, I'm using the same android-5.1.1_r1.xml file in my custom victara.xml. After that, I'm adding Cyanogen Mod's Victara repo into /device/motorola/victara; and finally, I add motorola vendor files into /vendor/motorola. Am I crazy or this should be enough to get an almost pure AOSP build for the device?. I hope I'm on the right path, you can check the details of this in the victara.xml file.

Patches

As i used the same AOSP version you're using for mako, i guess the only patches that i need to review are the device ones (is that assumption ok?). I found 5 patches that are candidates to be ported to victara's build. I have rewritten some and i have doubts in others, so i will give a status for each one of them.

Thank you so much for reading this long text!

Best Regards.

mvines commented 8 years ago

This looks really nice! victara.xml seems good to me except that I don't see where the kernel is coming from? I see a reference to kernel/motorola/msm8974 here, but I don't see how that gets added to victara.xml.

...Am I crazy or this should be enough to get an almost pure AOSP build for the device

That may work. I'd suggest that you also confirm that the resulting AOSP build actually builds, boots, and works (IMO check out Camera, Wifi, BT, Sensors) before removing all the Android things and patching it into Silk to avoid wasting time debugging things on Silk that never worked on the original AOSP build to begin with.

About the patches that you had questions about:

0001-Don-t-enable-MTP-by-default.patch

This one probably doesn't matter. This was added to Mako because I found it annoying that my Linux machine always tried to connect with Mako over MTP every time it booted into Silk, even though we don't support MTP currently.

0001-Enable-firmware-to-mount-with-SELinux-disabled.patch

It looks like this isn't needed for Victara as it's fstab.qcom has no SELinux context mount argument. The symptom of not having this patch, if it's needed, would be that things like Wifi/BT/Camera won't work because the firmware for those subsystems can't be loaded from /firmware.

0001-Keep-2-cores-online-at-1.2ghz-to-stay-cool.patch

Yeah leave this one out too for now. Mako has really poor thermal mitigation and will happily melt itself into a puddle if you work it too hard for too long, so we added this patch to keep it cool. Kenzo is much better though, we're running that device with all 6 cores online and don't need this on that device either.

angelpipe commented 8 years ago

Hey Michael,

As always thanks for your help. I decided to change my approach because cyanogen resources were giving me a lot of trouble during build. I decided to take a working XML from an Aurora build for victara and cleaned it (as it looks you did with kenzo). It means i had to look closer at patches and i rewrote some of them in the device tree to make them work. I'm having a final issue: the patch _system/core/0001-Permit-executables-in-system-nodemodules.patch inside android-5.1.1_r1 does not work in my current codebase, but it is a general patch imported for all devices; so i figured out there are 2 possible solutions to this.

  1. Remove that patch from the android-5.1.1_r1 folder and add it on each device tree. so i can make a different one for victara.
  2. Don't use android-5.1.1_r1.xml in victara.xml and copy all its patches into victara's tree.

Do you have any idea?

Best regards

mvines commented 8 years ago

How about just placing a version of 0001-Permit-executables-in-system-node_modules.patch that does apply to Victara at bsp-gonk/board/victara/patch/system/core/0001-Permit-executables-in-system-node_modules.patch. Doing so will override override the patch in bsp-gonk/board/android-5.1.1_r1/patch/system/core/0001-Permit-executables-in-system-node_modules.patch that doesn't apply. Does that work?

angelpipe commented 8 years ago

Thanks!!. I didn't know that was possible. I will check and if all my tests go ok i think i will send a new PR and we should decline this one

andreasgal commented 8 years ago

@angelpipe you can just push an update to this pull request

angelpipe commented 8 years ago

Hi, I just sent my changes. There's a new victara.xml. I used kenzo patches, fixed some of them that were not working and wrote new patches for device. Unfortunately i'm having issues with my environment and I'm not able to build anything(even mako) so i don't know if these changes could contain errors :(, But everything is ok before running "make j*"

mvines commented 8 years ago

PR looks great. When you figure out your env issues (:scream:) and are able to actually test it out a little, please let me know here and we can get this merged!

angelpipe commented 8 years ago

Hey @mvines, i addressed the change you told me. About my environment, I'm on a Mac and i have got some problems to make it work. I will continue trying to build, but I'm running out of ideas. The error I'm getting right now is:

In file included from system/core/include/utils/String8.h:22, from system/core/include/utils/CallStack.h:22, from system/core/libutils/CallStack.cpp:19: system/core/include/utils/Unicode.h:37: error: expected ‘,’ or ‘...’ before ‘*’ token system/core/include/utils/Unicode.h:38: error: expected ‘,’ or ‘...’ before ‘*’ token system/core/include/utils/Unicode.h:39: error: expected ‘,’ or ‘...’ before ‘*’ token system/core/include/utils/Unicode.h:40: error: expected ‘,’ or ‘...’ before ‘*’ token system/core/include/utils/Unicode.h:41: error: expected constructor, destructor, or type conversion before ‘*’ token system/core/include/utils/Unicode.h:42: error: expected constructor, destructor, or type conversion before ‘*’ token system/core/include/utils/Unicode.h:51: error: expected ‘,’ or ‘...’ before ‘*’ token system/core/include/utils/Unicode.h:54: error: expected ‘,’ or ‘...’ before ‘*’ token system/core/include/utils/Unicode.h:57: error: expected ‘,’ or ‘...’ before ‘*’ token system/core/include/utils/Unicode.h:58: error: expected ‘,’ or ‘...’ before ‘*’ token system/core/include/utils/Unicode.h:64: error: expected ‘,’ or ‘...’ before ‘*’ token system/core/include/utils/Unicode.h:101: error: expected ‘,’ or ‘...’ before ‘*’ token system/core/include/utils/Unicode.h:116: error: expected ‘,’ or ‘...’ before ‘*’ token system/core/include/utils/Unicode.h:123: error: expected ‘,’ or ‘...’ before ‘*’ token system/core/include/utils/Unicode.h:153: error: ‘char32_t’ has not been declared system/core/include/utils/Unicode.h:165: error: expected constructor, destructor, or type conversion before ‘*’ token system/core/include/utils/Unicode.h:172: error: ‘char16_t’ has not been declared system/core/include/utils/Unicode.h:179: error: expected constructor, destructor, or type conversion before ‘*’ token In file included from system/core/include/utils/CallStack.h:22, from system/core/libutils/CallStack.cpp:19: system/core/include/utils/String8.h:55: error: expected ‘,’ or ‘...’ before ‘*’ token system/core/include/utils/String8.h:56: error: expected ‘,’ or ‘...’ before ‘*’ token system/core/include/utils/String8.h:56: error: ‘android::String8::String8(int)’ cannot be overloaded system/core/include/utils/String8.h:55: error: with ‘android::String8::String8(int)’ system/core/include/utils/String8.h:57: error: expected ‘,’ or ‘...’ before ‘*’ token system/core/include/utils/String8.h:57: error: ‘android::String8::String8(int)’ cannot be overloaded system/core/include/utils/String8.h:55: error: with ‘android::String8::String8(int)’ system/core/include/utils/String8.h:58: error: expected ‘,’ or ‘...’ before ‘*’ token system/core/include/utils/String8.h:58: error: ‘android::String8::String8(int)’ cannot be overloaded system/core/include/utils/String8.h:55: error: with ‘android::String8::String8(int)’ system/core/include/utils/String8.h:79: error: expected ‘,’ or ‘...’ before ‘*’ token system/core/include/utils/String8.h:80: error: expected ‘,’ or ‘...’ before ‘*’ token system/core/include/utils/String8.h:81: error: ‘android::status_t android::String8::setTo(int)’ cannot be overloaded system/core/include/utils/String8.h:79: error: with ‘android::status_t android::String8::setTo(int)’ system/core/include/utils/String8.h:96: error: ‘char32_t’ has not been declared make: *** [out/host/darwin-x86/obj32/STATIC_LIBRARIES/libutils_intermediates/CallStack.o] Error 1

I know this is not your concern, but maybe you have seen it before.

Have a nice day!

mvines commented 8 years ago

Hmm..do you have this patch applied? I've run into Unicode.h issues before on osx, although don't immediately recognize this particular error. https://github.com/silklabs/silk/blob/b65f76cee8bed4deea0cfb456f9e565fe4231a37/bsp-gonk/board/android-5.1.1_r1/patch/system/core/0001-OSX-Patches.patch

angelpipe commented 8 years ago

Yes, i have the patch applied. But this portion from Unicode.h looks suspicious

#include <sys/types.h>
#include <stdint.h>
#ifdef __APPLE__
#include <new>
#endif

extern "C" {

// Definitions exist in C++11
#if defined __cplusplus && __cplusplus < 201103L
#ifndef __APPLE__
typedef uint32_t char32_t;
typedef uint16_t char16_t;
#endif
#endif

Error shows up anytime the code tries to use _char_16t or _char_32t

angelpipe commented 8 years ago

Hi, After some digging, i've fixed some of the problems i faced with building but now im getting this one:

target thumb C++: silk-capture-daemon <= vendor/silk/capture/Capture.cpp
vendor/silk/bledroid/main.cpp:899:1: error: invalid conversion from 'void (*)(bt_bdaddr_t*, bt_bdname_t*, uint32_t) {aka void (*)(bt_bdaddr_t*, bt_bdname_t*, unsigned int)}' to 'pin_request_callback {aka void (*)(bt_bdaddr_t*, bt_bdname_t*, unsigned int, unsigned char)}' [-fpermissive]
 };
 ^
vendor/silk/bledroid/main.cpp:899:1: error: missing initializer for member 'bt_callbacks_t::le_lpp_write_rssi_thresh_cb' [-Werror=missing-field-initializers]
vendor/silk/bledroid/main.cpp:899:1: error: missing initializer for member 'bt_callbacks_t::le_lpp_read_rssi_thresh_cb' [-Werror=missing-field-initializers]
vendor/silk/bledroid/main.cpp:899:1: error: missing initializer for member 'bt_callbacks_t::le_lpp_enable_rssi_monitor_cb' [-Werror=missing-field-initializers]
vendor/silk/bledroid/main.cpp:899:1: error: missing initializer for member 'bt_callbacks_t::le_lpp_rssi_threshold_evt_cb' [-Werror=missing-field-initializers]
cc1plus: all warnings being treated as errors
make: *** [out/target/product/victara/obj/EXECUTABLES/silk-bledroid-daemon_intermediates/main.o] Error 1
make: *** Waiting for unfinished jobs....

#### make failed to build some targets (03:13 (mm:ss)) ####

Do you have any clue of what i'm doing wrong?

mvines commented 8 years ago

Yes, this is very familiar. The bluetooth HAL types tends to vary between different devices and SoCs, and we use build-time flags to manage this.

Victara seems to want QBLUETOOTH_L, which can be defined by the make flag TARGET_USES_QCOM_BSP.

Kenzo defines TARGET_USES_QCOM_BSP := true in it's device/qcom/msm8952_64/BoardConfig.mk (visible here), how about adding the same in bsp-gonk/device/moto/victara/BoardConfig.mk?

angelpipe commented 8 years ago

Hi @mvines,

At the end i was able to build with the latest patches i have sent. Unfortunately, the build is not working good. I'm almost sure i'm having problems with SELinux, but i don't know what I'm doing wrong. I say problems are related to SELinux because of this kind of errors:

--------- beginning of main
W/auditd  (  270): type=2000 audit(0.0:1): initialized
I/auditd  (  270): type=1403 audit(0.0:2): policy loaded auid=4294967295 ses=4294967295
I/chown   (  242): type=1400 audit(0.0:3): avc: denied { setattr } for uid=0 name="mmcblk0p20" dev="tmpfs" ino=10416 scontext=u:r:init:s0 tcontext=u:object_r:block_device:s0 tclass=blk_file permissive=1
I/ueventd (  258): type=1400 audit(0.0:4): avc: denied { search } for uid=0 name="/" dev="mmcblk0p1" ino=2 scontext=u:r:ueventd:s0 tcontext=u:object_r:unlabeled:s0 tclass=dir permissive=1
I/kmsg    (  387): <6>[   24.077935,0] pil-q6v5-mss fc880000.qcom,mss: mba: Brought out of reset
I/kmsg    (  387): <6>[   24.081561,0] pil-q6v5-mss fc880000.qcom,mss: modem: factory modem check
I/kmsg    (  387): <3>[   24.081747,0] pil-q6v5-mss fc880000.qcom,mss: modem: cert not found!
I/kmsg    (  387): <6>[   24.081905,0] pil-q6v5-mss fc880000.qcom,mss: modem: loading from 0x07700000 to 0x0d100000
I/ThermalEngine(  364): Sensor:chg_therm:32000 mC
I/kmsg    (  387): <3>[   24.468519,0] init: Warning!  Service silk needs a SELinux domain defined; please fix!
I/kmsg    (  387): <3>[   24.469098,0] init: Warning!  Service silk-bledroid needs a SELinux domain defined; please fix!
I/kmsg    (  387): <3>[   24.469559,0] init: Warning!  Service silk-sensors needs a SELinux domain defined; please fix!
E/QC-QMI  (  364): qmi_client [364]: unable to connect to server, errno=[2:No such file or directory], attempt=20

Maybe i translated wrong this patch https://github.com/angelpipe/silk/blob/pr/victara/bsp-gonk/board/victara/patch/device/moto/victara/0001-Add-silk-product.patch. But i tried without applying it too and I'm getting the same errors. Once again, do you ha a clue?

Thanks for your help!

mvines commented 8 years ago

Nice, try adding androidboot.selinux=permissive to the victara's BOARD_KERNEL_CMDLINE.

diff --git a/BoardConfig.mk b/BoardConfig.mk
index 84a0b8e..0ef3e1a 100644
--- a/BoardConfig.mk
+++ b/BoardConfig.mk
@@ -39,7 +39,7 @@ TARGET_CPU_VARIANT := krait
 TARGET_USE_QCOM_BIONIC_OPTIMIZATION := true

 # Kernel
-BOARD_KERNEL_CMDLINE := console=ttyHSL0,115200,n8 androidboot.hardware=qcom msm_rtb.filter=0x37 ehci-hcd.park=3 vmalloc=
+BOARD_KERNEL_CMDLINE := androidboot.selinux=permissive console=ttyHSL0,115200,n8 androidboot.hardware=qcom msm_rtb.filte
 BOARD_KERNEL_BASE := 0x80200000
 BOARD_KERNEL_PAGESIZE := 2048
 BOARD_MKBOOTIMG_ARGS := --ramdisk_offset 0x02000000 --tags_offset 0x01e00000
angelpipe commented 8 years ago

The file already had that parameter added :(

mvines commented 8 years ago

Hmm, I don't see too much else interesting from that log snippet unfortunately. What does running adb shell /system/silk/bin/silk-init do? If you put the full output of adb logcat right from boot in a gist, I'll scan it to see if I spot anything unusual.

angelpipe commented 8 years ago

adb shell /system/silk/bin/silk-init:

H6460:victara fangel$ adb shell /system/silk/bin/silk-init
module.js:442
    throw err;
    ^

Error: Cannot find module 'silk-log/device'
    at Function.Module._resolveFilename (module.js:440:15)
    at Function.Module._load (module.js:388:25)
    at Module.require (module.js:468:17)
    at require (internal/module.js:20:19)
    at Object.<anonymous> (/system/silk/node_modules/silk-sysutils/lib/index.js:423:19)
    at __webpack_require__ (/system/silk/node_modules/silk-sysutils/lib/index.js:21:30)
    at Object.<anonymous> (/system/silk/node_modules/silk-sysutils/lib/index.js:74:16)
    at __webpack_require__ (/system/silk/node_modules/silk-sysutils/lib/index.js:21:30)
    at module.exports.Object.defineProperty.value (/system/silk/node_modules/silk-sysutils/lib/index.js:41:18)
    at Object.<anonymous> (/system/silk/node_modules/silk-sysutils/lib/index.js:44:10)

The long gist you asked for :P --> https://gist.github.com/angelpipe/cc6417649b1eb7bf347d8eb02b093c25

mvines commented 8 years ago

Thanks, so the logcat looks pretty good. I see things from init.silk.rc starting up, like our Bluetooth daemon. Silk-init is failing at boot in the same way as when it's run explicitly. This no longer looks like a victara porting issue and a more general silk build issue, so \o/ for that

For some reason, we're failing to require log/device at https://github.com/silklabs/silk/blob/master/bsp-gonk/vendor/silk/silk-sysutils/lib/index.js#L21. This dependency should be resolved at build time by webpack.

It's possible there was a build error for silk-sysutils that didn't halt the build. Could you try rebuilding sys-sysutils (run mm in https://github.com/silklabs/silk/tree/master/bsp-gonk/vendor/silk/silk-sysutils) to check for an error? Otherwise the contents of /system/silk/node_modules/silk-sysutils/lib/index.js would be interesting to look at. I'll double check later today that I'm not seeing the same in a Nexus build .

angelpipe commented 8 years ago

It's good to hear logcat is not plagued with errors :). I have run mm and it looks to run ok, this is the result:

H6460:silk-sysutils fangel$ mm
============================================
PLATFORM_VERSION_CODENAME=REL
PLATFORM_VERSION=5.1.1
TARGET_PRODUCT=aosp_victara
TARGET_BUILD_VARIANT=userdebug
TARGET_BUILD_TYPE=release
TARGET_BUILD_APPS=
TARGET_ARCH=arm
TARGET_ARCH_VARIANT=armv7-a-neon
TARGET_CPU_VARIANT=krait
TARGET_2ND_ARCH=
TARGET_2ND_ARCH_VARIANT=
TARGET_2ND_CPU_VARIANT=
HOST_ARCH=x86_64
HOST_OS=darwin
HOST_OS_EXTRA=Darwin-14.5.0-x86_64-i386-64bit
HOST_BUILD_TYPE=release
BUILD_ID=LMY47V
OUT_DIR=out
============================================
SILK_PRODUCT: gonkjs
SILK_BOARD: victara
============================================
No private recovery resources for TARGET_DEVICE victara
Export includes file: vendor/silk/silk-sysutils/Android.mk -- out/target/product/victara/obj/NPM/silk-sysutils_intermediates/export_includes

#### make completed successfully (7 seconds) ####

About /system/silk/node_modules/silk-sysutils/lib/index.js, i created a new gist so you can check it here: https://gist.github.com/angelpipe/20021477792c0d78a977a576c3a40bac

I hope we are really close to have a working port.

mvines commented 8 years ago

Thanks, the contents of index.js was very helpful. We had a bug, now fixed by a813116d7f9cacb59e70a00bfe53ea99aec40a39, whereby webpack was unable to locate the silk-log/device module at build time.

Please rebase the PR to the tip of the master branch and rebuild. Hopefully Victara now boots and we can land this PR!

angelpipe commented 8 years ago

I don't know if this is good news, your fix worked!. The problem now is im creating a simple silk project and doing silk run. Im getting this

H6460:test fangel$ silk run
Updating /data/node_modules/test
--------- beginning of main
06-19 19:58:37.379 I/silk-init( 4716): Starting Silk 
06-19 19:58:37.896 I/silk-init( 4683): initializing silk with main: test
06-19 19:58:38.039 I/silk-wifi( 4686): (+0ms) WiFi initializing
06-19 19:58:38.404 I/silk-wifi( 4686): (+366ms) wlan0 state| 213 0 00:00:00:00:00:00 0.0.0.0 0 down
06-19 19:58:38.404 I/silk-wifi( 4686): 
06-19 19:58:38.413 I/silk-wifi( 4686): (+10ms) wpaCli: supplicant not ready, waiting
06-19 19:58:38.423 I/silk-wifi( 4686): (+9ms) wpad restart: wpad error, reason=Error: connect ECONNREFUSED /dev/socket/wpad
06-19 19:58:39.420 I/silk-wifi( 4686): (+997ms) wpaCli: supplicant not ready, waiting
06-19 19:58:39.435 I/silk-wifi( 4686): (+15ms) wpad restart: wpad error, reason=Error: connect ENOENT /dev/socket/wpad

Could it be related to a missing driver?

jainanshul commented 8 years ago

@angelpipe this is due to a race condition where silk is trying to start wpa before it is ready. This usually resolves itself in couple of seconds when wpa subsystem is ready.

angelpipe commented 8 years ago

Hi @jainanshul. The problem is it keeps showing that message over and over until i stop it. I could leave it running for half an hour and the message would keep printing :(

angelpipe commented 8 years ago

i found this on device logcat, i think there's a problem with the wpa subsystem and it is my fault

I/wpa_supplicant(  564): Successfully initialized wpa_supplicant
I/kmsg    (  364): <38>[    4.621397,0] type=1400 audit(1466380924.772:50): avc: denied { execmem } for uid=0 pid=447 comm="node" scontext=u:r:init:s0 tcontext=u:r:init:s0 tclass=process permissive=1
E/wpa_supplicant(  564): Line 1: unknown global field 'mot_wpa_conf_version=3'.
E/wpa_supplicant(  564): Line 1: Invalid configuration line 'mot_wpa_conf_version=3'.
E/wpa_supplicant(  564): Line 25: unknown network field 'skip_auto_connect'.
E/wpa_supplicant(  564): Line 26: failed to parse network block.
I/wpa_supplicant(  564): Long line in configuration file truncated
I/wpa_supplicant(  564): rfkill: Cannot open RFKILL control device
E/wpa_supplicant(  564): Could not read interface wlan0 flags: No such device
E/wpa_supplicant(  564): Could not read interface wlan0 flags: No such device
E/wpa_supplicant(  564): wlan0: Failed to initialize driver interface
mvines commented 8 years ago

Oh, yep that looks suspicious!

I'm not sure if this is still the case, but on the previous device logcat I also noticed this which looked like the modem was not happy -- https://gist.github.com/angelpipe/cc6417649b1eb7bf347d8eb02b093c25#file-logcat-L2643-L2665 Generally we don't care too much about that because we don't have telephony support in Silk yet but that could potentially be a symptom of a larger issue.

mvines commented 8 years ago

@angelpipe -- hey are you still working on anything here, just wondering if this is PR ready to merge. Thanks!

angelpipe commented 8 years ago

Well.... i never stopped working, but i haven't been able to make the wpa_suplicant work. Right now i'm trying a new build structure

angelpipe commented 8 years ago

Hey @mvines. I haven't had luck nor fixing the wpa_supplicant problem nor trying to build with different sources. I don't know if i should spend more time trying to fix it or if i should try getting something that proved to work (like a CyanogenMod complete manifest) and migrate all patches to it to then remove the unnecessary things. Motorola has some info to build android for a variety of devices but sadly, Moto X 2014 is not on the instructions and repos look outdated. What would you recommend to do?

mvines commented 8 years ago

Ugh. In my experience it's super helpful to have a buidable Android image to start from that demonstrates all the desired features (eg, wifi works) . Given the amount of vendor variance in Android I'd not want to start with anything less, there's to much risk of running into a binary blob that does something different with very little way to debug/trace it

angelpipe commented 8 years ago

Hey... I built CyanogenMod 12.1 for victara, it works good when i do it following instructions. Then i tried to integrate my manifest into the bsp-gonk board scheme and built. I had some errors and i managed to fix them to get a successful build. The device keeps booting in the "android" screen. It even does not show the cyanogen logo(which i think is weird as it comes from a cyanogen manifest). I cannot check the logcat cause cyanogenmod comes with that disabled (Maybe i can tinker to make it work from scratch?). The thing is when i build the Silk style, i don't get the cyanogen zip, i just get the images and the generated recovery does not work. Maybe i am flashing it the wrong way?. This way is not proving to be easier :(. Do you have any thoughts?.

PD: Im not using any patches, just the necessary ones to get a successful build.