search

lineage-rpi / android_frameworks_native

Other
1 stars 4 forks source link

NOOBS/PINN format #2

Closed procount closed 2 years ago

procount commented 6 years ago

Hi,

I have been asked if I can support your Lineage OS build in my PINN OS Installer / multi-boot manager for the RPi3, like I did with RTAndroid. This would require converting your image to NOOBS tar.xz format and providing some additional meta-file information. (I haven't tried converting it yet, but I assume it will work in the same way as RTAndroid).

I note from your webpage that you prefer people not to mirror your images, so I was wondering if you would be prepared to host such a NOOBS/PINN image of your OS that PINN can simply reference? This would also ensure that you could update your PINN image regularly as your image main changes.

If you are amenable to this request I can try converting your Lineage image and provide information on how you can do this conversion in the future.

KonstaT commented 6 years ago

Hi.

I haven't looked into NOOBS/PINN or any other multi-boot solution for Pi 3. Does e.g. installing gapps still work if you use this method to install?

My LineageOS build has one less partition (no cache partition) than android-rpi (never tried RTAndroid either) so that's probably the only relevant change.

I've asked not to mirror my builds mostly that people don't end up with outdated stuff (better just always to link the original source). It's also nice to have to somewhat accurate download/usage statistics. This hasn't been much of a problem lately.

I'm pretty swamped with other things for the next few weeks but we'll see later. If a NOOBS/PINN image can be created and it's simple enough procedure I see no reason why this couldn't happen. :)

Thanks.

procount commented 6 years ago

Thanks for your favourable response.

Does e.g. installing gapps still work if you use this method to install?

I guess it depends on how you install gapps normally (via adb? or script?). IIRC, RTAndroid ran a script, but it had hardcoded partitions in it and needed adaption, but I don't think it was ever done.

I will try and create a PINN image. I may have some questions if I come across any problems, but I hope my experience on RTAndroid will stand me in good stead ;)

Since I provide 30-50 OS versions now, I have automated the conversion process with a bash script for Ubuntu, so once it is done you should be able to update the image very easily whenever you have a new release.

There is really only one thing I would ask of you if the conversion process is successful - During the installation of an OS, PINN displays a slideshow for each of the OSes it is installing, so it would be really good if you could provide a few png files (2-5?) that could be used for Lineage. Typically these describe some of the features, benefits, included programs etc. that the OS includes (marketing type stuff, sometimes just screenshots of the OS webpage are sufficient). This is just a heads up for you to think about. It's certainly not essential at the moment because I need to create a working image first!

KonstaT commented 6 years ago

I guess it depends on how you install gapps normally (via adb? or script?). IIRC, RTAndroid ran a script, but it had hardcoded partitions in it and needed adaption, but I don't think it was ever done.

Using the same gapps script from RTAndroid so that's going to be an obvious problem for some users.

Sounds good otherwise.

procount commented 6 years ago

IIRC, that script looked like it could be suitably modified, so I'll take a look at it when I get that far. Do you also use the same scripts as RTAndroid for modifying the display for the RPF touchscreen, or the wavescreen devices?

KonstaT commented 6 years ago

No, I don't have any display hardware to test with so that's still WIP. I have the required drivers for original Raspberry display included in the image but user still needs to manually make couple of modifications to the configs to get that going.

procount commented 6 years ago

I just completed step 1 which was to just burn the standard lineage image to an SD card and boot it. I used a RPi3 on a Motorola Lapdock. The only problem I saw was that the mouse cursor was corrupted and just looked like a 16x16 array of random dots.

I made a first attempt at step2: converting the image, but I'll have to adapt my script as Lineage is packaged a bit different to RTAndroid, but the adaptions to ramdisk for the different partitions are similar, so it is looking promising.

procount commented 6 years ago

Q: In the Lineage image, there are 3 partitions: boot, system and data. The data partition is 2.7GB in size and is empty. What are the constraints or recommendations for the size of this partition? is 2.7GB the minimum recommended size? Should it grow to fill the remaining space on the disk?

Q: Also, What about the system partition? Should this also grow in size? If a user installs apps, are they stored on the system or data partition?

procount commented 6 years ago

I have completed my first image conversion and installed it to mmcblk0p6,7&8 When I booted it, I got no screen output, but I found that the data partition has now been populated with lots of folders: anr, app, app-asec ...user,user_de so I guess it has actually booted and mounted the correct partitions. Do you think this is just a display problem, or can I tell how far it got into the startup process?

KonstaT commented 6 years ago

Q: In the Lineage image, there are 3 partitions: boot, system and data. The data partition is 2.7GB in size and is empty. What are the constraints or recommendations for the size of this partition? is 2.7GB the minimum recommended size? Should it grow to fill the remaining space on the disk?

Yes, 2.7GB is the minimum recommended data partition size and it can grow to fill the remaining space.

Q: Also, What about the system partition? Should this also grow in size? If a user installs apps, are they stored on the system or data partition?

System partition is mounted read-only and it doesn't change after you've written the image (and optionally installed gapps). User installed apps go to data partition.

Do you think this is just a display problem, or can I tell how far it got into the startup process?

Do you have a serial console adapter? It's quite difficult to get logs otherwise (you could add services to init that write dmesg/logcat somewhere on the storage).

procount commented 6 years ago

I guessed right then, as that is how I have configured the partitions.

Yes, I have a USB<->serial debug cable somewhere. I'll dig it out tonight. I assume it is enabled by default and will act just like a Linux shell terminal, or do I need to use adb? (I confess I have never developed on Android nor used adb. The closest I've got is using my smartphone)

KonstaT commented 6 years ago

Serial console is enabled by default on the rpi3 and you need to use minicom/putty/etc to connect to the device your adapter creates (usually /dev/ttyUSB0). See https://developer.android.com/things/hardware/raspberrypi.html#serial_debug_console

Can't use adb until you've connected to wifi.

procount commented 6 years ago

Here is the log from the serial console. I didn't see anything standout, but maybe you can. At the end of the log it just stopped and was unresponsive (i.e. no full terminal shell as I was expecting). This log was from using the RPF touchscreen, but when I used a HDMI monitor, it also remained blank.

[ 0.000000] Booting Linux on physical CPU 0x0 ... [ 4.897972] init: Service logd does not have a SELinux domain defined. [ 4.907359] init: SELinux: Could not set context for /sys/kernel/debug/tracing: Operation not supported on transport endpoint [ 4.928600] init: SELinux: Could not set context for /cache: Read-only file system [ 4.946353] init: Service exec 1 (/system/bin/recovery-refresh) does not have a SELinux domain defined. [ 4.960692] init: (Loading properties from /system/build.prop took 0.00s.) [ 4.967787] init: (Loading properties from /vendor/build.prop took 0.00s.) [ 4.974870] init: (Loading properties from /factory/factory.prop took 0.00s.) [ 4.982350] init: /recovery not specified in fstab [ 4.988042] init: Service debuggerd does not have a SELinux domain defined. [ 4.995308] init: do_start: Service debuggerd64 not found [ 5.004064] init: Service vold does not have a SELinux domain defined. [ 5.014808] init: Not bootcharting.

KonstaT commented 6 years ago

Looks fine apart from the 'Service xxx does not have a SELinux domain defined.' lines. Does all the files in /system have proper SELinux context (see e.g. 'ls -Z /system/bin')?

procount commented 6 years ago

OK, I'll have to check. Just so you know, what I do is to loop mount the lineage image partitions, and then tar each one up with bsdtar --numeric-owner --format gnutar -cpvf $output/$part1.tar . Would that discard any SELinux contexts?

procount commented 6 years ago

I just did a simple check by tarring and untarring one of the loop mounts, and it does discard the SELinux contexts 😞. I'll have to think of a way around that....

EDIT: I think I found a way round it. I shall test later.

procount commented 6 years ago

Which models of RPi is this compatible with? I just put the plain image I had burned to an SD card into an RPi 2B v1 (the BCM2836 model) and it worked (apart from the mouse pointer issue). I suppose RPi3 is the preferred model for performance reasons, but it looks like it will work on the RPi2 v1 and v1.2 (BCM2837 model). What about CM3 or CM3L?

One of the meta files that needs to be created for PINN requires a list of RPi models that the OS can be installed on to prevent installing on incompatible models. Do you have a list, or should it only be targeted at the RPi3?

procount commented 6 years ago

It works! The mouse cursor is still corrupted and the Gallery App "keeps stopping", but I am posting this note from a PINN installed version of Lineage!

KonstaT commented 6 years ago

Well done! :)

procount commented 6 years ago

Actions remaining: (for me):

(for you):

Questions:

KonstaT commented 6 years ago

(I see from your youtube video, that the mouse cursor problem is common) is there a problem with the gallery App, or is it just me? Is the serial console just for monitoring the bootup logs, or is it a proper shell? The keyboard seems unresponsive when connected over putty. Am I right in thinking that lineage does not use the green LED to indicate SD card access? It looks like it is using the red power LED for this purpose.

procount commented 6 years ago

Hi KonstaT,

I was told of an updated version of your Lineage 14.1 (20180615), so I have been converting it to be installed by PINN. When I uploaded it to Sourceforge, their virus checker said the System partition was infected with Android.Trojan.Ransom.gTGSI. I ran McAfee over it and it came up clean, so it could be just a false positive. The only thing I have done to convert it is to mount the system partition from your image and archive the files into a .tar.xz file. You can access it at https://sourceforge.net/projects/pinn/files/os_next/lineage/system_lin.tar.xz/download I don't know if you are able to check this file and somehow confirm that there is no malware in it or reassure me? I'm sure it is just a false positive, but obviously, I don't want to release it if there is a problem with it. Thanks, Procount

KonstaT commented 6 years ago

I haven't added any malware and even if there was one, I've provided source code for it. :P You can always build from source yourself if you don't trust my builds. https://github.com/lineage-rpi/android_local_manifest/blob/cm-14.1/README.md

procount commented 6 years ago

Thanks. Like I said, I'm sure it's just their virus checker flagging up a false positive. But I'm not so experienced on linux or Android malware so I just wanted to refute what their checker was saying so it can be whitelisted.

procount commented 6 years ago

FYI - I checked it with McAfee (not sure if that includes Android type malware) and I installed ClamAV and tested it with that. Both tested clean, as I expected, so I hope that will be enough evidence for them.

procount commented 6 years ago

Hi KonstaT, Sourceforge have come back to me with more information about the suspected virus. app/WallpaperPicker/WallpaperPicker.apk is the file that triggered the virus scan. It's VirusTotal report is at https://www.virustotal.com/#/file/f2655b7951e8be98624b78fa1398c0648739c81635dbc27e54a3d44dad979588/detection It seems to have triggered 10 virus engines.

KonstaT commented 6 years ago

Source code for WallpaperPicker.apk: https://github.com/LineageOS/android_packages_apps_WallpaperPicker/tree/cm-14.1 If it helps, all LineageOS 14.1 downloads on the official server (https://download.lineageos.org/) are also affected (tested WallpaperPicker.apk from lineage-14.1-20180808-nightly-falcon-signed.zip).

Poked around a bit more and this is the exact offending commit (https://github.com/LineageOS/android_packages_apps_WallpaperPicker/commit/2b021c4a1c0fd542f48d2e4f6e1a6caed402a3f9). Revert it and it passes VirusTotal (well, only 1/60). Same commit is also in lineage-15.1 branch but that WallpaperPicker.apk passes. Safe to say that it's a false positive.

procount commented 6 years ago

Many thanks! Good to know. ( I feel like "piggy in the middle" here 😄 ) I have passed back this info.

procount commented 6 years ago

Finally, I got it whitelisted! Now the only problem is getting it to work properly...

pr0fsmith commented 5 years ago

Was this ever completed? I'd love to be able to install Lineage OS 16 on my Raspberry Pi 3 b+ and be able to dual boot it with another OS.

procount commented 5 years ago

Not yet. I had some problems with the latest versions, but I'll have another go at it when I start updating my OSes again for PINN.

pr0fsmith commented 5 years ago

response_container_BBPPID{font-family: initial; font-size:initial; color: initial;} Will your current setup work on Raspberry Pi 3 b+? I'd really like to do this. Sent via BlackBerry Hub+ Inbox for Android From: notifications@github.comSent: August 27, 2019 4:01 AMTo: android_frameworks_native@noreply.github.comReply to: reply@reply.github.comCc: strivingforholiness@gmail.com; comment@noreply.github.comSubject: Re: [lineage-rpi/android_frameworks_native] NOOBS/PINN format (#2) Not yet. I had some problems with the latest versions, but I'll have another go at it when I start updating my OSes again for PINN.

—You are receiving this because you commented.Reply to this email directly, view it on GitHub, or mute the thread.

procount commented 5 years ago

I had reports that my current conbersions are somehow broken so I need to work on Konsta's latest v16.0

pr0fsmith commented 5 years ago

response_container_BBPPID{font-family: initial; font-size:initial; color: initial;} Darn. Thanks for responding. Is this something I can help with? I am a newb at this stuff though. Sent via BlackBerry Hub+ Inbox for Android From: notifications@github.comSent: August 27, 2019 7:13 AMTo: android_frameworks_native@noreply.github.comReply to: reply@reply.github.comCc: strivingforholiness@gmail.com; comment@noreply.github.comSubject: Re: [lineage-rpi/android_frameworks_native] NOOBS/PINN format (#2) I had reports that my current conbersions are somehow broken so I need to work on Konsta's latest v16.0

—You are receiving this because you commented.Reply to this email directly, view it on GitHub, or mute the thread.

procount commented 4 years ago

Hi @KonstaT

I'm just finalising my conversion of your RPI3 and RPI4 Feb 2020 versions for installation using PINN. I have managed to convert the TWRP recovery program as well! But I have some questions:

  1. The RPi3 version has separate images for installation to USB. I notice that not only does fstab.rpi3 change mmcblk0p->sda between these 2 versions, but the same mods are made in vendor.file.contexts. 1a. Up til now I haven't updated vendor.file.contexts, but should I also fixup these references? Not sure what this file is. Is it for information purposes only, or is it enforcing policies in SELinux? 1b. I think I can dispense with ramdisk-usb.img as PINN will update these partition references and these seem to be the only differences.

  2. The ramdisk-recovery-usb.img still seems to have fstab.rpi3 and vendor.file.contexts, but these seem to be identical to ramdisk-recovery.img, so can I assume they are not being used?

  3. /etc/recovery.fstab seems to be the one that is used instead. But does vendor.file.contexts still need updating accordingly?

  4. If PINN is updating recovery.fstab, Can I also dispense with ramdisk-recovery-usb.img?

  5. Partition size wise, PINN creates the partitions first and then copies the files afterwards, so partition sizes are not constrained by an image file. A minimum partition size is provided, but some can be expanded to fill the disk (equally shared with other OS partitions). Currently I use: Boot: 127MB, fixed System: 1GB, fixed Vendor: 248MB, expandable Data: 2581MB, expandable

Does that sound ok, or should the Vendor partition size be fixed?

  1. In a previous version I made the partition names "required", which prevented multiple copies of LineageOS being installed, or even other OSes being installed if they shared a common partition name. I just did a test and called the partitions, B,S,V & D and it still worked, so I believe the partition names are just informational and can be changed in the case of clashes. Can you confirm?

Thanks.

KonstaT commented 4 years ago
  1. SELinux is in permissive mode (for now at least). You'll see some denials in audit log if you don't change the sepolicy but it doesn't prevent anything from working. Only fstab changes are needed for USB boot so whatever works for you.

2 & 3. That is correct, fstab.rpi3 is not used in recovery - /etc/recovery.fstab is. Also recovery has SELinux permissive so same as 1.

  1. Yes, same as 1.

  2. Partitions size are OK. There's no need for vendor partition to grow so rather make it fixed (it's mounted read-only and normally never changed).

  3. Partition names are not used. Only using the device paths (e.g. /dev/block/mmcblk0p*).

procount commented 4 years ago

Thanks. Nearly there! (SELinux is a mystery to me, so I'm glad I don't have to deal with it) Do you know if it would work if I use PARTUUID refs in fstab? I think that's the default if PINN installs to USB to avoid the /dev/sda device names changing when there are multiple USB drives. Not sure how that would affect the usb-otg functionality, though.

KonstaT commented 4 years ago

No, AFAIK using PARTUUID doesn't work in Android fstab.

procount commented 4 years ago

OK. I may have to remove that option then for now. Thanks.

procount commented 4 years ago

Hmmm. 200212_rpi4 version now works with PINN, but I can't get TWRP working on the 200207_rpi3. Even if I flash the raw image and swap ramdisk.img with ramdisk-recovery.img (nothing else) without using PINN. After the rainbow screen, I get a blank screen that keeps changing screen resolution every couple of seconds but otherwise shows nothing. The LineageOS ramdisk works fine though and the MD5 checksum was ok. I tried it with the RPF touchscreen and my HDMIPI, with a 3B and a 3B+.

KonstaT commented 4 years ago

Pi 3 needs changes to config.txt as well because TWRP doesn't use hw accelerated graphics. Check /system/bin/rpi3-recovery.sh for required changes.

procount commented 4 years ago

Ahh! that explains it. In fact I as just doing a test install of LineageOS with PINN from the internet and I followed your FAQ to switch to recovery mode from the terminal and that worked. So all's good. Just me being ignorant. Thanks.