search

xc-racer99 / u-boot-aries

U-boot port for first-gen Galaxy S devices. Flashed in place of kernel with stock IBL/PBL/SBL. Kernel read from OneNAND/MMC/SD/Serial.
9 stars 2 forks source link

Create new partitions layout on aries and pass it to kernel #9

Closed PabloPL closed 6 years ago

PabloPL commented 6 years ago

Since we have uboot working, maybe it would be good idea to change partition table a bit. We could:

So uboot would pass those mtd partitions to kernel, which could use them for loading (for example android). We could then also enable fastboot for flashing boot/recovery.

PabloPL commented 6 years ago

@xc-racer99 Do You know if partition table layout is much different between aries variants? I'm using now stock PIT file, do you have any for galaxys4, so i could compare them and hardcode in uboot?

MYEUHD commented 6 years ago

This would also allow us to:

The GSM variants have the same partition layout while each one of the fascinate and the galaxy S 4G has it's own. Fasciante PIT: https://forum.xda-developers.com/attachment.php?attachmentid=405371&d=1285089689 Galaxy S 4G partition layout (commented at the end of the file): https://github.com/xc-racer99/android_kernel_samsung_aries/blob/aosp-7.1/drivers/mtd/onenand/samsung_galaxys4g.h

xc-racer99 commented 6 years ago

@PabloPL The big difference in partition layout is that the SGS4G has double the storage capacity. The first few partitions (bootloaders) are the same as on the other GSM versions.

By default, stock has the data partition on the oneNand. Here's a copy of the .pit (VibrantPlus is the SGS4G, Hawk and Sidekick are two more variants that were sold on smaller American carriers IIRC and never had any after-market support to speak of)

Hawk_Sidekick_VibrantPlus_831.pit.zip

@MYEUHD links for the SGS4G partition layout is correct and is the best way to compare (IMO) between the different variants. Note that partition sizes have been adjusted over the different aftermarket versions, notably that the /system size kept increasing :) i believe the other GSM variants also had a few minor changes around ICS but I don't remember exactly what.

The fascinate is essentially the same as the GSM variants, it just doesn't have an EFS partition, I'm not sure how the radio is configured on it.

xc-racer99 commented 6 years ago

It should also be noted that currently the Android kernel doesn't support cmdline partitions due to some modifications made to the driver by Google/Samsung. It is possible to do so, however I have had some issues mapping the BML reservoir which is used in Android when updating the boot.img and there are bad blocks.

PabloPL commented 6 years ago

@xc-racer99 So maybe we should use ubifs? It looks like it's supported on android kernel (3.0). We wouldn't have also problems with resizing partitions (if they are too small like it was done for /system). So we would need:

Partitions with bootloaders and uboot won't be mapped.

xc-racer99 commented 6 years ago

I have played around with UBIFS, the unfortunate thing is that with 3.0's version of UBIFS SELinux isn't supported so Android won't run nicely if at all (dependent on Android version). Trying to backport would be quite difficult as selinux support wasn't introduced until ~3.18 IIRC. I did backport the ubiblock code (so you could have a read-only block device), but this is only useful for system and modem.

The resizing of /system on the OneNAND was only the SGS4G, the GSM variants had the system partition on LVM so that was easy enough to change. Prior to the introduction of LVM it was a bit of a pain.

efs needs to be read-write as on some variants (mine for sure) the modem will make changes to it.

modem definitely doesn't need its own partition as it's actually just a symlink (https://github.com/CyanogenMod/android_device_samsung_aries-common/blob/cm-10.2/init.aries.rc#L148) to support the closed-source blob. Replicant's Samsung-RIL also uses the original path, so adjusting the symlink is the best way to go.

reservoir (if we map it) should be read-write. However, the only time it is used is when updating u-boot so I would favor not even mapping it for userspace.

uboot-env - does it need to be available from userspace?

I would personally favor having the "boot" or the "recovery" partition in the location of the RECOVERY partition in the stock .pit (same for GSM and SGS4G) so that it can easily be updated by Heimdall. This means the offset is 102*SZ_256K

So, here would be my idea: Mapped: boot recovery uboot-env efs (need to determine what filesystem this should use as we probably don't want to use yaffs2 since it isn't mainline) cache (android-specific and useful for Android's recovery) misc (remaining space (stopping at beginning of reservoir) going to a partition which could be used as needed for Android support (eg system or datadata))

Unmapped: BL1 BL2 U-boot Reservoir

PabloPL commented 6 years ago

@xc-racer99 I like partition Your partition layout. Just few comments.

If we make uboot-env available from userspace, we can use linux tools to change it params (https://elinux.org/U-boot_environment_variables_in_linux). Could be dangerous, but usefull for people without serial cable. Let's hide (if not possible, mark as read only).

About making boot/recovery partitions updatable by heimdall, i was rather thinking about enabling fastboot support (so users could use fastboot for flashing boot/recovery/other partitions).

xc-racer99 commented 6 years ago

Sure, I'm good with making uboot-env available from userspace. However, if it's marked as read-only whats the point? For hidden partitions, are you talking about locking them? I can't find any references to hidden mtd partitions. I'm unsure if the OneNAND driver supports locking or not.

With fastboot, I have a couple of concerns: 1) Does the u-boot fastboot support include onenand (ie does common/fb_nand.c support OneNAND as well as regular NAND)? 2) I'm still not very confident that I properly fixed writing on the OneNAND - it might be temporarily causing the bad blocks (ie why self-updating and updating the OneNAND boot partition weren't working properly)

PabloPL commented 6 years ago

For uboot-env - You're right, it doesn't make any sense to make it ro. For hidden partitions i was thinking about not passing them to kernel.

I don't know yet if fastboot supports onenand and looking at fb_nand.c it looks like it doesn't, so for now just skip this idea (sadly).

About writing on OneNAND, we can check if it's working fine under linux

PabloPL commented 6 years ago

Ok, for galaxy s it could look like this

new partition: efs
new partition offset: 2
new partition offset: 0x00080000 (2 * 256 * 1024), because one block = 256k. The same for size
new partition size: 40
new partition size: 10M

new partition: uboot-env
new partition offset: 102
new partition offset: 0x01980000
new partition size: 1
new partition size: 256k

new partition: boot
new partition offset: 103 
new partition offset: 0x019C0000
new partition size: 30
new partition size: 7680k

new partition: recovery
new partition offset: 133
new partition offset: 0x02140000
new partition size: 30
new partition size: 7680k

new partition: misc
new partition offset: 163
new partition offest: 0x028C0000
new partition size: 1651
new partition size: 422656k

new partition: cache
new partition offset: 1814
new partition offset: 0x1C580000
new partition size: 35m

new partition: modem
new partition offset: 1954
new partition offset: 0x1E880000
new partition size: 50
new partition size: 12800k

It would end as 

mtdparts=0:10M@0x00080000(efs),256k@0x01980000(uboot-env),7680k@0x019C0000(boot),7680k@0x02140000(recovery),422656k@0x028C0000(misc),35m@0x1C580000(cache),12800k@0x1E880000(modem)
xc-racer99 commented 6 years ago

Well, your efs partition would need to change - you're currently mapping the BML position of efs which since we don't have support for BML we can't access (plus, it's in a proprietary samsung filesystem - RFS). CyanogenMod got around this by create a new MTD efs partition elsewhere and backing up (from the stock recovery) the efs partition and copying it to the new MTD partition once it was in place. The modem is the same way, so CyanogenMod had a copy of the modem.bin (which had to be padded to exactly 14MB IIRC) and placed in its own MTD partition. The original modem partition on BML was overwritten while the EFS one was left untouched in case the user ever wanted to go back to stock. Both the (new) efs and modem partitions were formatted as yaffs2.

PabloPL commented 6 years ago

Well, your efs partition would need to change - you're currently mapping the BML position of efs which since we don't have support for BML we can't access

Even if i make a backup of efs and modem and then just mount those mtd partitions and put files there (for example on stock recovery/cm/etc)? It was long time since i last time worked with efs...

xc-racer99 commented 6 years ago

Overwriting the modem isn't an issue at all as it isn't device specific and can easily be flashed again. If you overwrite the efs and then somehow lose it again, you no longer have a backup since it's device specific. Since the efs partition doesn't have to be at that offset, it makes more sense to just leave the original blocks there and waste the 12MB of space by creating a new efs partition elsewhere.

IMO, the original efs should be treated as if it were a bootloader and not touched unless absolutely necessary.

MYEUHD commented 6 years ago

efs (need to determine what filesystem this should use as we probably don't want to use yaffs2 since it isn't mainline)

we could use ext4, like it's done on midas https://github.com/fourkbomb/android_device_samsung_midas/blob/lineage-15.1/rootdir/fstab.midas#L5

xc-racer99 commented 6 years ago

@MYEUHD Only if you know how to use ext4 on an mtd device... Read the FAQ at http://www.linux-mtd.infradead.org/faq/general.html for more info.

We're basically limited to UBIFS, JFFS2, or the out of tree YAFFS2. The issue with YAFFS2 is that it isn't in the kernel tree (has to be manually added), the issue with UBIFS is that it doesn't support selinux until around kernel 3.18 so Android doesn't work properly on it until them, I haven't investigated JFFS2.

I will probably investigate backporting a ton of UBI/UBIFS changes for selinux and proper xattr support, but last time I looked it was a huge job.

xc-racer99 commented 6 years ago

Alright, here's my proposal: 10240k@26112k(boot),10240k(recovery),4k(uboot-env),12800k(efs),19712k(radio),40960k(cache),{variable}(misc)

EFS and Radio would be able to be mounted on both Android and Linux and would have a JFFS2 filesystem on them (Android kernel needs it enabled, but it is functional wrt to SELinux even on 3.0). The variable misc partition would be the rest of the partition (starting at 120068k) up to the start of the reservoir (513024k on GSM, 1027072k on SGS4G) so misc sizes would be 392956k on GSM (currently datadata is 430080k) and 907004k on SGS4G.

MYEUHD commented 6 years ago

This seems good to me.

xc-racer99 commented 6 years ago

Branch created at https://github.com/xc-racer99/u-boot-galaxys4g/tree/mtdparts but has yet to be more than compile tested as I need to modify the Android kernel/updater.sh to work. Edit: That branch doesn't boot, needs more work

PabloPL commented 6 years ago

@xc-racer99 Did You enabled in kernnel CONFIG_MTD_CMDLINE_PARTS ? With this i was able to get mtd partitions in kernel.

Edit 1:

mtdparts=0:

It can't be 0, but name of mtd device (check in /proc/mtd or something like this). WIthout this, it won't also create mtd partitions.

Edit 2: It should be 

mtdparts=b0600000.onenand:
xc-racer99 commented 6 years ago

Yes, I did (plus needing a change so the driver will actually use the cmdline parts...). I haven't had an issue getting the cmdline parts working.

I might have just been flashing it improperly as I've been working on the ability to have a flashable zip that backups EFS, flashes u-boot, and then restore the EFS partition.

I also need to finish adjusting the Android build system to use the new partition layout. Will rebase this test branch and create PR when I have a booting Android.

xc-racer99 commented 6 years ago

@PabloPL Wait, can you boot this branch as is (accounting for the defconfig change?) I can't boot u-boot with CONFIG_ENV_IS_IN_ONENAND set

PabloPL commented 6 years ago

It's also not booting, but with current mtdparts i got following info in kernel log (after hardcoding partitions, enabling onenand in kernel etc)

[    2.613855] 7 cmdlinepart partitions found on MTD device b0600000.onenand
[    2.619340] Creating 7 MTD partitions on "b0600000.onenand":
[    2.623629] 0x000001980000-0x000002380000 : "boot"
[    2.627551] 0x000002380000-0x000002d80000 : "recovery"
[    2.631376] 0x000002d80000-0x000002d81000 : "uboot-env"
[    2.634794] mtd: partition "uboot-env" doesn't end on an erase/write block -- force read-only
[    2.652773] 0x000002d81000-0x000003a01000 : "efs"
[    2.659971] mtd: partition "efs" doesn't start on an erase/write block boundary -- force read-only
[    2.676906] 0x000003a01000-0x000004d41000 : "radio"
[    2.684186] mtd: partition "radio" doesn't start on an erase/write block boundary -- force read-only
[    2.701444] 0x000004d41000-0x000007541000 : "cache"
[    2.708840] mtd: partition "cache" doesn't start on an erase/write block boundary -- force read-only
[    2.726242] 0x000007541000-0x00001f500000 : "misc"
[    2.733546] mtd: partition "misc" doesn't start on an erase/write block boundary -- force read-only
xc-racer99 commented 6 years ago

Yeah, I guess they have to be on every 256k as that's the size of the erase block. Can you please test the version I just pushed?

PabloPL commented 6 years ago

After fixing 

0:256k@25856k(uboot-env)

to

b0600000.onenand:256k@25856k(uboot-env)

It looks like it's working fine

[root@alarm ~]# cat /proc/mtd
dev:    size   erasesize  name
mtd0: 00040000 00040000 "uboot-env"
mtd1: 00a00000 00040000 "boot"
mtd2: 00a00000 00040000 "recovery"
mtd3: 00c80000 00040000 "efs"
mtd4: 01340000 00040000 "radio"
mtd5: 02800000 00040000 "cache"
mtd6: 17fc0000 00040000 "misc"
[    0.902561] Muxed OneNAND 512MB 1.8V 16-bit (0x50)
[    0.905372] OneNAND version = 0x013e
[    0.908305] Chip support all block unlock
[    0.910287] Chip has 4KiB pagesize
[    0.912342] Chip has cache program feature
[    0.915152] Scanning device for bad blocks
[    1.955799] random: fast init done
[    2.618570] OneNAND eraseblock 2047 is an initial bad block
[    2.621519] 7 cmdlinepart partitions found on MTD device b0600000.onenand
[    2.626998] Creating 7 MTD partitions on "b0600000.onenand":
[    2.631287] 0x000001940000-0x000001980000 : "uboot-env"
[    2.635602] 0x000001980000-0x000002380000 : "boot"
[    2.639153] 0x000002380000-0x000002d80000 : "recovery"
[    2.642925] 0x000002d80000-0x000003a00000 : "efs"
[    2.651327] 0x000003a00000-0x000004d40000 : "radio"
[    2.658946] 0x000004d40000-0x000007540000 : "cache"
[    2.666540] 0x000007540000-0x00001f500000 : "misc"
xc-racer99 commented 6 years ago

After making fixing

0:256k@25856k(uboot-env)

to

b0600000.onenand:256k@25856k(uboot-env)

Ok, looks like some more work is needed then as it's a different name on 3.0 kernel (well, name isn't present...)

xc-racer99 commented 6 years ago

Implemented in c95e295f7c6cbe80d398f17fc10cabd6cd5b0f9d

uboot-env, recovery, and boot are their own separate partitions, everything else is in UBI. 3.0 kernel needs the patches from https://github.com/xc-racer99/android_kernel_samsung_aries/commits/ubifs-selinux plus the ubi volumes need to be created indendently with ubimkvol, ubiformat, and/or ubiattach. Example kernel for SGH-T959P which does it on boot is available at https://github.com/xc-racer99/android_kernel_samsung_aries/tree/ubifs-convert (relies on flashable zip which is more or less implemented at https://github.com/xc-racer99/galaxys4g_ubi_transform/tree/ubifs-mainline-convert)

PabloPL commented 6 years ago

@xc-racer99 Did You also saw errors when uboot is booting (unaligned memory access or something like this) + it's always restoring default env settings (try to add/modify something, savev env settings and reboot).

xc-racer99 commented 6 years ago

Yes, I noticed the unaligned memory access errors. It's related to https://github.com/xc-racer99/u-boot-galaxys4g/commit/2789210638a4cf784b49347709c27c85990af34e#diff-1145f263256c923716d2b8eade2f6689 which uses code I copied elsewhere. There's an upstream fix for where I copied it from at https://github.com/xc-racer99/u-boot-galaxys4g/commit/e8f65763ef07e0667f57dda7eece657f8fe136a7

Note that I've moved to https://github.com/xc-racer99/u-boot-galaxys4g/tree/fastboot-v2018.09 where I squished all of the aries-specific patches together and actually dropped the onenand DMA change - the difference in read speed is not significant enough for the bigger code base IMO

Edit: I haven't tried to save/restore the env, but will do so now to see if it works or not...

PabloPL commented 6 years ago

@xc-racer99 Any plans to upstream some of those fixes (by the way, are You going to submit also some Your linux kernel patches - for example bma/gp2a or i should submit them)?

xc-racer99 commented 6 years ago

I probably should go through the process of trying to get them submitted, I never have before. If you want to, please do so - this goes for both the u-boot and linux patches. I don't know when or if I will get the chance to.

Also, with the v2018.09-aries branch setenv/saveenv persists over reboot