search

CustomROMs / android_local_manifests_i9300

Local manifests and patches for i9300 / t0lte
7 stars 7 forks source link

Mainstreaming efforts for midas platform (N7100/N7105/I9305/I9300/I317) #1

Open tlaurion opened 5 years ago

tlaurion commented 5 years ago

Anyone suggested the manifest to be taken by LineageOs directly so that it is automatically built, and picked up by LineageOs for microg? Can't wait to use this on a maintained OTA i9300 phone with Delta updates. Which would raise developer's attention. And keep that phone alive.

If any help is needed contact me. Awesome work, btw.

You read that? That needs to fly and be upstreamed: https://blog.forkwhiletrue.me/posts/an-almost-fully-libre-galaxy-s3/

Sent from my Galaxy S3 using FastHub-Libre

ChronoMonochrome commented 5 years ago

Hi, @tlaurion.

Anyone suggested the manifest to be taken by LineageOs directly so that it is automatically built, and picked up by LineageOs for microg?

What exactly do you mean? The LineageOS manifest is used in these builds being unmodified (and always a recent version). All the modifications comes to these local manifests.

Can't wait to use this on a maintained OTA i9300 phone with Delta updates.

IIRC this would require a server for that. Anyway if anyone have some ideas on implementing OTA updates, pull requests are always welcome.

You read that? That needs to fly and be upstreamed: https://blog.forkwhiletrue.me/posts/an-almost-fully-libre-galaxy-s3/

Yup, before I ever started any works on S3 I began by reading these exciting articles :)

tlaurion commented 5 years ago

@ChronoMonochrome : i'm talking about making LineageOS build the rom themselves by asking them to pickup your manifest and changeset :)

The only reason why the exynos board phones (i9300, i9305 LTE, n7100...) were dropped from LineageOS was because they were supported by 14.1 only on which support was dropped following their "major version -2 support drop" which made 14.1 supported devices unsupported radically.

Now that you have a "working" build (frontal camera aside, on which I see that you are working on), proposing reintegration would make sense, and will make its way into microG for LineageOS as well which create build for the same supported devices, and will also be supported by /e/.

You have complementary/oppositional information?

ChronoMonochrome commented 5 years ago

@ChronoMonochrome : i'm talking about making LineageOS build the rom themselves by asking them to pickup your manifest and changeset :)

The only reason why the exynos board phones (i9300, i9305 LTE, n7100...) were dropped from LineageOS was because they were supported by 14.1 only on which support was dropped following their "major version -2 support drop" which made 14.1 supported devices unsupported radically.

It's not that easy, I'm afraid. Device doesn't yet meet all the requirements proposed by LineageOS charter plus some hacks that are still used in the sources aren't gonna be merged easily for us. I'm not saying to stop working on exynos stuff but rather saying we are probably (likely) still not ready to claim being OFFICIAL, so need more work on it to do so.

tlaurion commented 5 years ago

@ChronoMonochrome Could you refer to that list?

ChronoMonochrome commented 5 years ago

@ChronoMonochrome Could you refer to that list?

https://github.com/LineageOS/charter/blob/master/device-support-requirements.md

tlaurion commented 5 years ago

@ChronoMonochrome Don't hesitate to edit this post with references of work needing to be done after having checked everything you know meet requirements.

ChronoMonochrome commented 5 years ago

@tlaurion, nice idea! I thought of creating a fork of the charter repo to indicate the progress being done, though.

tlaurion commented 5 years ago

@ChronoMonochrome : Pinpoint where it's at! Interested in making this go forward.

Could do a CI and modify LineageOS Updater url to point to the updater backend on AWS (or whatever free server recommended) for the pre-official community to test this properly?

Never did that before but it seems to be the way to go! What do you think?

ChronoMonochrome commented 5 years ago

Could do a CI and modify LineageOS Updater url to point to the updater backend on AWS (or whatever free server recommended) for the pre-official community to test this properly?

Never did that before but it seems to be the way to go! What do you think?

Yes, that should do. Though I have no clues about which free servers can be suitable for LineageOS updater.

pgarrity18 commented 5 years ago

Apologize if this is a dumb question...

And there we have it: we've successfully removed almost all blobs from the Exynos 4412 found in the i9300. The one blob remaining is probably not replaceable: it's both encrypted and signed by Samsung. However, since we have access to where the code is stored, it would be trivial to wipe every trace of it from RAM once U-Boot starts. (And similarly, it's easy to dump the code that's been loaded into RAM - it is not even remotely interesting...)

forkbomb calls this approach "almost fully libre". How does this differ from the approach used to deblob and remove the Intel ME on GM45 chipsets? Is this something that could ultimately get fsf RYF certification? Or is it just really close?

Just trying to understand how significant this is. Should I order my Note 2 before they all get scooped up? Thanks guys!

tlaurion commented 5 years ago

@pgarrity18:

TLDR: i9300 won't get RYF certification, for the same reason the Lenovo x220 didn't under minifree attempt, and why Librem laptops won't ever receive it: the second CPU still runs binary blobs.

From my understanding, that would not get RYF certification, just like Ivy bridge and Sandy bridge based platforms can't, since Intel ME is still present, boots, and pass control to main CPU, but without having any interface to interact with it from a booted system.

RYF certification requires "complete removal" aka disablement. GM45 permitted that.

For Ivy and Sandy bridges, we talk about neutering, since only ROMP and BUP modules are required to pass control to the main CPU.

For later bridge implementation in platforms, we talk about disablement, since ME is "asked" to disable itself from discovered hidden instructions from positive research, while still requiring a bunch of other modules to be present in ROM to pass control to main CPU. They cannot be removed, me_cleaner can't trim the space. They are there.

In current case, what @fourkbomb means there is that a complete bypass is possible, since there is a kexec happening, from which all memory mappings can be wiped.

There is a possibility to remove any trace in memory of that blob, and that the other CPU having passed control doesn't know that its access had been revoqued. But that needs to be tested thoroughly.

Note the absence of further posts on that blog. Would be nice if @fourkbomb replied to contact attempts. But didn't.

Sent from my Galaxy S3 using FastHub-Libre

pgarrity18 commented 5 years ago

Thank you for the response. It is interesting indeed and I too wish that forkbomb was more engaged with the community. I am trying to wrap my brain around what he did, what these guys did ..

https://blog.fossencdi.org/u-boot-galaxys3.html,

and what other possibilities might still exist. For example, could you repackage the S3 after shorting the bridge and forcing a boot from the SD card via u-boot? Even without RYF certification, the work you guys are doing is exceptional!

tlaurion commented 5 years ago

@fourkbomb: can you help in accomplishing what is missing to mainstream this?

tlaurion commented 5 years ago

Attempted @fourkbomb contact by email.

Sent from my Galaxy S3 using FastHub-Libre

herbsmn commented 5 years ago

The Replicant project is working to port the i9305/i9300 to LineageOS 16.1 using the mainline Linux kernel. Help is wanted. https://redmine.replicant.us/projects/replicant/wiki/Porting_Replicant_to_Android_9/#LineageOS-16

Hit us up on IRC or our email list if you want to join in the fun: https://redmine.replicant.us/projects/replicant/wiki#Contact

herbsmn commented 5 years ago

Commits are happening here: https://git.replicant.us/contrib/GNUtoo/device_i9305.git/log/?h=GNUtoo/lineage-16.0-base

ChronoMonochrome commented 5 years ago

@herbsmn, it's nice to see other people also working on Midas! I've attempted to build LineageOS 15.1 with Lima userspace libs, but shortly gave up. Will try out a Replicant OS build to see if I can get it work on I9300.

herbsmn commented 5 years ago

@ChronoMonochrome are you able to jump in #Replicant on Freenode's IRC to chat with us as you go? ping GNUtoo, Putti, or sensiblemn if you do.

ChronoMonochrome commented 5 years ago

@herbsmn, I've successfully built a i9300 ROM and got to the bootanimation (as article states, it's very slow). Is there any interest to work on it until graphics issues are resolved? I gave another try to Lima driver, but it seems like Lima userspace lib doesn't implement DRI API methods thus it fails to load.

herbsmn commented 5 years ago

The Replicant project has a large interest on continuing to work on mainline LineageOS on midas for a long time. We also plan on replacing the proprietary s-boot with mainline u-boot. Anytime you want to jump in and help out, we would be glad to collaborate with you.

ChronoMonochrome commented 5 years ago

Okay, if I'll get anything useful, I'll contact. Though I'm not sure if I'm able to fix graphics issue, which seems to be the most crucial (unless folks supporting Lima can fix it), but I'll give a try.

herbsmn commented 5 years ago

Sounds great! Here's a bit of an outline on our plan of attack. https://redmine.replicant.us/projects/replicant/wiki/TasksFunding#Graphics-acceleration Any questions or comments you might have would be very welcome.

herbsmn commented 5 years ago

Graphics are now working, but it is very very slow. We were able to get to the homescreen though and we confirmed that the touchscreen is working. We now will be working on graphics acceleration, but perhaps other work can be done now too even though the graphics are super slow.

herbsmn commented 5 years ago

Replicant just had a bit of a breakthrough today on the graphics using a somewhat big hack that needs to be fleshed out. You can see the speed of our new build with the hack on the devices on the left of the videos here, with the devices on the right being what we were previously experiencing: https://grimkriegor.zalkeen.net/public/alaunch.mp4 https://grimkriegor.zalkeen.net/public/slides.mp4 We hope to be pushing code that impliments this new user experience soon, so that anyone can build it.

If anyone wants to help us run LineageOS 16 with a mainline kernel and a mainline bootloader (with proprietary s-boot and proprietary TrustZone removed), with nearly zero proprietary blobs you can contact us here: https://redmine.replicant.us/projects/replicant/wiki#Contact

Documentation on the porting work to Android 9 for the i9300 (and other midas devices) can be found here: https://redmine.replicant.us/projects/replicant/wiki/Porting_Replicant_to_Android_9

ChronoMonochrome commented 5 years ago

@herbsmn,

wow, such a nice progress. I was trying to bring up either Lima or Mali drivers to the mainline kernel, but didn't got any progress so far. I'm still quite busy with other projects, but I'll rebuild it as soon as I have enough time.

tlaurion commented 5 years ago

Awesome guys. Have a n7100, no i9305 yet. Will contact once I equipped myself. Thanks for your awesome work guys!

tlaurion commented 5 years ago

Hmmm. Is it a n7100 I see in the video? :grinning:

dllud commented 5 years ago

@tlaurion it is two i9305 in the video.

herbsmn commented 5 years ago

@ChronoMonochrome @tlaurion glad to hear that you two are interested in Replicant's progress and want to join in the fun! here's our blog post about the awesome work that @dllud and others are doing on the graphics side of things: https://blog.replicant.us/2019/07/graphics-support-for-replicant-9/

herbsmn commented 5 years ago

@ChronoMonochrome I noticed someone over at the ODROID forums tested the lima driver on an Exynos4412 dev board with kernel 5.3-rc3 and it didn't result in a fatal error: https://forum.odroid.com/viewtopic.php?f=55&t=3691&sid=f73a6ad3052ead98bb90fa7854d061db&start=400#p264856

ChronoMonochrome commented 5 years ago

@herbsmn, AndroX (Note 2 developer) claimed that he got Lima drivers running on 4.9 (with a huge support from Simon and other people from Code Aurora Forum). There are several screenshots posted in Note2 Telegram group:

Screenshots ![Screenshot from 2019-08-22 20-39-00.png](https://i.loli.net/2019/08/23/rxM4fpeoCXbDjTU.png) ![photo6249219910542666525.jpg](https://i.loli.net/2019/08/23/FYhuRn5KfqgVAe6.jpg) ![Screenshot from 2019-08-22 20-37-39.png](https://i.loli.net/2019/08/23/uilEKgXMn5dB3cV.png)

Unfortunately, for now no sources are available publicly, and even not clear whether they are going to be released (at least not earlier than first test builds are released). Maybe someone could ping those people from Code Aurora Forum or Simon for getting some help on this.

About Lima driver: I don't really know what is so special about LK 4.9. I've tried myself running 4.9, it seems that it boots but without any graphics. The oldest mainline-ish kernel that still has a working DRM subsystem for me was 4.11 (with a several backports from 4.12). I don't know if that makes sense to go deeper and trying to lower the kernel version to match exactly 4.9 (it will still have to rely on 4.12 backports in order to get DRM working).

By the way, some months ago I found that Samsung has a kernel source code for their Tizen OS (which is almost mainline, the newest version I could find was 4.1): https://git.tizen.org/cgit/platform/kernel/linux-exynos/log/?h=tizen_5.0

It has Mali graphics support integrated, I have some suspects that the platform specific changes may be useful for Lima as well. By itself, LK 4.1 doesn't boot on ReplicantOS (it seems to be the same issues with DRM subsystem), so rebasing changes on top of at least 4.4 or 4.9 is needed (I will try that if I don't succeed with Lima bringup on LK 5.2 / 5.3).

EDIT: Gave it a more attempts to boot up with a different kernel version. On 5.3-rc5 I have:

[    0.872750] exynos-dsi 11c80000.dsi: failed to get regulators: -517
[    0.875231] lima 13000000.gpu: bus rate = 24000000
[    0.875263] lima 13000000.gpu: mod rate = 50000000
[    0.875313] lima 13000000.gpu: failed to get regulator: -517
[    0.875341] lima 13000000.gpu: regulator init fail -517
[    0.875371] lima 13000000.gpu: Fatal error during GPU init

4.17 is a bit more informative:

[    3.123287] exynos-dsi 11c80000.dsi: Failed to get supply 'vddcore': -517
[    3.130099] exynos-dsi 11c80000.dsi: failed to get regulators: -517
[    3.138778] lima 13000000.gpu: bus rate = 160000000
[    3.142192] lima 13000000.gpu: mod rate = 24000000
[    3.146982] lima 13000000.gpu: failed to get regulator: 0
[    3.152374] lima 13000000.gpu: regulator init fail -517
[    3.157556] lima 13000000.gpu: Fatal error during GPU init

4.13 and 4.11 just hangs so I couldn't get any logs here.

For some reason the voltage regulator is not initialized. I'm quite new to this stuff, yet googling for a similar issues didn't give any results so far. Maybe checking how the stock kernel initializes this regulator can give some hints why the mainline kernel fails to do so.

ChronoMonochrome commented 5 years ago

Lima kernel driver is now loading with hacks: https://github.com/ReplicantOS-midas/android_kernel_samsung_smdk4412/commits/master

08-23 05:08:05.858     0     0 I exynos-dsi 11c80000.dsi: failed to get regulators: -517
08-23 05:08:05.860     0     0 I lima 13000000.gpu: bus rate = 24000000
08-23 05:08:05.860     0     0 I lima 13000000.gpu: mod rate = 50000000
08-23 05:08:05.860     0     0 E lima 13000000.gpu: failed to get regulator: -517
08-23 05:08:05.861     0     0 I lima 13000000.gpu: gp - mali400 version major 1 minor 1
08-23 05:08:05.861     0     0 I lima 13000000.gpu: pp0 - mali400 version major 1 minor 1
08-23 05:08:05.861     0     0 I lima 13000000.gpu: pp1 - mali400 version major 1 minor 1
08-23 05:08:05.861     0     0 I lima 13000000.gpu: pp2 - mali400 version major 1 minor 1
08-23 05:08:05.861     0     0 I lima 13000000.gpu: pp3 - mali400 version major 1 minor 1
08-23 05:08:05.861     0     0 I lima 13000000.gpu: l2 cache 128K, 4-way, 64byte cache line, 64bit external bus
08-23 05:08:05.862     0     0 I         : [drm] Initialized lima 1.0.0 20190217 for 13000000.gpu on minor 0
...
08-23 05:08:05.973     0     0 E regulator_load_fn: init
08-23 05:08:05.973     0     0 E regulator_load_fn: enabled DSI regulator, ret=0
...

It seems to be ok, according to kernel log, this voltage regulator can be loaded but needs a time to initialize (and it's initialized at least without any error codes).

But now there is another problem, it doesn't look like drm hwcomposer is compatible with Lima:

08-23 05:08:14.964   331   331 E hwc-drm-device: Failed to set universal plane cap -1
08-23 05:08:14.964   331   331 E hwc-resource-manager: Failed to initialize any displays
08-23 05:08:14.964   331   331 E hwc-drm-two: Can't initialize the resource manager -22
08-23 05:08:14.964   331   331 E hwc-drm-two: Failed to initialize DrmHwcTwo err=6
08-23 05:08:14.964   331   331 E ComposerHal: failed to open hwcomposer device: Invalid argument
08-23 05:08:14.965   331   331 E android.hardware.graphics.composer@2.1-service: Could not get passthrough implementation for android.hardware.graphics.composer@2.1::IComposer/default.

In the case if anyone interesting, here is the full boot log: https://gist.github.com/ChronoMonochrome/3ea0ab3bfc6677f412810e100ac9a6da

herbsmn commented 5 years ago

Very exciting! Thanks for sharing!

I'm curious, are you using freedesktop's version of drm hwcomposer? https://gitlab.freedesktop.org/drm-hwcomposer/drm-hwcomposer

dllud commented 5 years ago

Great news @ChronoMonochrome !

It is normal for drm_hwcomposer to not work with Lima. AFAIK Lima should only expose a render node (the Mali GPU is not connected to a display). You will have to use drm_hwcomposer with the master node from drm/exynos. drm/exynos and drm/lima can then share buffers through PRIME.

ChronoMonochrome commented 5 years ago

@herbsmn, Yes, I'm using the same drm hwcomposer as posted in the ReplicantOS manifest (so it's freedesktop version). I had to update mesa3d repo and rebase a few patches on top of it (building with original repo was giving errors with Lima enabled).

@dllud, oh, seems to be complicated. I have left DRM_EXYNOS enabled in kernel, but I guess, some extra changes are still needed in drm_hwcomposer?

dllud commented 5 years ago

It should just be a matter of pointing drm_hwcomposer to the correct dri node on your system (probably /dev/dri/card0). You can take a look at our system.prop.

I haven't tried the buffer sharing part yet, so I am helpless at that. Before diving into Lima we will first try to have drm_hwcomposer and gbm_gralloc working properly with DRM Magic (more info is available at the Replicant wiki: Graphics on Replicant 9).

PS. If you feel that sharing ideas live will help you, please drop by at Replicant's IRC channel on Freenode (#replicant).

herbsmn commented 5 years ago

Here's some pictures of a person successfully running Android on MALI-400 using lima with a Orange Pi Plus 2E: https://gitlab.freedesktop.org/lima/mesa/issues/120

ChronoMonochrome commented 5 years ago

Here's some pictures of a person successfully running Android on MALI-400 using lima with a Orange Pi Plus 2E: https://gitlab.freedesktop.org/lima/mesa/issues/120

These are exciting news to hear! Finally I was myself able to get Lima running on S3 too (thanks to changes proposed in OrangePi Plus 2E repos), have a look at https://github.com/ReplicantOS-midas (had to update mesa3d, device tree and kernel). It's running a somewhat better than with a software rendering (still it could be probably much faster, because for now Mali seems to be running at 50 MHz). Having the same issues described in the link above (graphic artifacts and freezes), gonna try a fix proposed.

herbsmn commented 5 years ago

This is all very exciting indeed! Thanks for the news! Looking forward to hearing if the proposed fix for the artifacts and freezes worked for you.

ChronoMonochrome commented 5 years ago

@herbsmn,

applying the suggested fix indeed fixed some textures being not rendered (however, some UI elements are still missing or rendered not properly). Freezes seems to have a different nature. It's quite difficult for me to get any logs at this point, because when it freezes, the ADB connection drops as well, and after a hard reboot it's impossible to get logs from a previous boot. So far I was able to get just one log (https://gist.github.com/ChronoMonochrome/5e72ed87061f46539863c1fd572ebd63) which seems to be related to some sort of memory allocations issues. I've applied a patch that hopefully should resolve memory allocation issues - https://github.com/ReplicantOS-midas/android_kernel_samsung_smdk4412/commit/0a1164f2400d1f8c3f3e84aa18b28e4452deeeac (I can't recall right now where exactly to find an original patch, but this idea of hacking the MAX_ORDER was present in some of Samsung kernel sources).

Still having a frequent freezes though (and sometimes they occur even when issuing a dmesg or logcat in ADB - I've also took USB configs from Orange PI Plus 2E device tree, but have no clues if it's anyhow a cause of freezes).

fourkbomb commented 5 years ago

As far as I know, lima should be able to use non-contiguous memory on exynos. Make sure you have Exynos IOMMU enabled in the kernel? You might also have to tweak the way gralloc allocates memory to get it working.

ChronoMonochrome commented 5 years ago

@fourkbomb, thanks for participation in a discussion!

Make sure you have Exynos IOMMU enabled in the kernel?

Yeah, I think it should be already enabled. I might've wrongly assumed that memory allocation issue I faced was the only problem. There probably should be something else causing UI freezes. Anyway I'll check how frequently allocation errors appears in log (without MAX _ORDER hack).

You might also have to tweak the way gralloc allocates memory to get it working.

Umm, honestly, I don't know what exactly to look for. Anyway I'll try to log how much memory it needs to allocate at once to see if we indeed exceeding the limit set by MAX_ORDER.

herbsmn commented 5 years ago

Here's a video of lima running on Replicant 9: http://www.belg.in/replicant_9.webm

javifo commented 5 years ago

Wow! I'm impressed with the progress!

I would like to build it and test it. I have to organize me to get a couple hours/week free to fix things. I'm interested mainly in gralloc/hwcomposer.

javifo commented 5 years ago

Lima driver do allow non-cma memory

https://gitlab.freedesktop.org/lima/mesa/issues/29

herbsmn commented 4 years ago

Hi @ChronoMonochrome. I just wanted to check in to see if you've been working on this at all.

ChronoMonochrome commented 4 years ago

Hi, @herbsmn

Last time I attempted using LOS16.0 on a mainline (it was about month ago), I can say it's already much better than it was before (I got rid of kernel crash / freeze issue, I think it was totally unrelated to Lima). There's also some progress implementing audio support (again, only thanks to @fourkbomb work). Managed to get the speaker to work (but that's all, neither headphones nor microphones do work). I switched for now to the non-mainline kernel because of issues I can't solve (Lima still has crashes in userspace + audio-related issues). I'm planning to check soon if the headset can be fixed by using a proper audio policy config. I'll leave the link to the audio patches in the case anyone can find it useful: https://github.com/ReplicantOS-midas/android_kernel_samsung_smdk4412/commits/lk-5.3

herbsmn commented 4 years ago

Hi @ChronoMonochrome,

Replicant would love to work more collaboratively on this project with you, if you'd be interested. Please don't hesitate to pop into our IRC or email list from time to time to provide us with details of small victories or frustrating problems. Otherwise, at least two of us are following this github thread if you prefer to post updates here. Thanks so much for continuing this important endevour! Cheers!

ChronoMonochrome commented 4 years ago

@herbsmn

I was planning to post on IRC that aforementioned kernel freezes are resolved now (but since it appeared they aren't related to the Lima at all, but rather caused by my mods to the device tree, I thought that doesn't even worth mentioning). I don't know if I will return to the mainline kernel any soon, but if I'll find anything that worth any interest (either it's a win or a fail), I'll post it on IRC.

Viciouss commented 4 years ago

Hey there, I'm very happy I stumbled upon this thread, a lot of useful links to read through.

I'm quite new to the kernel world and Android (like 6 months) and still learning new things every day but I try to work my way through all of this so that I can run a mainline kernel on my galaxy note 10.1, which is very similar to the i9300. I'm able to boot into buildroot with a 5.4 kernel, display, touch, wifi and some other stuff already working. I can run kmscube but I think it's not using the lima driver but software rendering as it's really slow. I'm also setting up an Android build (10 right now) in parallel with drm hwcomposer but I'm stuck at selinux context initialization in the init program so no positive boot yet.

I wanted to thank you all for your great work on the exynos4 as I found a lot of resources all over the place which helped me to grow my knowledge bit by bit and especially @fourkbomb because your blog motivated me to look beyond my own nose and do something different from my 9-to-5 java employment in the first place.