A home for proprietary blobs

[dk1978]

Hi Max, I have two related issues: 1) Where can we store and maybe even version-control all the proprietary blobs needed for FP3? I vaguely remember someone writing that one of the Github repos in WeAreFairphone might serve for that purpose, but I do not see evidence of anyone doing this - it might cause legal problems as well. I suppose we should simply document the source location and process of extraction? 2) Where and how did you get the vendor files for the 0111 update? Was it something like this location? I would like to create or recreate a full update zip, just like we have now for 0110 and examine any changes to vendor files. If nothing bad pops up, then I suppose it is worthwhile to update the README file with instructions for using 0111 and tag our device tree repo with something like FP3_RELEASE_0111.

[dk1978]

I just downloaded an unzipped the 0111 image I found here. I mounted the vendor images for 0110 and 0111 side-by-side and compared them recursively. This is what I have found:

$ sudo diff -r --no-dereference system.0110/vendor system.0111/vendor
Binary files system.0110/vendor/bin/hal_proxy_daemon and system.0111/vendor/bin/hal_proxy_daemon differ
Binary files system.0110/vendor/bin/mm-audio-ftm and system.0111/vendor/bin/mm-audio-ftm differ
Binary files system.0110/vendor/bin/sensors.qti and system.0111/vendor/bin/sensors.qti differ
Binary files system.0110/vendor/bin/sigma_dut and system.0111/vendor/bin/sigma_dut differ
Binary files system.0110/vendor/bin/test_module_pproc and system.0111/vendor/bin/test_module_pproc differ
diff -r --no-dereference system.0110/vendor/build.prop system.0111/vendor/build.prop
3,5c3,5
< ro.vendor.build.date=Thu Jan 9 20:44:52 CST 2020
< ro.vendor.build.date.utc=1578573892
< ro.vendor.build.fingerprint=Fairphone/FP3/FP3:9/8901.2.A.0110.20200109/01092024:user/release-keys
---
> ro.vendor.build.date=Fri Jan 31 14:37:32 CST 2020
> ro.vendor.build.date.utc=1580452652
> ro.vendor.build.fingerprint=Fairphone/FP3/FP3:9/8901.2.A.0111.20200131/01311418:user/release-keys
Binary files system.0110/vendor/lib/hw/nfc_nci.nqx.default.so and system.0111/vendor/lib/hw/nfc_nci.nqx.default.so differ
Binary files system.0110/vendor/lib/hw/nfc_nci.sn100.default.so and system.0111/vendor/lib/hw/nfc_nci.sn100.default.so differ
Binary files system.0110/vendor/lib/libchromatix_imx363_2nd_common.so and system.0111/vendor/lib/libchromatix_imx363_2nd_common.so differ
Binary files system.0110/vendor/lib/libchromatix_imx363_2nd_default_preview_3a.so and system.0111/vendor/lib/libchromatix_imx363_2nd_default_preview_3a.so differ
Binary files system.0110/vendor/lib/libchromatix_imx363_2nd_default_video_3a.so and system.0111/vendor/lib/libchromatix_imx363_2nd_default_video_3a.so differ
Binary files system.0110/vendor/lib/libchromatix_imx363_2nd_hfr_120_3a.so and system.0111/vendor/lib/libchromatix_imx363_2nd_hfr_120_3a.so differ
Binary files system.0110/vendor/lib/libchromatix_imx363_2nd_hfr_60_3a.so and system.0111/vendor/lib/libchromatix_imx363_2nd_hfr_60_3a.so differ
Binary files system.0110/vendor/lib/libchromatix_imx363_2nd_hfr_90_3a.so and system.0111/vendor/lib/libchromatix_imx363_2nd_hfr_90_3a.so differ
Binary files system.0110/vendor/lib/libchromatix_imx363_2nd_zsl_preview_3a.so and system.0111/vendor/lib/libchromatix_imx363_2nd_zsl_preview_3a.so differ
Binary files system.0110/vendor/lib/libchromatix_imx363_2nd_zsl_video_3a.so and system.0111/vendor/lib/libchromatix_imx363_2nd_zsl_video_3a.so differ
Binary files system.0110/vendor/lib/libmmcamera2_iface_modules.so and system.0111/vendor/lib/libmmcamera2_iface_modules.so differ
Binary files system.0110/vendor/lib/libmmcamera2_imglib_modules.so and system.0111/vendor/lib/libmmcamera2_imglib_modules.so differ
Binary files system.0110/vendor/lib/libmmcamera2_mct.so and system.0111/vendor/lib/libmmcamera2_mct.so differ
Binary files system.0110/vendor/lib/libmmcamera2_sensor_modules.so and system.0111/vendor/lib/libmmcamera2_sensor_modules.so differ
Binary files system.0110/vendor/lib/libmmcamera2_stats_modules.so and system.0111/vendor/lib/libmmcamera2_stats_modules.so differ
Binary files system.0110/vendor/lib/libmmcamera_bokeh.so and system.0111/vendor/lib/libmmcamera_bokeh.so differ
Binary files system.0110/vendor/lib/libmmcamera_imglib.so and system.0111/vendor/lib/libmmcamera_imglib.so differ
Binary files system.0110/vendor/lib/libmmcamera_isp_linearization40.so and system.0111/vendor/lib/libmmcamera_isp_linearization40.so differ
Binary files system.0110/vendor/lib/libmmcamera_ppeiscore.so and system.0111/vendor/lib/libmmcamera_ppeiscore.so differ
Binary files system.0110/vendor/lib/libmmcamera_quadracfa.so and system.0111/vendor/lib/libmmcamera_quadracfa.so differ
Binary files system.0110/vendor/lib/libqcrilFramework.so and system.0111/vendor/lib/libqcrilFramework.so differ
Binary files system.0110/vendor/lib/libremosaic_daemon.so and system.0111/vendor/lib/libremosaic_daemon.so differ
Binary files system.0110/vendor/lib/libril-qc-hal-qmi.so and system.0111/vendor/lib/libril-qc-hal-qmi.so differ
Binary files system.0110/vendor/lib/modules/audio_adsp_loader.ko and system.0111/vendor/lib/modules/audio_adsp_loader.ko differ
Binary files system.0110/vendor/lib/modules/audio_analog_cdc.ko and system.0111/vendor/lib/modules/audio_analog_cdc.ko differ
Binary files system.0110/vendor/lib/modules/audio_apr.ko and system.0111/vendor/lib/modules/audio_apr.ko differ
Binary files system.0110/vendor/lib/modules/audio_cpe_lsm.ko and system.0111/vendor/lib/modules/audio_cpe_lsm.ko differ
Binary files system.0110/vendor/lib/modules/audio_digital_cdc.ko and system.0111/vendor/lib/modules/audio_digital_cdc.ko differ
Binary files system.0110/vendor/lib/modules/audio_hdmi.ko and system.0111/vendor/lib/modules/audio_hdmi.ko differ
Binary files system.0110/vendor/lib/modules/audio_machine_ext_sdm450.ko and system.0111/vendor/lib/modules/audio_machine_ext_sdm450.ko differ
Binary files system.0110/vendor/lib/modules/audio_machine_sdm450.ko and system.0111/vendor/lib/modules/audio_machine_sdm450.ko differ
Binary files system.0110/vendor/lib/modules/audio_mbhc.ko and system.0111/vendor/lib/modules/audio_mbhc.ko differ
Binary files system.0110/vendor/lib/modules/audio_native.ko and system.0111/vendor/lib/modules/audio_native.ko differ
Binary files system.0110/vendor/lib/modules/audio_pinctrl_wcd.ko and system.0111/vendor/lib/modules/audio_pinctrl_wcd.ko differ
Binary files system.0110/vendor/lib/modules/audio_platform.ko and system.0111/vendor/lib/modules/audio_platform.ko differ
Binary files system.0110/vendor/lib/modules/audio_q6.ko and system.0111/vendor/lib/modules/audio_q6.ko differ
Binary files system.0110/vendor/lib/modules/audio_q6_notifier.ko and system.0111/vendor/lib/modules/audio_q6_notifier.ko differ
Binary files system.0110/vendor/lib/modules/audio_stub.ko and system.0111/vendor/lib/modules/audio_stub.ko differ
Binary files system.0110/vendor/lib/modules/audio_swr_ctrl.ko and system.0111/vendor/lib/modules/audio_swr_ctrl.ko differ
Binary files system.0110/vendor/lib/modules/audio_swr.ko and system.0111/vendor/lib/modules/audio_swr.ko differ
Binary files system.0110/vendor/lib/modules/audio_usf.ko and system.0111/vendor/lib/modules/audio_usf.ko differ
Binary files system.0110/vendor/lib/modules/audio_wcd9335.ko and system.0111/vendor/lib/modules/audio_wcd9335.ko differ
Binary files system.0110/vendor/lib/modules/audio_wcd9xxx.ko and system.0111/vendor/lib/modules/audio_wcd9xxx.ko differ
Binary files system.0110/vendor/lib/modules/audio_wcd_core.ko and system.0111/vendor/lib/modules/audio_wcd_core.ko differ
Binary files system.0110/vendor/lib/modules/audio_wcd_cpe.ko and system.0111/vendor/lib/modules/audio_wcd_cpe.ko differ
Binary files system.0110/vendor/lib/modules/audio_wsa881x_analog.ko and system.0111/vendor/lib/modules/audio_wsa881x_analog.ko differ
Binary files system.0110/vendor/lib/modules/audio_wsa881x.ko and system.0111/vendor/lib/modules/audio_wsa881x.ko differ
Binary files system.0110/vendor/lib/modules/mpq-adapter.ko and system.0111/vendor/lib/modules/mpq-adapter.ko differ
Binary files system.0110/vendor/lib/modules/mpq-dmx-hw-plugin.ko and system.0111/vendor/lib/modules/mpq-dmx-hw-plugin.ko differ
Binary files system.0110/vendor/lib/modules/pronto_wlan.ko and system.0111/vendor/lib/modules/pronto_wlan.ko differ
Binary files system.0110/vendor/lib/nfc_nci.nqx.default.hw.so and system.0111/vendor/lib/nfc_nci.nqx.default.hw.so differ
Binary files system.0110/vendor/lib/nfc_nci.sn100.default.hw.so and system.0111/vendor/lib/nfc_nci.sn100.default.hw.so differ
Binary files system.0110/vendor/lib/texfat.ko and system.0111/vendor/lib/texfat.ko differ
Binary files system.0110/vendor/lib64/libqcrilFramework.so and system.0111/vendor/lib64/libqcrilFramework.so differ
Binary files system.0110/vendor/lib64/libril-qc-hal-qmi.so and system.0111/vendor/lib64/libril-qc-hal-qmi.so differ

Surprisingly, the vendor image for 0111 dates from Jan 31st, 2020 according to build.prop while the entire update itself appeared months later.

[z3ntu]

I can create a repo in https://github.com/FairBlobs for you, if you want to.

[dk1978]

@z3ntu: OK, I got mixed up at first. That collection already exists, so sure - let's create an FP3 vendor repo in analogy to what you did before.

[z3ntu]

https://github.com/FairBlobs/proprietary_vendor_fairphone exists now and you both have invites. Have fun :)

[dk1978]

Thanks a lot! I will wait for feedback from Max to decide what best to dump there.

[mstaz]

The extract-files.sh script usually buts all the binaries and generated makefiles in vendor/fairphone/fp3. I think we could maintain this in a repository as well. This would make it little more clear for all to build. Also we'd automatically compare the files on updating. @z3ntu Thanks for creating the repo. Are the legal concerns the reason why you maintain it in a separate space?

[z3ntu]

Not really, with DMCA requests (which happen very seldomly for those types of repos) only repos get taken down and not the whole organization but I'd rather be safe than sorry. :shrug:

[dk1978]

@mstaz: So if I get you right, the workflow would be: 1) Get vendor partition image, say from a 0110 firmware dump and if necessary do any dark magic on it. 2) Mount the vendor image somewhere, then run extract-files.sh on it to produce the vendor/fairphone/fp3 in the local build tree. 3) Upload the entire directory vendor/fairphone/fp3 to https://github.com/FairBlobs/proprietary_vendor_fairphone and decorate commits there with tags such as FP3_0110 and branch names such as lineage-16.0 just like the kernel and device trees.

As prerequisite we would adapt our local build manifest .repo/local_manifests/roomservice.xml to automatically pull in the vendor tree from FairBlobs when using the repo command to build our own LineageOS updates.

Sounds like a great plan, user-friendly and transparent!

[mstaz]

Yes, didn't think about it in that detail yet. But your points sound meaningful to me. 👍 I don't have time to start with it today. Please start with it if you want.