Closed DrYak closed 2 years ago
Olf0
commented
2 years ago Sorry, I currently fail to comprehend the issue:
/ and one for /home. IIRC the directory /opt resides on the root volume.
What do you intend to address with the term "/opt partition"?/opt directory there) not sufficient?Olf0
commented
2 years ago Actually I have more questions:
- for users of microG (like me), enabling the older mapsv1 API used by some older apps requires patching system.img
The way I understood your guides at TJC and FSO, especially your latest posting, you advise against patching system.img, because deploying and using microG works fine without it (and with a GUI-only setup process) on all officially supported devices since the Xperia XA2 series (i.e., all Xperias except the Xperia X), correct?
- as far as I've heard using the original Google Play Services requires patching system.img
So if a user keeps both a backup of system.img and the patched one, the /opt partition doesn't have enough room left for an updated system.img and the upgrade fails.
IMO backups shall not reside on system partitions, right?
I am critically questioning this on a technical level (which I fail to fully understand, yet) as well as I am trying to assess how relevant this is.
I will weigh the latter against how much effort I have to put into this and how intrusive this is (to the extant sfos-upgrade script).
- Why are the extant checks for space on the root volume (which include an /opt directory there) not sufficient?
Can you please provide a pull request or at least some shell code (in this thread here) to provide an example what you are thinking of.
Olf0
commented
2 years ago @DrYak: Ping.
Olf0
commented
2 years ago @DrYak, I will close this after Christmas due to the lack of (any) response, if this is still the case then.
DrYak
commented
2 years ago Sorry for the glavial speed of answers: extremely busy with the data analysis with the current pandemic.
IIRC the directory /opt resides on the root volume.
That is not waht I am observing on my Xperia XA2 Ultra:
> ssh xperia
Last login: Wed Dec 22 14:10:50 2021 from 192.168.2.3
,---
| Sailfish OS 4.3.0.12 (Suomenlinna)
'---
[nemo@Sailfish ~]$ mount
/dev/mapper/sailfish-root on / type ext4 (rw,noatime,data=ordered)
/dev/mmcblk0p68 on /odm type ext4 (ro,relatime,data=ordered)
/dev/mmcblk0p71 on /opt type ext4 (rw,relatime,data=ordered)
/dev/mmcblk0p42 on /firmware type vfat (ro,relatime,uid=1000,gid=1000,fmask=0337,dmask=0227,codepage=437,iocharset=iso8859-1,shortname=lower,errors=remount-ro)
/dev/mmcblk0p40 on /bt_firmware type vfat (ro,relatime,uid=1002,gid=3002,fmask=0337,dmask=0227,codepage=437,iocharset=iso8859-1,shortname=lower,errors=remount-ro)
/dev/mmcblk0p44 on /dsp type ext4 (rw,nosuid,nodev,relatime,nodelalloc,errors=panic,data=ordered)
/dev/mmcblk0p2 on /persist type ext4 (rw,nosuid,nodev,noatime,nodelalloc,errors=panic,data=ordered)
/dev/mmcblk1p1 on /run/media/nemo/Ulysse31 type btrfs (rw,relatime,thread_pool=8,ssd,noacl,space_cache,subvolid=257,subvol=/data)
/dev/mapper/sailfish-home on /home type ext4 (rw,noatime,data=ordered,jqfmt=vfsv0,usrjquota=aquota.user)
:Android smartphones tend to have a giant pile of various partition, and even after switching to Sailfish OS that's still the case on Sony Xperia:
[nemo@Sailfish ~]$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
loop0 7:0 0 334.7M 0 loop
mmcblk0 179:0 0 29.1G 0 disk
|-mmcblk0p1 179:1 0 2M 0 part
|-mmcblk0p2 179:2 0 32M 0 part /persist
|-mmcblk0p3 179:3 0 1K 0 part
|-mmcblk0p4 179:4 0 2M 0 part
|-mmcblk0p5 179:5 0 2M 0 part
|-mmcblk0p6 179:6 0 2M 0 part
|-mmcblk0p7 179:7 0 16M 0 part
|-mmcblk0p8 179:8 0 64K 0 part
|-mmcblk0p9 179:9 0 64K 0 part
|-mmcblk0p10 179:10 0 3.5M 0 part
|-mmcblk0p11 179:11 0 3.5M 0 part
|-mmcblk0p12 179:12 0 4M 0 part
|-mmcblk0p13 179:13 0 4M 0 part
|-mmcblk0p14 179:14 0 512K 0 part
|-mmcblk0p15 179:15 0 512K 0 part
|-mmcblk0p16 179:16 0 512K 0 part
|-mmcblk0p17 179:17 0 512K 0 part
|-mmcblk0p18 179:18 0 512K 0 part
|-mmcblk0p19 179:19 0 512K 0 part
|-mmcblk0p20 179:20 0 1M 0 part
|-mmcblk0p21 179:21 0 1M 0 part
|-mmcblk0p22 179:22 0 1M 0 part
|-mmcblk0p23 179:23 0 1M 0 part
|-mmcblk0p24 179:24 0 256K 0 part
|-mmcblk0p25 179:25 0 1M 0 part
|-mmcblk0p26 179:26 0 1M 0 part
|-mmcblk0p27 179:27 0 1M 0 part
|-mmcblk0p28 179:28 0 1M 0 part
|-mmcblk0p29 179:29 0 1M 0 part
|-mmcblk0p30 179:30 0 1M 0 part
|-mmcblk0p31 179:31 0 128K 0 part
|-mmcblk0p32 259:0 0 64M 0 part
|-mmcblk0p33 259:1 0 64M 0 part
|-mmcblk0p34 259:2 0 256K 0 part
|-mmcblk0p35 259:3 0 256K 0 part
|-mmcblk0p36 259:4 0 256K 0 part
|-mmcblk0p37 259:5 0 256K 0 part
|-mmcblk0p38 259:6 0 64M 0 part
|-mmcblk0p39 259:7 0 64M 0 part
|-mmcblk0p40 259:8 0 1M 0 part /bt_firmware
|-mmcblk0p41 259:9 0 1M 0 part
|-mmcblk0p42 259:10 0 110M 0 part /firmware
|-mmcblk0p43 259:11 0 110M 0 part
|-mmcblk0p44 259:12 0 16M 0 part /dsp
|-mmcblk0p45 259:13 0 16M 0 part
|-mmcblk0p46 259:14 0 4M 0 part
|-mmcblk0p47 259:15 0 4M 0 part
|-mmcblk0p48 259:16 0 32M 0 part
|-mmcblk0p49 259:17 0 32M 0 part
|-mmcblk0p50 259:18 0 1M 0 part
|-mmcblk0p51 259:19 0 4K 0 part
|-mmcblk0p52 259:20 0 256K 0 part
|-mmcblk0p53 259:21 0 256K 0 part
|-mmcblk0p54 259:22 0 4K 0 part
|-mmcblk0p55 259:23 0 32.7M 0 part
|-mmcblk0p56 259:24 0 4K 0 part
|-mmcblk0p57 259:25 0 1M 0 part
|-mmcblk0p58 259:26 0 8M 0 part
|-mmcblk0p59 259:27 0 1M 0 part
|-mmcblk0p60 259:28 0 16K 0 part
|-mmcblk0p61 259:29 0 8K 0 part
|-mmcblk0p62 259:30 0 512K 0 part
|-mmcblk0p63 259:31 0 2M 0 part
|-mmcblk0p64 259:32 0 1M 0 part
|-mmcblk0p65 259:33 0 64M 0 part
|-mmcblk0p66 259:34 0 64M 0 part
|-mmcblk0p67 259:35 0 512K 0 part
|-mmcblk0p68 259:36 0 450M 0 part /odm
|-mmcblk0p69 259:37 0 450M 0 part
|-mmcblk0p70 259:38 0 850M 0 part
|-mmcblk0p71 259:39 0 850M 0 part /opt
|-mmcblk0p72 259:40 0 1M 0 part
|-mmcblk0p73 259:41 0 16M 0 part
|-mmcblk0p74 259:42 0 32M 0 part
|-mmcblk0p75 259:43 0 20M 0 part
|-mmcblk0p76 259:44 0 20G 0 part
| |-sailfish-root 252:0 0 2.5G 0 lvm /
| `-sailfish-home 252:1 0 17.6G 0 lvm /home
|-mmcblk0p77 259:45 0 20M 0 part
|-mmcblk0p78 259:46 0 2.8G 0 part
`-mmcblk0p79 259:47 0 2.8G 0 part Here you can clearly see that the /opt is in its own 850 partition. (That's where the vendor's android installation would have went on a normal Android installation):
[nemo@Sailfish ~]$ devel-su blkid /dev/-mmcblk0p7[01]
Password:
/dev/mmcblk0p70: LABEL="vendor" UUID="8d792c1b-3680-44ad-b07a-c6e73c3c0bde" TYPE="ext4" PARTLABEL="vendor_a" PARTUUID="af51f270-1648-4538-87bd-c3e7e707cd28"
/dev/mmcblk0p71: LABEL="vendor" UUID="8d792c1b-3680-44ad-b07a-c6e73c3c0bde" TYPE="ext4" PARTLABEL="vendor_b" PARTUUID="88ba9aad-b155-4e71-b665-a01ce5d4ddb3"What do you intend to address with the term "/opt partition"? Why are the extant checks for space on the root volume (which include an /opt directory there) not sufficient?
If you have both a backup of the original system.img by Jolla and own patch file, the remaining space left isn't enough to unpack and update aliendalvik-system-10.0.0.58.2.1-1.5.1.jolla.armv7hl :
[nemo@Sailfish ~]$ ls -l /opt/alien/
total 690164
-rw-r--r-- 1 root root 4048 Oct 7 23:21 build.prop
lrwxrwxrwx 1 root root 17 Nov 14 12:23 system.img -> system.img.mapsv1
-rw-r--r-- 1 root root 350945280 Nov 14 12:21 system.img.mapsv1
-rw-r--r-- 1 root root 355770368 Oct 7 23:21 system.img.orig
[nemo@Sailfish ~]$ df /opt/
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mmcblk0p71 840320 691052 124644 85% /optyou advise against patching system.img, because deploying and using microG works fine without it (and with a GUI-only setup process) on all officially supported devices since the Xperia XA2 series (i.e., all Xperias except the Xperia X), correct?
Yes, indeed. Patching system.img is only required for corner cases (mapsv1 API). Still, some people do it (me), and get regularily bitten in the ass by forgetting to free that space before starting the upgrade. The official Sailfish OS doesn't check for space in /opt , it too only checks for space in /.
Source of this hearsay? On which devices and SailfishOS releases?
Last time I checked, getting the real-deal Google Play Service require having files in the Android's /system directory.
On Sailfish OS devices whose aliendalvik supports AOSP 8 and later, this directory is inside the system.img image.
Alos, the Google Play Service users where the first to not the "boken SELinux XATTR" bug that poses problems when decompressing and recompressing the image (and which is why my script instead uses some loopback and bind mounting instead of decompressing). I guess that these means they also still need to patch it.
Do you think that is really relevant (because SailfishOS is rather used to evade Google's software)?
Most of the users try indeed to stay as much within non-Google software as possible. Still some people are confronted with Apps that they absolutely need to run. (e.g.: in my case, my friends keep insisting on using WhatsApp, and Meta keeps insisting on banning users that don't use the official app). Some users' "that one app that the must run" might refuse to run on anything but the official Google Pay Service API. That was the case of some banking app and other such critical apps. (Though the recent signing of the AOSP image by Jolla has aleviated some of these problems).
IMO backups shall not reside on system partitions, right?
Depends on the use case. In my case I like having the option to switch back to the original file with a simple symlink.
(Though I haven't needed that).
Also for people not keeping the backup, there should be a risk that the first Sailfish upgrade decides to keep the old file as system.img.rpmsave because the content was changed, at which point you HAVE a risk of having no enough free space.
I am critically questioning this on a technical level (which I fail to fully understand, yet) as well as I am trying to assess how relevant this is.
My reasoning goes:
/, /home, etc./opt is another partition that is affected by upgrade where the the space might be affected.
But you're right that it might be too much of a corner case, for you to handle in your limited volunteered time.
I might write a pull request with the upgrade at some time in the future (Next SFOS upgrade ?)
Can you please provide a pull request or at least some shell code (in this thread here) to provide an example what you are thinking of.
For the general idea, I hope you got it with the example above. I'll try writing a PR with the actual shell code when time permits (around the next SFOS upgrade, probably).
Olf0
commented
2 years ago For the general idea, I hope you got it with the example above.
Yes, thank you very much. This was very helpful to comprehend your issue.
I'll try writing a PR with the actual shell code when time permits (around the next SFOS upgrade, probably).
Never mind: As you pointed out, that is easy.
/opt is on a separate partition.(Or did you consider a different approach?)
What I do need help with, is to assess, what exactly to check for. Currently I am considering to check for at least 400 MBytes free space, because the size of a system image is ≥ 350 MBytes. Opinions, other ideas?
P.S.: Have pleasant Christmas holidays, and please mind (you sure know that longer and better than I do): This pandemic will not go away anytime soon, so after 1,5 years and more to come there is no reason to overly stress oneself. IMO there is nothing to be achieved quickly. :wink:
Olf0
commented
2 years ago I created commit 4e44e4c, but I am not convinced that this is the right thing to do!
system.img file in /opt, that may just be a reason to patch it (see next point).Your technical point was:
Also for people not keeping the backup, there should be a risk that the first Sailfish upgrade decides to keep the old file as system.img.rpmsave because the content was changed, at which point you HAVE a risk of having no enough free space.
But that is not true, please take a look at the output of rpm -ql aliendalvik-system, which lists /opt/alien/system.img as the only file in that RPM, and rpm -qc aliendalvik-system, which shows that no file in that RPM is defined as a "config file". Thus Jolla does simply overwrites the extant system.img in /opt/alien when updating it, regardless if its content was altered.
Consequently when upgrading SailfishOS there is no need for more than a few 10 MBytes free space on an /opt partition the way Jolla has designed this.
Only if one insists in keeping backups of the system.img file in /opt, space there may become scarce.
Thus, despite having developed a "solution" in the branch Olf0-check-opt, I still lack to see the problem it solves.
As the system.img file does not contain any privacy relevant data (regardless if patched or not), one can backup the original and the patched version anywhere, e.g. on (unencrypted) SD-card or "in the cloud". But most appropriate places (somewhere under /home, on an encrypted SD-card or per scp on one's PC) even provide confidentiality and tamper resistance.
To use an arbitrarily named symbolic link (e.g., system.img.patched) as a reminder which points to a patched system.img is fine (vice versa your scheme), but to keep a backup of this huge file in /opt is structurally wrong IMO: There should only be one system.img file in /opt/alien/, which should be named system.img. Both, the original and the patched system.img files shall be backed up at one of the aforementioned appropriate places.
If you still want to pursue this, please provide a scheme which ensures that regular users (i.e., those without multiple system.img files in /opt/alien/) are not spammed with senseless warnings when using sfos-upgrade (because for them a few 10 MBytes free space on an /opt partition is sufficient for upgrading SailfishOS successfully), preferably as a Merge Request against the Olf0-check-opt branch.
Olf0
commented
2 years ago @DrYak: Ping.
Olf0
commented
2 years ago Closing due to lack of feedback. Will reopen this issue, if these points are addressed convincingly, specifically the last paragraph.
DrYak
commented
2 years ago Consequently when upgrading SailfishOS there is no need for more than a few 10 MBytes free space on an /opt partition the way Jolla has designed this.
That's not my own experience. AFAIK (but I am not certain, I haven't check the code) RPM doesn't overwrite files in-place (perhaps in order to be able to abort installation?) so even with way more than 10MiB (close to 128MiB in my case) the aliendalvik-system package fails to install.
Also, for the sake of the finer details:
As the system.img file does not contain any privacy relevant data (regardless if patched or not), one can backup the original and the patched version anywhere, e.g. on (unencrypted) SD-card or "in the cloud". But most appropriate places (somewhere under /home, on an encrypted SD-card or per scp on one's PC) even provide confidentiality and tamper resistance.
My logic of keeping the backup on the same partition is for the ease switching while on the move, to not depend on any external device (so no cloud, no PC), while also avoiding the extremely limited space on /home (as currently aliendalvik will always use the space in /home for its storage, it can't offload to an SD card like a real Android device does).
SD card is indeed a viable alternative as you point out (though at the cost of some transfer speed) - at some point I even tested my patching script to check if it bahaves correctly when run from an SD card on the phone (though it's not my own typical use pattern, so I haven't checked recently if it still works).
Your non-technical arguments are trying to counter the "Jolla users want a Google-free system" assumption.
It's an assumption. While it is certainly true for you (I suppose you wouldn't touch anything like this even with a 10 foot pole) and me (I try as much as possible to reduce this kind of closed source crap, though keeping balanced with some practical needs), that definitely not the case for all users of Sailfish, see the various questions that pop-up from people wanting to run apps that rely on Google Play Service.
even more so people who insist in using software from Facebook (now "Meta"), e.g., Whatsapp, the Facebook app, the Twitter app etc.!
(Now I am completely veering of the initial point)
it's a bit more complex because the choice of the platform isn't 100% yours, but also depends on your circle of friends. And then the "network effect" is strong, with those who are completely into running closed smartphones and apps from psychopath companies "pulling in" the rest of their friends. At some point you risk losing contact with friends, specially if you're not that good at keeping contact, and said friends are too lazy to keep in touch with whom they consider "that weird friends who insist on not using a 'normal phone' like everybody else". I've shut myself off from friends as an accidental consequence of not using Facebook anymore.
On the other hand newer networks like Signal and Twitter (that aren't on a crusade to ban unauthorized 3rd party application) are helping a lot.
Still Jolla is eliminating hurdles for banking apps etc. step-by-step, and with microG and Jolla's system signing most apps should work.
Yup, in my opinion, this is the best path for the future, an alternative with true freedom, rather than the "going for a system that tries not be Google, but then putting back the Google in, because you want the apps". And I do appreciate the progress (my banking apps have stopped complaining about running on a rooted phone sign the signing!)
(End of tangeant about freedom, let's go back to the main discussion).
More importantly all this is no reason to keep a backup of the system.img file in /opt, that may just be a reason to patch it (see next point).
In the end, I think that my request is too much of a corner case that won't represent the typical user. I am not sure how many of them are really affected. Given the lack of other people jumping into this discussion, it seems to me that there isn't a big overlap between "people who want a patched system.img with google" and "people who run sfos-upgrade". The later being probably more technically minded and using solutions (microG) which don't normally require patching (except of extreme corner cases - apps require mapsv1 are a dying breed). While the former are not really digging into the details (they just want the Google Play Store to work for their favorite game or app to work, while not appreciating the irony of what they're trying to achieve). The complains from users who don't understand while they are still running the older alien-dalvik after upgrading sailfish that pop on forums (to me this rings like a aliendalvik-system package failing to install) could support that those users just patch their stuff and don't give much thoughs into that and don't seem to look for more advanced upgrading solution.
So yeah, I think you're right at closing the issue. Probably isn't worth the effort in maintaining, and the alienation that the false alerts will bring just for very few corner cases. (And I can try to remember checking my space in /opt)
Olf0
commented
2 years ago So yeah, I think you're right at closing the issue. Probably isn't worth the effort in maintaining, and the alienation that the false alerts will bring just for very few corner cases.
Thank you for your understanding. Specifically the "false alerts" for many, which may help very few, are my main concern.
Use case:
So if a user keeps both a backup of system.img and the patched one, the /opt partition doesn't have enough room left for an updated system.img and the upgrade fails.
Checking that the partition usage is <60% should do the trick