Allow SyncAPFS to work with multiple macOS system volumes in the same container

[MarioG-X]

Is your feature request related to a specific problem? It's not practical to put macOSs in their own container and own partition with many partitions, I may have up to 10 macOSs on a test machine, and several Linux's. On top of that, I have an HD and SSD with 3 containers, need to write down all the Preboot locations on paper, or boot the wrong one often since they change location often on a test machine. APFS makes life easy.

Describe the solution you would like Allow RP to use SyncAPFS for multiple macOS boot volumes in the same container.

Describe alternatives you have considered There are none currently.

Additional context None

[dakanji]

Please try X175 X175-BOOTx64.efi.zip

Note that this is also a narrow focus build for this specific item. Main thing you will notice is that it will work in text mode. You should still be able to identify whether the volumes are shown as required.

[MarioG-X]

The screen below has a few issues:

1. The first Preboot should be "Monterey Beta".
2. The second Preboot should be "FS 11.6". I booted this entry fine into FS 11.6.
3. I think there should be recovery entries for "Monterey Beta" and "FS 11.6". They are in different containers.

Log booting into the second Preboot entry: 21r21u5948.log

[dakanji]

Thanks. Seems Apple changed something in later versions. X176 should catch more. X176-BOOTx64.efi.zip

[MarioG-X]

Getting better but the screen below has 3 issues: 21r22a0433.log

1. The "Monterey Beta" system is missing, could be the first Preboot should be "Monterey Beta".
2. What was the second Preboot now shows as a blank line (see first pic), then as soon as you move the cursor it changes to "FS 11.6" (see second pic). I booted this entry fine into macOS on FS 11.6.
3. I think there should be recovery entries for "Monterey Beta" and "FS 11.6". They are in different containers.

[MarioG-X]

When I selected Preboot, RP showed some kind of error screen, I pressed a key and it went back to the menu but froze, had to Power off/on. Here is the log to that one: 21r22a1744.log I see the end log shows something about 10.15.7 but I did select "Preboot" which is the first macOS displayed.

Sorry to take so long, will do better to respond going forward.

[dakanji]

Thanks for the feedback

The "Monterey Beta" system is missing, could be the first Preboot should be "Monterey Beta".

• The first PreBoot is not as you found out.
• Monterey Beta is misnamed as the first "10.14.6". Don't know why as yet.

What was the second Preboot now shows as a blank line

• That is just a glitch in the text mode screen where the default item selected on screen load does not display until the selection is moved. Still trying to figure that out. If you go into 'About RefindPlus' and come back, you will see that effect on the selection. Doesn't affect graphics mode.

I think there should be recovery entries for "Monterey Beta" and "FS 11.6".

• RP has found and listed the recovery options available.
• You will note that the ones for versions up to 10.15 are shown individually but those for 11.6 and Monterey seem to have been combined.
• BTW, the code that was added to identify the volumes in the multi-volume container scenario was leveraged to identify the recovery instances (previous default is for all to have a generic name such as the first one) but something appears to have changed in 11.6 and Monterey in how this works and the recovery entry cannot be mapped to a Mac OS instance for those versions in the way that works for the others and I think, up to 11.5.

Back to the first PreBoot entry, this is a new instance (presumably added after a recent update) that behaves differently to the others. Not sure if it is an artefact of the update process but it also, like the recovery item discussed earlier, cannot be mapped to a specific system volume. It is not pointing to any volume visible to RP. I can change things to just not show it though. Needs testing on other installations.

I am half inclined to ditch the effort and go back to the limited scope but more resilient implementation (within its scope). Can perhaps take another look at this when what looks like ongoing changes to APFS by Apple settle down ... possibly sometime after Monterey comes out.

Can you run rEFInd to check whether a new PreBoot items shows up though? Also check how many recovery items it finds.

Will make a call on things later.

[MarioG-X]

Answers and info:

Yes there were updates but the diskutil list and original refind displays the same as before:

1. Updated 11.5 with security updates to 11.6, and renamed volume from 11.5 to 11.6.
2. Updated 10.15.7 with Apple security update.
3. Had created a backup system volume from 11.6 to disk3 container, then deleted it after test, diskutil shows no change.

Here is a diskutil list, you will find that it is identical to the one in the previous issue I sent except 11.5 is now renamed 11.6: duskutil2.list.txt

Original refind can boot all Preboot partitions. Here is the original refind log: refind.log

Below is a screen print of original refind. It is also identical to the one before the previously mentioned changes (original screen in previous issue). I tested booting from all Preboots and they work. Notice that original refind shows 4 recovery partitions with no name. RP shows the first as "Recovery" but the next three properly named, I did not test these.

The sequence of the macOSs in original refind are:

1. FX 10.15.7 Xcode
2. Preboot (boots Monterey)
3. Monterey Beta
4. Preboot (boots FX 10.15.7 Xcode)
5. Preboot (boots FS 10.15)
6. Preboot (boots FS 11.6)
7. FS 10.14.6 (not bootable from this entry)
8. FS 10.15.7 (not bootable from this entry)
9. FS 11.6 (not bootable from this entry)

[dakanji]

Thanks

Yes there were updates but the diskutil list and rEFInd displays the same as before:

The changes are under the hood and Apple's diskutil will have been adjusted to display the same final output while rEFInd was never doing anything special with APFS so yes, no change expected in either.

Here is a diskutil list, you will find that it is identical to the one in the previous issue

yeah ... expected.

Notice that rEFInd shows 4 recovery partitions with no name.

Confirms there is no longer a 1 to 1 pattern of System and Recovery volume. This appears to be something new with the recent update.

Anyway, please try X177 which fixes a bug spotted in the last one and will not show that first PreBoot if still present. X177-BOOTx64.efi.zip

[MarioG-X]

The good news: Monterey Beta and Big Sur 11.6 display and booted! Although not showing the system volume names. Log: 21r22r0216.log

The issues now:

1. The default still shows as a blank line (see first pic), then as soon as you move the cursor it changes to "FS 11.6" (see second pic). I booted this entry fine into macOS on "FS 11 5 - Data".
2. The volume name for "Monterey Beta" displays the data volume name "Monterey Beta - Data" instead of the system volume name.
3. The volume name for "FS 11.6" displays the data volume name "FS 11 5 - Data" instead of "FS 11.6", the actual system volume. "FS 11.6" was displayed in 176.

Before Pressing a key:

After pressing key:

[dakanji]

Brilliant. Working as expected.

APFS differs between Mac OS versions and continues to evolve. While mixing versions in a container may not yet have led to issues, this is almost certainly inevitable.

A change was made with Mac OS 11.6 and the PreBoot volume now points to the data volume and not the system volume.

Anyway, just need to test on some other setups

[MarioG-X]

Great! When you get closer to final, I will test boot all partitions as well as some on one of my externals which have several bootable partitions. BTW, I did test 176 booting an external and it found all volumes.

[MarioG-X]

Does that mean you want a final test from me? If so, what should I download? Thanks.

[dakanji]

Not strictly necessary but if you do want to test it, that was a push of the code which can now be built.

[MarioG-X]

Man, dakanji you rock! I rebuilt GOP and tested booting all volumes, you will be pleased to know that testing was all good!

Only minor issue is the " Data" volume is displayed for macOS 11 and 12, not a biggie since it can be renamed. Could be related to macOS "sealed" system volumes. Still WAY better than hunt and peck Preboots!

Tested Linux systems: 1 OpenSuse 15.3 BTRFS 2 OpenSuse 15.3 - 2 BTRFS 3 Elementary OS EXT4 4 Elementary OS - 2 EXT4

Tested macOS (all APFS): 5 FX 10.15.7 Xcode 6 Monterey Beta - Data 7 FS 10.14.6 8 FS 10.15.7 9 FS 11 6 - Data

Successfully booted macOS 10.14.6, 10.15.7 and 11.6 from an external drive

A log if desired: 21r23x3124.log

Boot screen, no external connected:

[dakanji]

not a biggie since it can be renamed

See: https://github.com/dakanji/RefindPlus/blob/d87d3fb3b46787440c6dace431b1a6a3517161a9/config.conf-sample#L41-L44

[MarioG-X]

I just thought that when the Mac is booted with the option key and it shows "FS 11.6", Monterey Beta" and not the data volumes it may be easy to implement. I'm pleased as punch it works with - Data so no biggie. BTW, I tested booting 10.15.7 and 11.6 from an external and edited notes above.

[dakanji]

The " - Data" string will be filtered out later for display. Obviously can't cover every language as it differs by localisation but set up for English and German as a start.

[MarioG-X]

May I suggest to leave it as is rather than simply removing the "- Data" because the names may still be different. For instance, my 11.6 volumes were "FS 11.6" (because I renamed from 11.5 after the update), and the attached volume stayed "FS 11 5 - Data". Just stripping "- Data" would result in "FS 11 5" which may be confusing, especially if there was a real 11.5 and 11.6 on the system. I would prefer leaving it as is if the correct system volume name can't be obtained.

BTW, I had backup scripts used to strip "- Data" and it caused nothing but headaches. I don't do that anymore.

[dakanji]

That seems to be more of a problem with your setup. It is not standard to keep changing the name of an installation volume to match updates.

[MarioG-X]

No problem with my setup, on two machines that I used for heavy testing. macOS allows you to change the system volume name anytime, but the data volume will stay unchanged. This will cause issues, I've seen it on the web many times. No harm in using the real - Data volume name so it's obvious what is booting. Removing "- Data" is asking for issues later on IMHO.

Either way you choose, it's leaps and bounds better than a dozen Preboots!

[dakanji]

I know that Mac OS allows changing volume names. The problem I meant was doing this with every update. Most users probably never change the name once set.

Any way, the filtering will be to try to change the displayed name to the system volume name ... so even your approach should be fine.

[MarioG-X]

I just remembered that at one point (see screen above: https://user-images.githubusercontent.com/20360699/134263704-ebd9e6eb-4dee-4127-8c65-607283faca63.jpeg) the correct system volume name displayed on the RP menu as "FS 11.6", so it is available. That is the name the Mac displays when I use the Option key to boot. You could use that for the menu and the name in Preboot for actual booting.

Edit: I see that the actual system volume names are in the log (under ReMap APFS Volumes: as well as the partition listing) so they are available. From log:

3:065 0:005 ReMap APFS Volumes: 3:070 0:004 - FX 10.15.7 Xcode 3:075 0:005 - Monterey Beta 3:080 0:004 - FS 10.14.6 3:086 0:005 - FS 10.15.7 3:090 0:004 - FS 11.6

[MarioG-X]

Add to above: There is another issue to my previous kludgy technique of simply stripping "- Data" (which I wound up abandoning and solving many issues): When macOS creates the - Data volume, it strips certain characters like periods and others, for example, my volumes are names contain a v.r.m like 11.6, 11.5.1, 12.15.7, etc, If a data volume is created at that level, the periods (and other specula characters if used) are stripped. Even if the data volume is never renamed, it is called "11 6", "11 5 1", etc. Other who may use a special character in the system volume would see a strange name. Using the correct name solves this issue as well as when the system volume name is renamed. You have been a perfectionists so far, I recommend leaving the - Data name in place if there is no way to associate the two volumes. Wish we knew how the Mac firmware does it since it displays the correct name in the option/boot menu.

And again, many thanks for your work!

[dakanji]

You have been a perfectionist

You have me wrong there as I lean heavily to the pragmatist side of things. This means not dropping something that will benefit the 99.9% for the sake of the 0.1%.

In this particular edge case however, I think that by pure chance of the implementation approach, you will end up with the Data Volume name unchanged.

The feature has been completed and pushed. You can build and see what you get but sorry, if it does not end up as I think it will (with the data volume name intact for your case), you will have to reevaluate your volume naming scheme and perhaps consider dropping special characters.

[MarioG-X]

No problem, it's WAY better than a bunch of changing Preboots! Forgive my concerns, having been a mainframe programmer who developed for hospitals and banks for 40 years, only 100 percent perfect in all conditions is acceptable is what I am used to. Many thanks again!

[dakanji]

I am sure that if RP was being developed by a corporation with a team of paid experts, it would also aim for being 100% perfect under any and every conceivable condition.

I'm afraid that in this case however, we have to settle for a target of a mere 100% perfect under all supported conditions and at this point in time, that particular edge case condition is not supported (although as said, I am pretty sure the anticipated issue will not materialise ... for now).

[MarioG-X]

Actually, I would have thought many people using RP are developers who are likely to do exactly what I do with several volumes on a Mac. As for supported, macOS supports it fine, if you mean RP supported conditions that's different and we just have to live with it.

If you're set on not using the actual system volume name, then real Data volume name should be displayed instead of displaying a non existent volume, "FS 11.5 - Data" should not be displayed as "FS 11 5" when there is no such volume, simply display "FS 11 5 - Data", a volume that actually exists.

EDIT: How about making displaying the real - Data volume an option for people who want to see the actual volume name we're booting from?

[dakanji]

Too complicated to explain in detail but at a high level, you are not booting the system or data volume whatever label RP puts there ... you are booting the PreBoot volume.

Anyway, not much point discussing this item further as the actions have been taken and I will like to move on to other items.

Thanks for your assistance though.

EDIT: To clarify, if after running the current code, there is a specific confirmed issue, as opposed to an issue you think might happen, then please report this to be dealt with.

Thanks again.