Need to configure with --enable-host-pie

[simonjwright]

This is related to GCC PR 110467. That PR says that the problem only arises when Ada is enabled, which might explain why it's not been seen already.

I found that you can avoid the problem by giving --enable-host-pie at the top level configure.

[iains]

right, I believe I commented on a gcc-patches thread related to that PR, that the current configuration for Ada seems to force PIE off (which is no longer needed) - but AFAIR there has been no response. So the issue here is that we are now using the upstream PIE build fixes - rather than our local Darwin ones (that also handled Ada).

I need to check back over the threads to see if there's an incremental patch to propose - meanwhile you have a workaround, I see?

[simonjwright]

Yes, the workround works: the acats tests look good,

        === acats Summary ===
# of expected passes        2325
# of unexpected failures    3
*** FAILURES: cb1010a cb1010d  c250002 

which is usual, I think? (the first 2 are stack overflow check failures, the last is to do with filenames containing Latin-1 international characters vs the Darwin filesystem).

[iains]

yes, that looks "normal"

(I have not got as far as looking into those fails; there are others we need to tackle ahead of those). Probably, the stack-checking stuff would turn out to be common code, but not sure yet.

As for character set issues, that test is either skipped or passes on earlier/x86_64 Darwin versions, so we might want to poke a bit at what is going wrong there.

[simonjwright]

As for character set issues, that test is either skipped or passes on earlier/x86_64 Darwin versions, so we might want to poke a bit at what is going wrong there.

The source file in this old (ACATS 2.6) version is c250002.aw, converted to c250002.a; the conversion changes e.g. package C250002_["C1"] is to package C250002_Á is, where the Á is a single Latin-1 character; gnatchop tries to create a .ads file with (the lower-case version of) that name and fails. Maybe HFS+ would have worked?

Interestingly, Terminal (running bash) can create a file with such a name (e.g. via Touch), but the name gets converted to UTF-8 somewhere.

The equivalent test in modern ACATS is in c250002.au, which is UTF-8, but you may remember that there are issues with gnatchop not converting to lower-case properly, and the Mac FS changing the input NFC encoding to NFD (see PR 81114 c1).

[iains]

As for character set issues, that test is either skipped or passes on earlier/x86_64 Darwin versions, so we might want to poke a bit at what is going wrong there.

The source file in this old (ACATS 2.6) version is c250002.aw, converted to c250002.a; the conversion changes e.g. package C250002_["C1"] is to package C250002_Á is, where the Á is a single Latin-1 character; gnatchop tries to create a .ads file with (the lower-case version of) that name and fails. Maybe HFS+ would have worked?

Can you comment on whether the test works properly on an x86_64 box with a) an HFSx filesystem b) an APFS one? Have you tried case sensitive / not CS?

(so, we have all permutations of Arm64 and x86_64 + HFSx and APFS + case sensitive).

I always build OSS on case sensitive partitions, since some make that assumption so my experimental data do not cover much of that space.

[simonjwright]

On 8 Sep 2023, at 20:58, Iain Sandoe @.***> wrote:

As for character set issues, that test is either skipped or passes on earlier/x86_64 Darwin versions, so we might want to poke a bit at what is going wrong there.

The source file in this old (ACATS 2.6) version is c250002.aw, converted to c250002.a; the conversion changes e.g. package C250002_["C1"] is to package C250002_Á is, where the Á is a single Latin-1 character; gnatchop tries to create a .ads file with (the lower-case version of) that name and fails. Maybe HFS+ would have worked?

Can you comment on whether the test works properly on an x86_64 box with a) an HFSx filesystem b) an APFS one? Have you tried case sensitive / not CS?

On an Intel box (MacBook Pro 2012, Catalina) the test works OK on an (external) HFS disk; gnatchop fails on the internal APFS disk and on a ExFAT USB stick. (I didn’t know that a complete reinstall would set up the system disk as APFS, even though it’s an HDD).

Exactly the same on an M1 Air, with the same external HFS disk.

(so, we have all permutations of Arm64 and x86_64 + HFSx and APFS + case sensitive).

I always build OSS on case sensitive partitions, since some make that assumption so my experimental data do not cover much of that space.

I don’t use case-sensitive partitions (well, as previously discussed, GNAT regards filesystems on aarch64 as case-sensitive).

[iains]

On 8 Sep 2023, at 20:58, Iain Sandoe @.***> wrote: As for character set issues, that test is either skipped or passes on earlier/x8664 Darwin versions, so we might want to poke a bit at what is going wrong there. The source file in this old (ACATS 2.6) version is c250002.aw, converted to c250002.a; the conversion changes e.g. package C250002["C1"] is to package C250002_Á is, where the Á is a single Latin-1 character; gnatchop tries to create a .ads file with (the lower-case version of) that name and fails. Maybe HFS+ would have worked? Can you comment on whether the test works properly on an x86_64 box with a) an HFSx filesystem b) an APFS one? Have you tried case sensitive / not CS? On an Intel box (MacBook Pro 2012, Catalina) the test works OK on an (external) HFS disk; gnatchop fails on the internal APFS disk and on a ExFAT USB stick. (I didn’t know that a complete reinstall would set up the system disk as APFS, even though it’s an HDD). Exactly the same on an M1 Air, with the same external HFS disk. (so, we have all permutations of Arm64 and x86_64 + HFSx and APFS + case sensitive). I always build OSS on case sensitive partitions, since some make that assumption so my experimental data do not cover much of that space.

Hmm so it seems that APFS is doing something different - I guess this probably means that we need to update libada to reflect these changes; I do not think Adacore are supporting macOS any more so it will be down to us to make patch and post it.

I don’t use case-sensitive partitions (well, as previously discussed, GNAT regards filesystems on aarch64 as case-sensitive).

There's an environment var that has to be set for CS partitions: GNAT_FILE_NAME_CASE_SENSITIVE=1 Otherwise, AFAIK it will treat macOS / MacOSX FS as case-preserving-case-insensitive (i.e. the default for the install).

BTW: Since the 'rootless' changes there is a subtlety - IFF you make the system partition case censitive, actually the read-only portion [with the immutable part of the macOS install] will remain case insensitive, it is only the partition that is mounted on /System/Data/ that is actually case sensitive (that fooled me at first - since it looked as if the disk utility was ignoring my setting). When you examine the system disk it looks as if it's case insensitive.

My build partitions on macOS 12 are external case sensitive APFS and 25002 fails there too.

[iains]

I don’t use case-sensitive partitions (well, as previously discussed, GNAT regards filesystems on aarch64 as case-sensitive).

Hmm maybe I missed something of this point - are you saying that there's some arch-specific setting that's not accounting for macOS (it would be reasonable for the default to be case sensitive on other *nix). Maybe we are missing an aarch64-darwin OS file somewhere.

[iains]

I don’t use case-sensitive partitions (well, as previously discussed, GNAT regards filesystems on aarch64 as case-sensitive).

Hmm maybe I missed something of this point - are you saying that there's some arch-specific setting that's not accounting for macOS (it would be reasonable for the default to be case sensitive on other *nix). Maybe we are missing an aarch64-darwin OS file somewhere.

NOTE: setting GNAT_FILE_NAME_CASE_SENSITIVE will override the default, and therefore should give correct behaviour if you set it right for your test disk (so that's not the origin of the fault, I suppose) but the default is, indeed, to assume case sensitive - not sure if that's correct for macOS (it might well be for iOS)

      /* By default, we suppose filesystems aren't case sensitive on
         Windows and Darwin (but they are on arm-darwin).  */
#if defined (WINNT) || defined (__DJGPP__) \
  || (defined (__APPLE__) && !(defined (__arm__) || defined (__arm64__)))
      file_names_case_sensitive_cache = 0;
#else
      file_names_case_sensitive_cache = 1;
#endif

[simonjwright]

I just dug out my K&R C. Running this code

#include <stdio.h>

int main() {
  char name[] = "c250002_x.ads";
  name[8] = 0xe1; // Latin 1 lower-case a acute
  printf("name: '%s'\n", name);
  FILE *f = fopen(name, "w");
  if (f) {
    printf("file opened.\n");
    fclose(f);
  } else {
    perror("file not opened");
  }
  return 0;
}

has the following results:

On the ExFat USB stick, file not opened: Invalid argument On AFPS, file not opened: Illegal byte sequence On HFS, success.

So, it looks as though HFS assumes we know what we’re doing (rather like Linux), but APFS checks whether the proposed file name string is legal UTF-8.

This check (c250002) has been updated in ACATS 4 (possibly even in 3) to use utf-8 strings, which is what the ARM now requires (ARM 2.1(16)): but note that GNAT doesn’t properly handle filenames with international characters in case-insensitive filesystems (PR81114, see above), and this is made worse on macOS because of the way that characters’ normalization forms are changed silently on file creation but not on file searching, so that I can create a file and then be unable to open it using the same name.

I’d be inclined to disregard c250002 failures; that would still leave the stack overflow check failures (cb1010a, cb1010d) to be looked at.

[iains]

I just dug out my K&R C.

heh.. you mean you do not have a build of GCC ?

the fail is indeed, as man 2 open says: [EILSEQ] The filename does not match the encoding rules. errno == 92 == EILSEQ

I’d be inclined to disregard c250002 failures;

Hmm have to take a look and see if there's an XFAIL mechanism for acats.

that would still leave the stack overflow check failures (cb1010a, cb1010d) to be looked at.

This is not going to be any time soon - there has just been a big change in the stack checking and AFAIR it's also different from the macOS system version - so not a 5mins fix, by any means.

[simonjwright]

I just dug out my K&R C.

heh.. you mean you do not have a build of GCC ?

Distracted by the 200-point C on the cover

I’d be inclined to disregard c250002 failures;

Hmm have to take a look and see if there's an XFAIL mechanism for acats.

I’d be more inclined to look for a way of marking the test 'unsupported' - or perhaps just not running it on macOS? Is there a simple test for APFS?

[iains]

Hmm have to take a look and see if there's an XFAIL mechanism for acats.

I’d be more inclined to look for a way of marking the test 'unsupported' - or perhaps just not running it on macOS? Is there a simple test for APFS?

(either of these ^^ would do in the short term)

--- in the longer term...

I'm guessing that HFSx does the translation from what's supplied to the UTF-8 that's actually used.

What I'm wondering about is whether the Ada library needs to provide appropriate conversions when the FS is APFS and if we can reasonably easily detect that situation (or perhaps have some default based on OS version, with an env override as is done for case sensitive).