Big endian issues (+ Snow Leopard)

[eboasson]

This makes Cyclone build and run its test suite on big-endian machines again. This is based on running the tests on Linux on a 64-bit power architecture machine. The Snow Leopard build fixes were taken from #2020 as agreed.

Fixes #1958 Fixes #2019 Fixes #2020

@barracuda156 perhaps you can give it a try?

[barracuda156]

@eboasson Sure, I will run the build now and let you know.

[barracuda156]

@eboasson If I run it from f23e45b079eb80fdd66b175068317bcc5fdb9b3e commit, it fails here:

[ 15%] Building C object src/ddsrt/CMakeFiles/ddsrt-internal.dir/src/environ/posix/environ.c.o
cd /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_cyclonedds/cyclonedds/work/build/src/ddsrt && /opt/local/bin/gcc-mp-13  -I/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_cyclonedds/cyclonedds/work/build/src/ddsrt/ddsrt-internal/include -I/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_cyclonedds/cyclonedds/work/cyclonedds-f23e45b079eb80fdd66b175068317bcc5fdb9b3e/src/ddsrt/include -I/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_cyclonedds/cyclonedds/work/cyclonedds-f23e45b079eb80fdd66b175068317bcc5fdb9b3e/src/ddsrt/src -I/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_cyclonedds/cyclonedds/work/build/src/ddsrt/include -pipe -Os -DNDEBUG -isystem/opt/local/include/LegacySupport -I/opt/local/include -std=gnu99 -arch ppc -mmacosx-version-min=10.6 -Wall -Wextra -Wconversion -Wmissing-prototypes -MD -MT src/ddsrt/CMakeFiles/ddsrt-internal.dir/src/environ/posix/environ.c.o -MF CMakeFiles/ddsrt-internal.dir/src/environ/posix/environ.c.o.d -o CMakeFiles/ddsrt-internal.dir/src/environ/posix/environ.c.o -c /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_cyclonedds/cyclonedds/work/cyclonedds-f23e45b079eb80fdd66b175068317bcc5fdb9b3e/src/ddsrt/src/environ/posix/environ.c
/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_cyclonedds/cyclonedds/work/cyclonedds-f23e45b079eb80fdd66b175068317bcc5fdb9b3e/src/core/ddsi/src/ddsi_udp.c: In function 'ddsi_udp_conn_read':
/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_cyclonedds/cyclonedds/work/cyclonedds-f23e45b079eb80fdd66b175068317bcc5fdb9b3e/src/core/ddsi/src/ddsi_udp.c:138:23: error: field 'ip4' has incomplete type
  138 |     struct in_pktinfo ip4;
      |                       ^~~
/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_cyclonedds/cyclonedds/work/cyclonedds-f23e45b079eb80fdd66b175068317bcc5fdb9b3e/src/core/ddsi/src/ddsi_udp.c: In function 'joinleave_ssm_mcgroup':
/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_cyclonedds/cyclonedds/work/cyclonedds-f23e45b079eb80fdd66b175068317bcc5fdb9b3e/src/core/ddsi/src/ddsi_udp.c:768:29: error: storage size of 'gsr' isn't known
  768 |     struct group_source_req gsr;
      |                             ^~~
/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_cyclonedds/cyclonedds/work/cyclonedds-f23e45b079eb80fdd66b175068317bcc5fdb9b3e/src/core/ddsi/src/ddsi_udp.c:773:57: error: 'MCAST_JOIN_SOURCE_GROUP' undeclared (first use in this function)
  773 |     rc = ddsrt_setsockopt (socket, IPPROTO_IPV6, join ? MCAST_JOIN_SOURCE_GROUP : MCAST_LEAVE_SOURCE_GROUP, &gsr, sizeof (gsr));
      |                                                         ^~~~~~~~~~~~~~~~~~~~~~~
/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_cyclonedds/cyclonedds/work/cyclonedds-f23e45b079eb80fdd66b175068317bcc5fdb9b3e/src/core/ddsi/src/ddsi_udp.c:773:57: note: each undeclared identifier is reported only once for each function it appears in
[ 15%] Building C object src/ddsrt/CMakeFiles/ddsrt-internal.dir/src/filesystem/posix/filesystem.c.o
/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_cyclonedds/cyclonedds/work/cyclonedds-f23e45b079eb80fdd66b175068317bcc5fdb9b3e/src/core/ddsi/src/ddsi_udp.c:773:83: error: 'MCAST_LEAVE_SOURCE_GROUP' undeclared (first use in this function)
  773 |     rc = ddsrt_setsockopt (socket, IPPROTO_IPV6, join ? MCAST_JOIN_SOURCE_GROUP : MCAST_LEAVE_SOURCE_GROUP, &gsr, sizeof (gsr));
      |                                                                                   ^~~~~~~~~~~~~~~~~~~~~~~~
/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_cyclonedds/cyclonedds/work/cyclonedds-f23e45b079eb80fdd66b175068317bcc5fdb9b3e/src/core/ddsi/src/ddsi_udp.c:768:29: warning: unused variable 'gsr' [-Wunused-variable]
  768 |     struct group_source_req gsr;
      |                             ^~~
/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_cyclonedds/cyclonedds/work/cyclonedds-f23e45b079eb80fdd66b175068317bcc5fdb9b3e/src/core/ddsi/src/ddsi_udp.c:778:27: error: storage size of 'mreq' isn't known
  778 |     struct ip_mreq_source mreq;
      |                           ^~~~
cd /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_cyclonedds/cyclonedds/work/build/src/ddsrt && /opt/local/bin/gcc-mp-13  -I/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_cyclonedds/cyclonedds/work/build/src/ddsrt/ddsrt-internal/include -I/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_cyclonedds/cyclonedds/work/cyclonedds-f23e45b079eb80fdd66b175068317bcc5fdb9b3e/src/ddsrt/include -I/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_cyclonedds/cyclonedds/work/cyclonedds-f23e45b079eb80fdd66b175068317bcc5fdb9b3e/src/ddsrt/src -I/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_cyclonedds/cyclonedds/work/build/src/ddsrt/include -pipe -Os -DNDEBUG -isystem/opt/local/include/LegacySupport -I/opt/local/include -std=gnu99 -arch ppc -mmacosx-version-min=10.6 -Wall -Wextra -Wconversion -Wmissing-prototypes -MD -MT src/ddsrt/CMakeFiles/ddsrt-internal.dir/src/filesystem/posix/filesystem.c.o -MF CMakeFiles/ddsrt-internal.dir/src/filesystem/posix/filesystem.c.o.d -o CMakeFiles/ddsrt-internal.dir/src/filesystem/posix/filesystem.c.o -c /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_cyclonedds/cyclonedds/work/cyclonedds-f23e45b079eb80fdd66b175068317bcc5fdb9b3e/src/ddsrt/src/filesystem/posix/filesystem.c
/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_cyclonedds/cyclonedds/work/cyclonedds-f23e45b079eb80fdd66b175068317bcc5fdb9b3e/src/core/ddsi/src/ddsi_udp.c:786:55: error: 'IP_ADD_SOURCE_MEMBERSHIP' undeclared (first use in this function); did you mean 'IP_ADD_MEMBERSHIP'?
  786 |     rc = ddsrt_setsockopt (socket, IPPROTO_IP, join ? IP_ADD_SOURCE_MEMBERSHIP : IP_DROP_SOURCE_MEMBERSHIP, &mreq, sizeof (mreq));
      |                                                       ^~~~~~~~~~~~~~~~~~~~~~~~
      |                                                       IP_ADD_MEMBERSHIP
[ 16%] Building C object src/ddsrt/CMakeFiles/ddsrt-internal.dir/src/process/posix/process.c.o
cd /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_cyclonedds/cyclonedds/work/build/src/ddsrt && /opt/local/bin/gcc-mp-13  -I/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_cyclonedds/cyclonedds/work/build/src/ddsrt/ddsrt-internal/include -I/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_cyclonedds/cyclonedds/work/cyclonedds-f23e45b079eb80fdd66b175068317bcc5fdb9b3e/src/ddsrt/include -I/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_cyclonedds/cyclonedds/work/cyclonedds-f23e45b079eb80fdd66b175068317bcc5fdb9b3e/src/ddsrt/src -I/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_cyclonedds/cyclonedds/work/build/src/ddsrt/include -pipe -Os -DNDEBUG -isystem/opt/local/include/LegacySupport -I/opt/local/include -std=gnu99 -arch ppc -mmacosx-version-min=10.6 -Wall -Wextra -Wconversion -Wmissing-prototypes -MD -MT src/ddsrt/CMakeFiles/ddsrt-internal.dir/src/process/posix/process.c.o -MF CMakeFiles/ddsrt-internal.dir/src/process/posix/process.c.o.d -o CMakeFiles/ddsrt-internal.dir/src/process/posix/process.c.o -c /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_cyclonedds/cyclonedds/work/cyclonedds-f23e45b079eb80fdd66b175068317bcc5fdb9b3e/src/ddsrt/src/process/posix/process.c
/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_cyclonedds/cyclonedds/work/cyclonedds-f23e45b079eb80fdd66b175068317bcc5fdb9b3e/src/core/ddsi/src/ddsi_udp.c:786:82: error: 'IP_DROP_SOURCE_MEMBERSHIP' undeclared (first use in this function); did you mean 'IP_DROP_MEMBERSHIP'?
  786 |     rc = ddsrt_setsockopt (socket, IPPROTO_IP, join ? IP_ADD_SOURCE_MEMBERSHIP : IP_DROP_SOURCE_MEMBERSHIP, &mreq, sizeof (mreq));
      |                                                                                  ^~~~~~~~~~~~~~~~~~~~~~~~~
      |                                                                                  IP_DROP_MEMBERSHIP
/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_cyclonedds/cyclonedds/work/cyclonedds-f23e45b079eb80fdd66b175068317bcc5fdb9b3e/src/core/ddsi/src/ddsi_udp.c:778:27: warning: unused variable 'mreq' [-Wunused-variable]
  778 |     struct ip_mreq_source mreq;
      |                           ^~~~
[ 16%] Building C object src/ddsrt/CMakeFiles/ddsrt-internal.dir/src/random/posix/random.c.o
make[2]: *** [src/core/CMakeFiles/ddsc.dir/ddsi/src/ddsi_udp.c.o] Error 1
make[2]: *** Waiting for unfinished jobs....

(This is not directly comparable with the earlier successful build, since that was of 0.10.5 and not from the master.)

Passing -DENABLE_IPV6=0 -DENABLE_SOURCE_SPECIFIC_MULTICAST=0 still leaves it with the error:

/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_cyclonedds/cyclonedds/work/cyclonedds-f23e45b079eb80fdd66b175068317bcc5fdb9b3e/src/core/ddsi/src/ddsi_udp.c: In function 'ddsi_udp_conn_read':
/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_cyclonedds/cyclonedds/work/cyclonedds-f23e45b079eb80fdd66b175068317bcc5fdb9b3e/src/core/ddsi/src/ddsi_udp.c:138:23: error: field 'ip4' has incomplete type
  138 |     struct in_pktinfo ip4;
      |                       ^~~

But this looks rather like an SDK issue, not endianness.

[eboasson]

Right, I should try to effectively do -DENABLE_IPV6=0 -DENABLE_SOURCE_SPECIFIC_MULTICAST=0 automatically, it is just one more check using the availability macros.

/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_cyclonedds/cyclonedds/work/cyclonedds-f23e45b079eb80fdd66b175068317bcc5fdb9b3e/src/core/ddsi/src/ddsi_udp.c: In function 'ddsi_udp_conn_read':
/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_cyclonedds/cyclonedds/work/cyclonedds-f23e45b079eb80fdd66b175068317bcc5fdb9b3e/src/core/ddsi/src/ddsi_udp.c:138:23: error: field 'ip4' has incomplete type
  138 |     struct in_pktinfo ip4;
      |                       ^~~

We can live without that one, I'll have a look on how can I most easily avoid needing it.

[barracuda156]

In principle IPV6 was supported from 10.5, so it should work, but perhaps not everything needed is available in the SDK or, possibly, is available under a different name. But this is not something I know well myself.

[eboasson]

@barracuda156 I just pushed a few updates. It now tries to auto-detect IPv6 and source-specific multicast in CMake and adapt accordingly (I won't do any checking if it is disabled by the user, so you might want to throw away CMakeCache.txt to check).

I tried to change the IP_PKTINFO related code to avoid it on snow leopard. Not sure if it'll work.

Perhaps you can give it another go?

@dpotman Thanks for catching those two things 😀 I feel really stupid for fixing the thing mentioned in the FIXME and then not removing the FIXME itself ... 🤦🤣

[eboasson]

@barracuda156 I'm going to merge this as it does fix the big-endian issues (at least in my testing). Hopefully the latest changes make it work out-of-the-box on Snow Leopard as well. If not, please just create a new issue. Thanks!

[barracuda156]

@eboasson Thank you! I will try it tonight and let you know.

[barracuda156]

@eboasson Yes, from 1deaa93b2019e6e23cb34d73594fc25327d6966c it builds without any patches and configure args. Thank you!

[barracuda156]

@eboasson BTW, could you make a new release? I generated a patch, diffing master and 0.10.5, and it is 200k+ LoC and 9.5 MB, that won't be accepted :)