ipv6 support

[grepwood]

Some applications require IPv6 support, like ioquake3. Can we please have ipv6 support in this toolchain?

[zeldin]

PSL1GHT just forwards socket calls to the system network SPRX (part of the PS3 firmware). If IPv6 does not work it is because Sony does not support it, not because ps3dev doesn't.

If you need IPv6, it is supported by PS3 Linux.

[grepwood]

@zeldin That's fine if SPRX does it. Can we at least have some headers? I'll pass you the compile errors when I come back home. Also, building ioquake3 for PS3 Linux is a piece of cake. I wouldn't be filing this ticket if I was targetting Linux :)

[grepwood]

CC code/qcommon/net_ip.c
code/qcommon/net_ip.c:143:26: error: field 'addr' has incomplete type
  struct sockaddr_storage addr;
                          ^~~~
code/qcommon/net_ip.c:144:26: error: field 'netmask' has incomplete type
  struct sockaddr_storage netmask;
                          ^~~~~~~
code/qcommon/net_ip.c: In function 'NetadrToSockadr':
code/qcommon/net_ip.c:226:29: error: dereferencing pointer to incomplete type 'struct sockaddr_in6'
   ((struct sockaddr_in6 *)s)->sin6_family = AF_INET6;
                             ^~
code/qcommon/net_ip.c:227:43: error: dereferencing pointer to incomplete type 'struct in6_addr'
   ((struct sockaddr_in6 *)s)->sin6_addr = * ((struct in6_addr *) &a->ip6);
                                           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
code/qcommon/net_ip.c:234:51: error: invalid use of undefined type 'struct ipv6_mreq'
   ((struct sockaddr_in6 *)s)->sin6_addr = curgroup.ipv6mr_multiaddr;
                                                   ^
code/qcommon/net_ip.c: In function 'Sys_StringToSockaddr':
code/qcommon/net_ip.c:289:11: error: implicit declaration of function 'getaddrinfo' [-Werror=implicit-function-declaration]
  retval = getaddrinfo(s, NULL, hintsp, &res);
           ^~~~~~~~~~~
code/qcommon/net_ip.c:322:4: error: implicit declaration of function 'freeaddrinfo'; did you mean 'readlink'? [-Werror=implicit-function-declaration]
    freeaddrinfo(res);
    ^~~~~~~~~~~~
    readlink
code/qcommon/net_ip.c:330:67: error: implicit declaration of function 'gai_strerror'; did you mean 'strerror'? [-Werror=implicit-function-declaration]
   Com_Printf("Sys_StringToSockaddr: Error resolving %s: %s\n", s, gai_strerror(retval));
                                                                   ^~~~~~~~~~~~
                                                                   strerror
code/qcommon/net_ip.c:330:58: warning: format '%s' expects argument of type 'char *', but argument 3 has type 'int' [-Wformat=]
   Com_Printf("Sys_StringToSockaddr: Error resolving %s: %s\n", s, gai_strerror(retval));
                                                         ~^        ~~~~~~~~~~~~~~~~~~~~
                                                         %d
code/qcommon/net_ip.c: In function 'Sys_SockaddrToString':
code/qcommon/net_ip.c:348:21: error: invalid application of 'sizeof' to incomplete type 'struct sockaddr_in6'
   inputlen = sizeof(struct sockaddr_in6);
                     ^~~~~~
code/qcommon/net_ip.c:352:5: error: implicit declaration of function 'getnameinfo' [-Werror=implicit-function-declaration]
  if(getnameinfo(input, inputlen, dest, destlen, NULL, 0, NI_NUMERICHOST) && destlen > 0)
     ^~~~~~~~~~~
code/qcommon/net_ip.c: In function 'Sys_StringToAdr':
code/qcommon/net_ip.c:362:26: error: storage size of 'sadr' isn't known
  struct sockaddr_storage sadr;
                          ^~~~
code/qcommon/net_ip.c:362:26: warning: unused variable 'sadr' [-Wunused-variable]
code/qcommon/net_ip.c: In function 'NET_AdrToString':
code/qcommon/net_ip.c:468:27: error: storage size of 'sadr' isn't known
   struct sockaddr_storage sadr;
                           ^~~~
code/qcommon/net_ip.c:468:27: warning: unused variable 'sadr' [-Wunused-variable]
code/qcommon/net_ip.c: In function 'NET_GetPacket':
code/qcommon/net_ip.c:528:26: error: storage size of 'from' isn't known
  struct sockaddr_storage from;
                          ^~~~
code/qcommon/net_ip.c:528:26: warning: unused variable 'from' [-Wunused-variable]
code/qcommon/net_ip.c: In function 'Sys_SendPacket':
code/qcommon/net_ip.c:647:26: error: storage size of 'addr' isn't known
  struct sockaddr_storage addr;
                          ^~~~
code/qcommon/net_ip.c:681:81: error: invalid application of 'sizeof' to incomplete type 'struct sockaddr_in6'
    ret = sendto( ip6_socket, data, length, 0, (struct sockaddr *) &addr, sizeof(struct sockaddr_in6) );
                                                                                 ^~~~~~
code/qcommon/net_ip.c:647:26: warning: unused variable 'addr' [-Wunused-variable]
  struct sockaddr_storage addr;
                          ^~~~
code/qcommon/net_ip.c: In function 'NET_IPSocket':
code/qcommon/net_ip.c:90:24: error: implicit declaration of function 'ioctl'; did you mean 'iscntrl'? [-Werror=implicit-function-declaration]
 # define ioctlsocket   ioctl
                        ^
code/qcommon/net_ip.c:836:6: note: in expansion of macro 'ioctlsocket'
  if( ioctlsocket( newsocket, FIONBIO, &_true ) == SOCKET_ERROR ) {
      ^~~~~~~~~~~
code/qcommon/net_ip.c:836:30: error: 'FIONBIO' undeclared (first use in this function); did you mean 'SO_NBIO'?
  if( ioctlsocket( newsocket, FIONBIO, &_true ) == SOCKET_ERROR ) {
                              ^~~~~~~
                              SO_NBIO
code/qcommon/net_ip.c:836:30: note: each undeclared identifier is reported only once for each function it appears in
code/qcommon/net_ip.c: In function 'NET_IP6Socket':
code/qcommon/net_ip.c:885:22: error: storage size of 'address' isn't known
  struct sockaddr_in6 address;
                      ^~~~~~~
code/qcommon/net_ip.c:908:30: error: 'FIONBIO' undeclared (first use in this function); did you mean 'SO_NBIO'?
  if( ioctlsocket( newsocket, FIONBIO, &_true ) == SOCKET_ERROR ) {
                              ^~~~~~~
                              SO_NBIO
code/qcommon/net_ip.c:930:23: error: 'in6addr_any' undeclared (first use in this function); did you mean 'in_addr_t'?
   address.sin6_addr = in6addr_any;
                       ^~~~~~~~~~~
                       in_addr_t
code/qcommon/net_ip.c:885:22: warning: unused variable 'address' [-Wunused-variable]
  struct sockaddr_in6 address;
                      ^~~~~~~
code/qcommon/net_ip.c: In function 'NET_SetMulticast6':
code/qcommon/net_ip.c:969:22: error: storage size of 'addr' isn't known
  struct sockaddr_in6 addr;
                      ^~~~
code/qcommon/net_ip.c:981:18: error: invalid use of undefined type 'struct ipv6_mreq'
  memcpy(&curgroup.ipv6mr_multiaddr, &addr.sin6_addr, sizeof(curgroup.ipv6mr_multiaddr));
                  ^
code/qcommon/net_ip.c:981:69: error: invalid use of undefined type 'struct ipv6_mreq'
  memcpy(&curgroup.ipv6mr_multiaddr, &addr.sin6_addr, sizeof(curgroup.ipv6mr_multiaddr));
                                                                     ^
code/qcommon/net_ip.c:988:11: error: invalid use of undefined type 'struct ipv6_mreq'
   curgroup.ipv6mr_interface = if_nametoindex(net_mcast6iface->string);
           ^
code/qcommon/net_ip.c:988:31: error: implicit declaration of function 'if_nametoindex' [-Werror=implicit-function-declaration]
   curgroup.ipv6mr_interface = if_nametoindex(net_mcast6iface->string);
                               ^~~~~~~~~~~~~~
code/qcommon/net_ip.c:992:11: error: invalid use of undefined type 'struct ipv6_mreq'
   curgroup.ipv6mr_interface = 0;
           ^
code/qcommon/net_ip.c:969:22: warning: unused variable 'addr' [-Wunused-variable]
  struct sockaddr_in6 addr;
                      ^~~~
code/qcommon/net_ip.c: In function 'NET_JoinMulticast6':
code/qcommon/net_ip.c:1008:5: error: implicit declaration of function 'IN6_IS_ADDR_MULTICAST'; did you mean 'IN_MULTICAST'? [-Werror=implicit-function-declaration]
  if(IN6_IS_ADDR_MULTICAST(&boundto.sin6_addr) || IN6_IS_ADDR_UNSPECIFIED(&boundto.sin6_addr))
     ^~~~~~~~~~~~~~~~~~~~~
     IN_MULTICAST
code/qcommon/net_ip.c:1008:35: error: invalid use of undefined type 'struct sockaddr_in6'
  if(IN6_IS_ADDR_MULTICAST(&boundto.sin6_addr) || IN6_IS_ADDR_UNSPECIFIED(&boundto.sin6_addr))
                                   ^
code/qcommon/net_ip.c:1008:50: error: implicit declaration of function 'IN6_IS_ADDR_UNSPECIFIED' [-Werror=implicit-function-declaration]
  if(IN6_IS_ADDR_MULTICAST(&boundto.sin6_addr) || IN6_IS_ADDR_UNSPECIFIED(&boundto.sin6_addr))
                                                  ^~~~~~~~~~~~~~~~~~~~~~~
code/qcommon/net_ip.c:1008:82: error: invalid use of undefined type 'struct sockaddr_in6'
  if(IN6_IS_ADDR_MULTICAST(&boundto.sin6_addr) || IN6_IS_ADDR_UNSPECIFIED(&boundto.sin6_addr))
                                                                                  ^
In file included from /usr/local/ps3dev/ppu/include/netinet/in.h:92:0,
                 from code/qcommon/net_ip.c:69:
code/qcommon/net_ip.c:1015:78: error: invalid use of undefined type 'struct sockaddr_in6'
   if((multicast6_socket = NET_IP6Socket(net_mcast6addr->string, ntohs(boundto.sin6_port), NULL, &err)) == INVALID_SOCKET)
                                                                              ^
/usr/local/ps3dev/ppu/include/arpa/inet.h:10:26: note: in definition of macro 'ntohs'
 #define ntohs(netshort) (netshort)
                          ^~~~~~~~
code/qcommon/net_ip.c:1022:13: error: invalid use of undefined type 'struct ipv6_mreq'
  if(curgroup.ipv6mr_interface)
             ^
code/qcommon/net_ip.c:1024:37: error: 'IPPROTO_IPV6' undeclared (first use in this function); did you mean 'IPPROTO_IP'?
   if (setsockopt(multicast6_socket, IPPROTO_IPV6, IPV6_MULTICAST_IF,
                                     ^~~~~~~~~~~~
                                     IPPROTO_IP
code/qcommon/net_ip.c:1024:51: error: 'IPV6_MULTICAST_IF' undeclared (first use in this function); did you mean 'IP_MULTICAST_IF'?
   if (setsockopt(multicast6_socket, IPPROTO_IPV6, IPV6_MULTICAST_IF,
                                                   ^~~~~~~~~~~~~~~~~
                                                   IP_MULTICAST_IF
code/qcommon/net_ip.c:1025:24: error: invalid use of undefined type 'struct ipv6_mreq'
      (char *) &curgroup.ipv6mr_interface, sizeof(curgroup.ipv6mr_interface)) < 0)
                        ^
code/qcommon/net_ip.c:1025:58: error: invalid use of undefined type 'struct ipv6_mreq'
      (char *) &curgroup.ipv6mr_interface, sizeof(curgroup.ipv6mr_interface)) < 0)
                                                          ^
code/qcommon/net_ip.c:1038:50: error: 'IPV6_JOIN_GROUP' undeclared (first use in this function)
  if (setsockopt(multicast6_socket, IPPROTO_IPV6, IPV6_JOIN_GROUP, (char *) &curgroup, sizeof(curgroup)))
                                                  ^~~~~~~~~~~~~~~
code/qcommon/net_ip.c:1038:93: error: invalid application of 'sizeof' to incomplete type 'struct ipv6_mreq'
  if (setsockopt(multicast6_socket, IPPROTO_IPV6, IPV6_JOIN_GROUP, (char *) &curgroup, sizeof(curgroup)))
                                                                                             ^
code/qcommon/net_ip.c: In function 'NET_LeaveMulticast6':
code/qcommon/net_ip.c:1058:34: error: 'IPPROTO_IPV6' undeclared (first use in this function); did you mean 'IPPROTO_IP'?
    setsockopt(multicast6_socket, IPPROTO_IPV6, IPV6_LEAVE_GROUP, (char *) &curgroup, sizeof(curgroup));
                                  ^~~~~~~~~~~~
                                  IPPROTO_IP
code/qcommon/net_ip.c:1058:48: error: 'IPV6_LEAVE_GROUP' undeclared (first use in this function)
    setsockopt(multicast6_socket, IPPROTO_IPV6, IPV6_LEAVE_GROUP, (char *) &curgroup, sizeof(curgroup));
                                                ^~~~~~~~~~~~~~~~
code/qcommon/net_ip.c:1058:92: error: invalid application of 'sizeof' to incomplete type 'struct ipv6_mreq'
    setsockopt(multicast6_socket, IPPROTO_IPV6, IPV6_LEAVE_GROUP, (char *) &curgroup, sizeof(curgroup));
                                                                                            ^
code/qcommon/net_ip.c: In function 'NET_AddLocalAddress':
code/qcommon/net_ip.c:1256:21: error: invalid application of 'sizeof' to incomplete type 'struct sockaddr_in6'
    addrlen = sizeof(struct sockaddr_in6);
                     ^~~~~~
code/qcommon/net_ip.c: In function 'NET_GetLocalAddress':
code/qcommon/net_ip.c:1280:5: error: implicit declaration of function 'getifaddrs'; did you mean 'getitimer'? [-Werror=implicit-function-declaration]
  if(getifaddrs(&ifap))
     ^~~~~~~~~~
     getitimer
code/qcommon/net_ip.c:1284:45: error: dereferencing pointer to incomplete type 'struct ifaddrs'
   for(search = ifap; search; search = search->ifa_next)
                                             ^~
code/qcommon/net_ip.c:1287:25: error: 'IFF_UP' undeclared (first use in this function)
    if(ifap->ifa_flags & IFF_UP)
                         ^~~~~~
code/qcommon/net_ip.c:1291:3: error: implicit declaration of function 'freeifaddrs' [-Werror=implicit-function-declaration]
   freeifaddrs(ifap);
   ^~~~~~~~~~~
code/qcommon/net_ip.c: In function 'NET_Sleep':
code/qcommon/net_ip.c:1695:11: error: implicit declaration of function 'select'; did you mean 'sleep'? [-Werror=implicit-function-declaration]
  retval = select(highestfd + 1, &fdr, NULL, NULL, &timeout);
           ^~~~~~
           sleep
code/qcommon/net_ip.c: At top level:
code/qcommon/net_ip.c:124:25: error: storage size of 'curgroup' isn't known
 static struct ipv6_mreq curgroup;
                         ^~~~~~~~
code/qcommon/net_ip.c:126:28: error: storage size of 'boundto' isn't known
 static struct sockaddr_in6 boundto;
                            ^~~~~~~

[zeldin]

Ok, struct sockaddr_in6 is indeed missing from netinet/in.h. That could be added. Did you check that the SPRX actually supports IPv6 though (i.e. that socket(PF_INET6, SOCK_DGRAM, 0) doesn't just return an error)? This seems a bit moot otherwise. :slightly_smiling_face: The values of the setsockopt parameters, if those specific setsockopts indeed exist, are OS-specific (only the names are "standardized"), so be prepared to do a bit of detective work to find those out...

[dee12452]

@zeldin I was able to verify that the net sprx does NOT support IPV6 with what you suggested. I used socket(PF_INET6, SOCK_DGRAM, 0) and it returned -1 and set errno to EPROTONOSUPPORT 123 /* Unknown protocol */. So someone would have to write custom IPV6 software emulation support into libnet (or wherever) if really desired...

However, I understand @grepwood's pain here because I'm working on porting a library that uses some of these definitions because it supports both IPV4 and IPV6. So I think the ask here would be to add the missing definitions to the headers, and just make sure any time there are functions that take IPV6 options to return an error case and set errno appropriately.

[zeldin]

Fair enough, although if the library has an option to build it without IPv6 support you should use it because otherwise you'll just end up with dead code in the .a...

[zeldin]

And regarding the IPPROTO_IPV6 and related sockopts, these are optional in POSIX, so no code targeting POSIX should use them without testing for existence (margin code "IP6" in the POSIX standard document). There is also nothing we could reasonably define them to since the SPRX does not have any corresponding functionality.

And actually struct sockaddr_in6 is also optional (at first read in the standard I thought the margin note applied only to in6_addr but it actually extends to cover sockaddr_in6, in6addr_any and in6addr_loopback as well), so code unconditionally using it should be fixed.

[zeldin]

The correct way to handle this in the library/application is to #include <unistd.h>, and then test #ifdef _POSIX_IPV6. Unless there is already autoconf style tests to handle this, like there is in e.g. libcurl.

[dee12452]

Oh wow you're right, a lot of the IPV6 definitions in the netinet/in.h are optional. Which makes sense. So yeah as you're saying, since we know the PS3 doesn't natively support IPV6 there's no sense in adding those. There are other items in there that are not necessarily optional e.g. sockaddr_storage which should exist in <sys/socket.h> according to https://pubs.opengroup.org/onlinepubs/007908799/xns/syssocket.h.html

[grepwood]

@dee12452 @zeldin please check out https://github.com/ec-/Quake3e/issues/55 where the developer has made IPv6 completely optional. It still fails and crumbles in a billion places which pretty much means the existing stack is either largely incomplete or there is a magical header missing that would translate all of the standard IPv4 calls into socket(blargh,bork) calls. And what about https://github.com/ps3dev/PSL1GHT/issues/67? If psl1ght will eventually support it, that would be really great.

[dee12452]

@grepwood it's very likely just that the toolchain is incomplete. From just looking at it, you can kinda tell that the net stack was built on a "build it when you need it" mindset, so it's missing a lot of standard things even defined by POSIX standards. I too am not able to port some things I need for my project because of missing structs, macros, and functions which aren't necessarily dependent on IPV6 support. I would do more to help right now since I also need a lot of what you need, but I'm also brand new and still learning the stack. I'm guessing me being new probably warrants leeriness of me pushing code too.

As for https://github.com/ps3dev/PSL1GHT/issues/67, I actually started a POC over in newlib to move the net stack over to there, but it didn't take me long to realize that to do what is suggested would require a big change to how the toolchain is built in general. Right now gcc + newlib are installed without knowing anything about PSL1GHT, and since the net code (like socket.c, poll.c, inet funcs, etc.) depends on the net.h header definitions + the vsh exports in the libnet sprx, the toolchain build process is going to have to change significantly to support the suggested changes. Such a significant change that I wonder if it'd be worth it to make a new project for it entirely...

[zeldin]

@dee12452 All newlib code which needs to use sprx exports has to do it through callbacks in __syscalls, see libgloss/libsysbase in newlib. PSL1GHT implements the callback and can do the actual sprx call as well as translate any input/output parameters if needed.

[zeldin]

That beging said, it's unclear to me what the benefit of moving things over to newlib would be, since newlib does not currently contain any networking code that is not linux specific...

[dee12452]

@zeldin ah I see how that works now. And the callbacks are defined in newlib lv2/sys/syscallbacks.h. Good to know, I was definitely missing that piece.

With that being said, I'm aware that moving the network code stack to newlib provides nothing for this particular issue, but instead just supports the ideas explained in the v3 RFC over in PSL1GHT. @grepwood do note that the RFC you're referring to won't necessarily add all the things that you and I need, it's essentially just a large cleanup task with what's currently implemented in PSL1GHT. The things you're asking for here need to be added to the headers whether someone moves them to newlib or not.