IPv6 patch against 2.0 CVS tree, as of 20010624

22a6cad4-a2c3-4f35-b3c6-c3a6d1e8153e commented 24 years ago

BPO	401196
Nosy	@gvanrossum, @loewis, @freddrake
Files	None: None ipv6-010624.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = 'https://github.com/loewis' closed_at = created_at = labels = ['interpreter-core'] title = 'IPv6 patch against 2.0 CVS tree, as of 20010624' updated_at = user = 'https://bugs.python.org/itojun' ``` bugs.python.org fields: ```python activity = actor = 'loewis' assignee = 'loewis' closed = True closed_date = None closer = None components = ['Interpreter Core'] creation = creator = 'itojun' dependencies = [] files = ['2717', '2718'] hgrepos = [] issue_num = 401196 keywords = ['patch'] message_count = 14.0 messages = ['33881', '33882', '33883', '33884', '33885', '33886', '33887', '33888', '33889', '33890', '33891', '33892', '33893', '33894'] nosy_count = 6.0 nosy_names = ['gvanrossum', 'nobody', 'loewis', 'fdrake', 'moshez', 'itojun'] pr_nums = [] priority = 'normal' resolution = 'accepted' stage = None status = 'closed' superseder = None type = None url = 'https://bugs.python.org/issue401196' versions = [] ```

22a6cad4-a2c3-4f35-b3c6-c3a6d1e8153e commented 24 years ago

freddrake commented 24 years ago

Postponed until Python 2.1 -- there's not enough time to review this and get it sufficiently tested on enough IPv6-connected platforms in time for 2.0, and we're already in feature freeze. This should go into the tree very quickly once Python 2.0 has been released.

Assigned to myself to open it back up after Python 2.0.

gvanrossum commented 23 years ago

A new patch is available. I've changed the subject accordingly.

Due to upload size restrictions, the patch is now at

http://www.itojun.org/diary/20001230/python-2.0-v6-20001230.diff.gz

1fb001b0-327a-48b2-868c-8ccd3dcdb0f5 commented 24 years ago

Assigned to Tim, since he's in charge of postponing new features. I'm to timid to postpone it myself.

61337411-43fc-4a9c-b8d5-4060aede66d0 commented 23 years ago

I got *many* rejects when trying to apply this patch to today's CVS tree. I recommend that patches for generated files (config.h.in, configure) are not included in the patch because they outdate too easily. A number of changes in this patch have already been done by somebody else; others just don't fit into the current code anymore (perhaps due to indentation changes?). Anyway, I'll mark the patch as out-of-date. Please let me know when you upload a new version.

61337411-43fc-4a9c-b8d5-4060aede66d0 commented 23 years ago

After a shallow review of this patch, I found the following issues:

configure.in does not need to list both enable and disable options.

When running configure, I got the following error message on Linux checking whether to enable ipv6... yes checking ipv6 stack type... linux-glibc ./configure: test: too many arguments using libc

The call to /usr/bin/test should be corrected; I could not find out which specific invocation caused the problem.

HAVE_ADDRINFO is not used. Perhaps getaddrinfo.c/getnameinfo.c is missing in the patch?

22a6cad4-a2c3-4f35-b3c6-c3a6d1e8153e commented 24 years ago

this is revised version of patch bpo-101186 (now with my SourceForge accout... i'm not familiar with the system here, so forgive my possible mistake).

1.6b1 patch applied mostly clean to 2.0. It is confirmed that:

1.6b1 + IPv6 patch works fine on NetBSD 1.4.2 + KAME, and NetBSD 1.5
1.6b1 + IPv6 patch works fine on NetBSD 1.4.2 (NOT an IPv6 ready machine)
2.0 CVS tree + IPv6 patch works fine on NetBSD + KAME

forgot to attach the following into the diff - so i attach it (README.v6) here as comment. I have submitted the patch for 1.5.1, 1.5.2 and 1.6b1, all hit a bad timing - bad luck.

contact: core@kame.net, or itojun@kame.net

--- IPv6-ready python 1.6 KAME Project $KAME: README.v6,v 1.9 2000/08/15 02:40:38 itojun Exp $

This patchkit enables python 1.6 to perform AF_INET6 socket operations. The only affected module is Modules/socketmodule.c.

Modules/socketmodule.c In most cases, IPv6 address can be placed where IPv4 address fits.

sockaddr
sockaddr tuple is formatted as follows:
    IPv4: (host, port)
    IPv6: socket class methods always generate
        (host, port, flowinfo, scopeid).
      socket class methods will accept 2, 3, or 4 tuple
      (for backward compatibility).

Compatibility warning: Some of the scripts assume that the sockaddr
structure is 2 tuple, like:
    host, port = sock.getpeername()
this will fail if you are connected to IPv6 node.

    socket.getaddrinfo(host, port [, family, socktype, proto, flags])
    host: String or None
    port: String, Int or None
    family, socktype, proto, flags: Int, can be omitted

Perform getaddrinfo(3).  Returns List of the following 5 tuple:
    (family, socktype, proto, canonname, sockaddr)
    family: Int
    socktype: Int
    proto: Int
    canonname: String
    sockaddr: sockaddr (see above)

See [Lib/httplib.py](https://github.com/python/cpython/blob/main/Lib/httplib.py) for typical usage on the client side.

    socket.getnameinfo(sockaddr, flags)
    sockaddr: sockaddr
    flags: Int

Perform getnameinfo(3).  Returns the following 2 tuple:
    host: String, numeric or hostname depending on flgags
    port: String, numeric or portname depending on flgags

    socket.gethostbyname2(host, af)
    host: String
    af: Int

Performs gethostbyname2(3).  Returns numeric address representation
for "host".

socket.gethostbyaddr(addr) (behavior change if IPv6 support is compiled in)
addr: String

Performs gethostbyaddr(3).  Returns string address representation for
"addr".

The function can take IPv6 numeric address as well.  This behavior
is not problematical, because
- if you pass numeric "addr" parameter, we can always identify address
  family for it
- return value is string address reprsentation, where IPv6 and IPv4
  are not distinguishable.

 socket.bind(sa), socket.connect(sa) and others.
(No behavior change, but be careful)

See above for sockaddr format change.

With Python "addr" portion of sockaddr (first element) can be string
hostname.  When the string hostname resolved to numeric address, it
will obey address family of the socket (which was specified when
socket.socket() was called).
If you give some string that does not give matching address family,
you will get some error.
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    \# this is okay, if 'localhost' resolves to both IPv4/v6
    s.connect('localhost', 80)
    \# this is not okay, of course
    s.connect('::1', 80)
    \# this is not okay, as v6only.kame.net will not resolve to IPv4
    s.connect('v6only.kame.net', 80)

Lib/httplib.py IPv6 ready. "host" in HTTP(host) will accept the following 3 forms:

    host:port   there must be only single colon
    host
This is to allow IPv6 numeric URL (http://[host]:port/) in documents.

IMHO "host:port" parsing should be implemented in urllib.py, not here.

Lib/ftplib.py IPv6 ready. This uses EPSV/EPRT on IPv6 ftp. See RFC2428 for protocol details.

Lib/SocketServer.py IPv6 ready. Wildcard bind on TCPServer, i.e. TCPServer(('', port)), will bind to wildcard socket on TCPServer.address_family. TCPServer.addresss_family is set to AF_INET by default, so ('', port) will usually bind AF_INET.

Lib/smtplib.py, Lib/telnetlib.py, Lib/poplib.py IPv6 ready. Not much to say about protocol details - they just use TCP over IPv6.

configure Configure has extra option, --enable-ipv6 and --disable-ipv6. The option controls IPv6 support feature.

dynamic link issues in Modules/socketmodule.c Modules/socketmodule.c can be dynamically loaded only in the following situations:

getaddrinfo(3) and getnameinfo(3) are supplied by OS vendor in libc, and libc is dynamic link library.
OS vendor is NOT supplying getaddrinfo(3) nor getnameinfo(3), and You are configuring this package with --disable-ipv6. In this case, you'll be using missing/get{addr,name}info.c and they will refer to gethostby{name,addr}. gethostnameby{name,addr} can usually be found in dynamic-linking libc.

In other situations, such as the following, please link Modules/socketmodule.c into python itself.
getaddrinfo(3) and getnameinfo(3) are supplied by OS vendor, but they are in statically linked library like libinet6.a. (KAME falls into this category)

python usually links Modules/socketmodule.c into python itself (due to its popularity) so there should be no problem.

restrictions

The patched tree will not use gethostbyname_r and other thread-ready libraries. Instead, it will use getaddrinfo() and getnameinfo() throughout the operation.

todo

Patch bunch of library files in Lib/*.py.

compatibility issues with existing scripts If you disable IPv6 support (./configure --disable-ipv6), the patched code is mostly compatible with original Python (except files in "Lib" directory modified for dual stack support).

User script may choke if:
- IPv4/v6 dualstack libc is supplied, python is compiled for dual
  stack, and script assumes some of IPv4-only behavior (especially
  sockaddr)
- IPv4/v6 dualstack libc is supplied, python is compiled for IPv4 only,
  and script assumes some of IPv4-only behavior.
  In this case, Python socket class itself does not support IPv6,
  however, name resolution functions can return IPv6 names since
  they use IPv6-ready libc functions!  I do not recommend this
  configuration.
- script assumes certain IPv4-only version behavior in Lib/*.py.

compilation If you use IPv6 features, it is assumed that you have working getaddrinfo() and getnameinfo() library functions. We have noticed that some of IPv6 stack is shipped with broken getaddrinfo(). In such cases, use missing/get{addr,name}info.c instead (but then, you need to have working getipnodeby{name,addr}).

If you compile this on IPv4-only machine without get{addr,name}info,
missing/get{addr,name}info.c will be used.  They are from KAME IPv6
distribution and is #ifdef'ed for IPv4 only support.  They are
fairly complete implementation and you don't need to bother with
bind 8.2 (bind 8.2 get{addr,name}info() has bugs).

When compiling this kit on IPv6 node, you may need to specify some
additional library paths or cpp defs. (like -linet6 or -DINET6)
--enable-ipv6 will give you some warning, if the IPv6 stack is unknown
to the "configure" script.  Currently, the following IPv6 stacks
are officially supported (i.e. we've checked that the package works
well):
- KAME IPv6 stack, http://www.kame.net/

References RFC2553, for getaddrinfo(3) and getnameinfo(3).

Author contacts http://www.kame.net/ mailto:core@kame.net

22a6cad4-a2c3-4f35-b3c6-c3a6d1e8153e commented 23 years ago

Logged In: YES user_id=63767

about /usr/bin/test argument: does linux /usr/bin/test have -d \<dir> support? if not, we may need to change configure.in slightly.

you are correct that fallback getaddrinfo/getnameinfo.c was missing in the patch. sorry. a question i need to ask is, do we need to supply Python function Socket.getaddrinfo on platforms that do not have getaddrinfo(3)?

HAVE_ADDRINFO is used in Include/addrinfo.h, which is also missing in the patch set i have submitted.

i've put the missing files into http://www.itojun.org/diary/20001230/missing.shar.

gvanrossum commented 23 years ago

Logged In: YES user_id=6380

See http://mail.python.org/pipermail/python-dev/2001-May/014889.html for comments from MvL.

I'm unassigning this from Fred, he has nothing to do with this.

3772858d-27d8-44b0-a664-d68674859f36 commented 23 years ago

Logged In: NO

i looked at python-dev email. the proposal (split patches) looks fine, but the exact example given in python-dev email is not reasonable. i cannot just send out configure.in change separately from source code changes, period. i can split patches for *.py files separately though.

there's more important issue, which is, APi changes for Socket class. i really hoped to get some comment on that part. i really appreciate your comments. i would like to propose that once we nailed down API changes, integrate the patch into the tree. with all #ifdef INET6 in place there should be no impact on IPv4-only builds.

i have trouble tracking python development (i'm not a sourceforge expert!), so forgive me for delays in patch submissions.

61337411-43fc-4a9c-b8d5-4060aede66d0 commented 23 years ago

Logged In: YES user_id=21627

On the API, I have the following comments:

Why is it necessary to introduce gethostbyname2? I recommend to give gethostbyname an optional argument for the address family.
getaddrinfo, when raising a socket error, should include the EAI error number. Perhaps there should be a way tod istinguish EAI errnos from other errnos, e.g. by subclassing socket error.

Otherwise, the API of the C part looks good to me. Ih aven't looked at the Lib part, yet.

On the implementation:

I still have problems building the code. Currently, I get the following rejects: ./Lib/BaseHTTPServer.py.rej ./Lib/ftplib.py.rej ./Lib/poplib.py.rej ./Lib/smtplib.py.rej ./Modules/socketmodule.c.rej ./Objects/fileobject.c.rej
The fileobject.c chunk seems to be unnecessary.
On the test problem: It occurs in + test -d -a -f /lib.a ./configure: test: too many arguments which comes from ipv6libdir and ipv6libdir being empty.
The WIDE files should be included in the Modules directory, as they are only used from socketmodule.c. In particular, addrinfo.h should not be installed.
If you can, please include a patch to Doc/lib/libsocket.tex. If not, I will try to draft one.

61337411-43fc-4a9c-b8d5-4060aede66d0 commented 23 years ago

Logged In: YES user_id=21627

I have uploaded a new version sent by itojun.

61337411-43fc-4a9c-b8d5-4060aede66d0 commented 23 years ago

Logged In: YES user_id=21627

The socketmodule.c part has been committed as socketmodule.c 1.153.

61337411-43fc-4a9c-b8d5-4060aede66d0 commented 23 years ago

Logged In: YES user_id=21627

The last chunk of this patch was committed as BaseHTTPServer.py 1.16 SocketServer.py 1.26 ftplib.py 1.54 httplib.py 1.36 poplib.py 1.15 smtplib.py 1.37 telnetlib.py 1.12

python / cpython

IPv6 patch against 2.0 CVS tree, as of 20010624 #32944