search

GNS3 / gns3-server

GNS3 server
GNU General Public License v3.0
798 stars 262 forks source link

Support dual stack for gns3server API and consoles #1673

Open candlerb opened 4 years ago

candlerb commented 4 years ago

I would like gns3server to support dual-stack operation. What I have found is:

$ sudo netstat -natp | grep LISTEN.*python
tcp6       0      0 :::5001                 :::*                    LISTEN      1681/python
tcp6       0      0 :::3080                 :::*                    LISTEN      1681/python
$ telnet 127.0.0.1 3080
Trying 127.0.0.1...
telnet: Unable to connect to remote host: Connection refused
$ telnet ::1 3080
Trying ::1...
Connected to ::1.
Escape character is '^]'.

This appears to be enforced by asyncio.create_server. In asyncio/base_events.py there is code that explicitly disables dual-stack socket support when binding to IPv6 address family:

                    # Disable IPv4/IPv6 dual stack support (enabled by
                    # default on Linux) which makes a single socket
                    # listen on both address families.
                    if (_HAS_IPv6 and
                            af == socket.AF_INET6 and
                            hasattr(socket, 'IPPROTO_IPV6')):
                        sock.setsockopt(socket.IPPROTO_IPV6,
                                        socket.IPV6_V6ONLY,
                                        True)

(PEP-3156 describes the asyncio library, but it doesn't mention why it sets IPV6_V6ONLY)

The create_server documentation says:

  • The host parameter can be set to several types which determine where the server would be listening:

    • If host is a string, the TCP server is bound to a single network interface specified by host.
    • If host is a sequence of strings, the TCP server is bound to all network interfaces specified by the sequence.
    • If host is an empty string or None, all interfaces are assumed and a list of multiple sockets will be returned (most likely one for IPv4 and another one for IPv6).

Therefore, I think the right thing to do is to pass None or empty string for the host. However I couldn't get this to work in gns3_server.conf. I tried:

[Server]
host =

which gives:

2019-10-27 20:16:27 INFO __init__.py:66 Controller is starting
2019-10-27 20:16:27 INFO compute.py:63 Create compute local
2019-10-27 20:16:27 CRITICAL run.py:262 Critical error while running the server: Conflict
Traceback (most recent call last):
  File "/usr/share/gns3/gns3-server/lib/python3.6/site-packages/gns3server/controller/project.py", line 906, in open
    await self.add_node(compute, name, node_id, dump=False, **node)
  File "/usr/share/gns3/gns3-server/lib/python3.6/site-packages/gns3server/controller/project.py", line 549, in add_node
    await compute.post("/projects", data=data)
  File "/usr/share/gns3/gns3-server/lib/python3.6/site-packages/gns3server/controller/compute.py", line 569, in post
    response = await self.http_query("POST", path, data, **kwargs)
  File "/usr/share/gns3/gns3-server/lib/python3.6/site-packages/gns3server/controller/compute.py", line 339, in http_query
    raise ComputeError("Cannot connect to compute '{}' with request {} {}".format(self._name, method, path))
gns3server.controller.compute.ComputeError: Cannot connect to compute 'brian-kit' with request POST /projects

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/share/gns3/gns3-server/lib/python3.6/site-packages/gns3server/run.py", line 256, in run
    server.run()
  File "/usr/share/gns3/gns3-server/lib/python3.6/site-packages/gns3server/web/web_server.py", line 297, in run
    if self._run_application(self._handler, ssl_context) is False:
  File "/usr/share/gns3/gns3-server/lib/python3.6/site-packages/gns3server/web/web_server.py", line 85, in _run_application
    self._server, startup_res = self._loop.run_until_complete(asyncio.gather(srv, self._app.startup(), loop=self._loop))
  File "/usr/lib/python3.6/asyncio/base_events.py", line 484, in run_until_complete
    return future.result()
  File "/usr/share/gns3/gns3-server/lib/python3.6/site-packages/aiohttp/web_app.py", line 389, in startup
    await self.on_startup.send(self)
  File "/usr/share/gns3/gns3-server/lib/python3.6/site-packages/aiohttp/signals.py", line 34, in send
    await receiver(*args, **kwargs)  # type: ignore
  File "/usr/share/gns3/gns3-server/lib/python3.6/site-packages/gns3server/web/web_server.py", line 212, in _on_startup
    await Controller.instance().start()
  File "/usr/share/gns3/gns3-server/lib/python3.6/site-packages/gns3server/controller/__init__.py", line 107, in start
    await self._project_auto_open()
  File "/usr/share/gns3/gns3-server/lib/python3.6/site-packages/gns3server/controller/__init__.py", line 444, in _project_auto_open
    await project.open()
  File "/usr/share/gns3/gns3-server/lib/python3.6/site-packages/gns3server/utils/asyncio/__init__.py", line 165, in wrapper
    return await f(oself, *args, **kwargs)
  File "/usr/share/gns3/gns3-server/lib/python3.6/site-packages/gns3server/controller/project.py", line 947, in open
    raise aiohttp.web.HTTPConflict(text=str(e))
aiohttp.web_exceptions.HTTPConflict: Conflict
2019-10-27 20:16:27 WARNING crash_report.py:84 A .git directory exist crash report is turn off for developers

Note 1: gns3server --host "" gives the same result Note 2: brian-kit is the system's hostname. This resolves to 127.0.0.1 in /etc/hosts

And I tried:

[Server]
host = ""

which gives:

2019-10-27 20:16:57 INFO __init__.py:66 Controller is starting
2019-10-27 20:16:57 INFO compute.py:63 Create compute local
2019-10-27 20:16:57 INFO compute.py:360 Connecting to compute 'local'
2019-10-27 20:16:57 WARNING compute.py:363 Cannot connect to compute 'local': Cannot connect to host "":3080 ssl:None [Temporary failure in name resolution]
2019-10-27 20:16:57 INFO compute.py:360 Connecting to compute 'local'
2019-10-27 20:16:57 CRITICAL web_server.py:87 Could not start the server: [Errno -3] Temporary failure in name resolution

i.e. it appears to be resolving the literal domain name ""

There are several places where 0.0.0.0 and :: are treated as special cases, so maybe empty-string needs to be added to those.

ISTM that when this is working, empty string ought to become the default.

candlerb commented 4 years ago

Another possibility would be to allow the host setting to be a list, e.g.

host = 0.0.0.0, ::

since this is already supported by create_server.

Raizo62 commented 4 years ago

Hi

Can we connect to remote gns3server with IPv6 ? I have tried and i have failed.

candlerb commented 4 years ago

Yes, as described above you can configure in ~/.config/GNS3/2.2/gns3_server.conf

[Server]
host = ::

Unfortunately this means you can only connect on IPv6, and not on IPv4. That's what this ticket is about :-)

Raizo62 commented 4 years ago

Thanks for the answer. Yes, to choose between IPv4 or IPv6 is not super.

spikefishjohn commented 2 years ago

I went through a massive rabbit hole on this at one point. Basically it ended with looking at the source for asyncio I found this.

async def create_server <- i think this is where its defined.

https://github.com/python/cpython/blob/d71f5adc41569c2d626552269797e0545fc9122c/Lib/asyncio/events.py#L310

"If host is an empty string or None all interfaces are assumed and a list of multiple sockets will be returned (most likely one for IPv4 and another one for IPv6). The host parameter can also be a sequence (e.g. a list) of hosts to bind to."

So my assumption is if you passed a list of ['::', '0.0.0.0'] then it might bind to all? Also this in theory would work if you wanted GNS3 to listen on multiple ipv4 addresses.

The key is host would need to be a list and not a string, which most likely means refactoring a lot.

athompson-merlin commented 1 year ago

This sure shoots people - who used GNS3 to validate the dual-stacking of their environment before proceeding - in the foot. I haven't dived in to asyncio as deeply as @spikefishjohn, but I did also confirm that None as a hostname does not work, with the following log output:

2022-10-28 18:05:00 INFO web_server.py:318 Starting server on None:80
2022-10-28 18:05:00 INFO __init__.py:63 Load controller configuration file /home/gns3/.config/GNS3/2.2/gns3_controller.conf
2022-10-28 18:05:00 INFO __init__.py:67 Controller is starting
2022-10-28 18:05:00 INFO compute.py:64 Create compute local
2022-10-28 18:05:00 INFO compute.py:364 Connecting to compute 'local'
2022-10-28 18:05:00 CRITICAL web_server.py:88 Could not start the server: [Errno -2] Name or service not known

In the meantime, the only solution I had, now that I'm on a dual-stacked network, was to remove the AAAA record for my GNS3 server, set Host to 0.0.0.0 (the default) and for good measure, configure the client to connect directly to the IPv4 address instead of the hostname.

spikefishjohn commented 1 year ago

edit gns3server/web/web_server.py

I changed 

    def _run_application(self, handler, ssl_context=None):
        try:
            srv = self._loop.create_server(handler, self._host, self._port, ssl=ssl_context)
            self._server, startup_res = self._loop.run_until_complete(asyncio.gather(srv, self._app.startup()))
        except (RuntimeError, OSError, asyncio.CancelledError) as e:

to

    def _run_application(self, handler, ssl_context=None):
        try:
            srv = self._loop.create_server(handler, ['::', '0.0.0.0'], self._port, ssl=ssl_context)
            self._server, startup_res = self._loop.run_until_complete(asyncio.gather(srv, self._app.startup()))
        except (RuntimeError, OSError, asyncio.CancelledError) as e:
            log.critical("Could not start the server: {}".format(e))
            return False

root@compute01:~# netstat -anp | grep 1435171
tcp        0      0 0.0.0.0:3080            0.0.0.0:*               LISTEN      1435171/python
tcp        0      0 127.0.0.1:55378         127.0.0.1:3080          ESTABLISHED 1435171/python
tcp        0      0 127.0.0.1:3080          127.0.0.1:55378         ESTABLISHED 1435171/python
tcp6       0      0 :::3080                 :::*                    LISTEN      1435171/python
root@compute01

I was able to connect to IPv4 address. Sure looks like its running dual stack now.

I didn't do anything to the gui to try to get it to connect via IPv6.. uh... because I'm not running IPv6 on my lab (makes sad network sounds).

The problem is the host line is passed as a string. So if you make it None its sending str("None") vs None or str("['::', '0.0.0.0']") instead of ['::', '0.0.0.0'].

I'm just throwing this out there to show it seems possible. You seem to have a dual stack. Maybe you can do the other hack testing?

ghost commented 1 year ago

What GNS3 currently doesn't support is a list of IP addresses in the host field of gns3server.conf. But it already supports dual stack connections, when the host field contains a hostname, which has both IPv4 and IPv6 addresses.

For example, on host = localhost GNS3 listens for 127.0.0.1 and ::1 on port 3080 (GNS3 server) and the console ports 50xx:

$ ss -tln | egrep ":3080|:50"
LISTEN 0      1          127.0.0.1:5000       0.0.0.0:*          
LISTEN 0      100        127.0.0.1:3080       0.0.0.0:*          
LISTEN 0      100        127.0.0.1:5001       0.0.0.0:*          
LISTEN 0      100        127.0.0.1:5002       0.0.0.0:*          
LISTEN 0      100        127.0.0.1:5003       0.0.0.0:*          
LISTEN 0      1          127.0.0.1:5004       0.0.0.0:*          
LISTEN 0      1              [::1]:5000          [::]:*          
LISTEN 0      100            [::1]:3080          [::]:*          
LISTEN 0      100            [::1]:5001          [::]:*          
LISTEN 0      100            [::1]:5002          [::]:*          
LISTEN 0      100            [::1]:5003          [::]:*

When using host = iMac GNS3 listens for the IPv4 and all IPv6 addresses of iMac:

$ host imac
imac.lan has address 192.168.1.10
imac.lan has IPv6 address fd31:12e3:eb01::3e5
imac.lan has IPv6 address 2a01:71a0:8014:d000::3e5
$ ss -tln | egrep ":3080|:50"
LISTEN 0      1                    192.168.1.10:5000       0.0.0.0:*          
LISTEN 0      100                  192.168.1.10:3080       0.0.0.0:*          
LISTEN 0      100                  192.168.1.10:5001       0.0.0.0:*          
LISTEN 0      100                  192.168.1.10:5002       0.0.0.0:*          
LISTEN 0      100                  192.168.1.10:5003       0.0.0.0:*          
LISTEN 0      1                       127.0.0.1:5004       0.0.0.0:*          
LISTEN 0      1      [2a01:71a0:8014:d000::3e5]:5000          [::]:*          
LISTEN 0      1           [fd31:12e3:eb01::3e5]:5000          [::]:*          
LISTEN 0      100    [2a01:71a0:8014:d000::3e5]:3080          [::]:*          
LISTEN 0      100         [fd31:12e3:eb01::3e5]:3080          [::]:*          
LISTEN 0      100         [fd31:12e3:eb01::3e5]:5001          [::]:*          
LISTEN 0      100    [2a01:71a0:8014:d000::3e5]:5001          [::]:*          
LISTEN 0      100    [2a01:71a0:8014:d000::3e5]:5002          [::]:*          
LISTEN 0      100         [fd31:12e3:eb01::3e5]:5002          [::]:*          
LISTEN 0      100    [2a01:71a0:8014:d000::3e5]:5003          [::]:*          
LISTEN 0      100         [fd31:12e3:eb01::3e5]:5003          [::]:*

To listen on all IPv4 and IPv6 addresses I created the following entry in /etc/hosts:

0.0.0.0     anyhost
::      anyhost

When configuring host = anyhost GNS3 listens for the following IPs:

$ ss -tln | egrep ":3080|:50"
LISTEN 0      1            0.0.0.0:5000       0.0.0.0:*          
LISTEN 0      100          0.0.0.0:3080       0.0.0.0:*          
LISTEN 0      100          0.0.0.0:5001       0.0.0.0:*          
LISTEN 0      100          0.0.0.0:5002       0.0.0.0:*          
LISTEN 0      100          0.0.0.0:5003       0.0.0.0:*          
LISTEN 0      1          127.0.0.1:5004       0.0.0.0:*          
LISTEN 0      100             [::]:3080          [::]:*          
LISTEN 0      100             [::]:5001          [::]:*          
LISTEN 0      100             [::]:5002          [::]:*          
LISTEN 0      100             [::]:5003          [::]:*

When you closely look at it, the IPv6 listen for port 5000 is missing, that is a dynamips router. So that may need fixing. But the gns3server API (port 3080) as well as the consoles of Docker (here port 5001 and 5002) and QEMU (here port 5003) are available both via IPv4 and IPv6. For the curious: QEMU uses an additional internal port (here 5004), that listens on 127.0.0.1.

Of course the use of the anyhost host entry is just a workaround, allowing a list of IP addresses would be nicer.

spikefishjohn commented 1 year ago

Oh interesting. I was trying to wrap my head around how to patch config.py last night but didn’t get very far.

athompson-merlin commented 1 year ago

@b-ehlers thank you, thank you, thank you! This (using a dual-stacked hostname instead of 0.0.0.0 or :: or similar) works perfectly for my use case and I now have the GNS3 server and qemu consoles accessible via both IP protocol versions.