Is there an option to allow networking?

[rohanraarora]

I want to run a python code with limited time and memory which fetches data from an api. Is there any option to allow networking? I tried using --share-net option but it didn't work.

The code that I run (a.py):

import urllib.request
html = urllib.request.urlopen('https://google.com').read()
print(html)

Output :

Traceback (most recent call last):
  File "/usr/lib/python3.5/urllib/request.py", line 1254, in do_open
    h.request(req.get_method(), req.selector, req.data, headers)
  File "/usr/lib/python3.5/http/client.py", line 1107, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python3.5/http/client.py", line 1152, in _send_request
    self.endheaders(body)
  File "/usr/lib/python3.5/http/client.py", line 1103, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python3.5/http/client.py", line 934, in _send_output
    self.send(msg)
  File "/usr/lib/python3.5/http/client.py", line 877, in send
    self.connect()
  File "/usr/lib/python3.5/http/client.py", line 1253, in connect
    super().connect()
  File "/usr/lib/python3.5/http/client.py", line 849, in connect
    (self.host,self.port), self.timeout, self.source_address)
  File "/usr/lib/python3.5/socket.py", line 694, in create_connection
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
  File "/usr/lib/python3.5/socket.py", line 733, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "a.py", line 2, in <module>
    html = urllib.request.urlopen('https://google.com').read()
  File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.5/urllib/request.py", line 466, in open
    response = self._open(req, data)
  File "/usr/lib/python3.5/urllib/request.py", line 484, in _open
    '_open', req)
  File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 1297, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/usr/lib/python3.5/urllib/request.py", line 1256, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno -2] Name or service not known>

Isolate command: isolate -b 6 -r err.txt --share-net --full-env --run /usr/bin/python3 a.py

[seirl]

That's probably because you need to mount or copy /etc/resolv.conf in your sandbox.

[rohanraarora]

If I mount the /etc/ dir, the error changes to [Errno -3] Temporary failure in name resolution from [Errno -2] Name or service not known

Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/urllib/request.py", line 1317, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/opt/conda/lib/python3.7/http/client.py", line 1229, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/opt/conda/lib/python3.7/http/client.py", line 1275, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/opt/conda/lib/python3.7/http/client.py", line 1224, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/opt/conda/lib/python3.7/http/client.py", line 1016, in _send_output
    self.send(msg)
  File "/opt/conda/lib/python3.7/http/client.py", line 956, in send
    self.connect()
  File "/opt/conda/lib/python3.7/http/client.py", line 1384, in connect
    super().connect()
  File "/opt/conda/lib/python3.7/http/client.py", line 928, in connect
    (self.host,self.port), self.timeout, self.source_address)
  File "/opt/conda/lib/python3.7/socket.py", line 707, in create_connection
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
  File "/opt/conda/lib/python3.7/socket.py", line 748, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "SuicidesIndia.py", line 2, in <module>
    html = urllib.request.urlopen('https://google.com').read()
  File "/opt/conda/lib/python3.7/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/opt/conda/lib/python3.7/urllib/request.py", line 525, in open
    response = self._open(req, data)
  File "/opt/conda/lib/python3.7/urllib/request.py", line 543, in _open
    '_open', req)
  File "/opt/conda/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/opt/conda/lib/python3.7/urllib/request.py", line 1360, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/opt/conda/lib/python3.7/urllib/request.py", line 1319, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno -3] Temporary failure in name resolution>

[gollux]

Are you using --share-net now?

[rohanraarora]

@gollux I get the same error irrespective of the --share-net option. This is the exact command that I am running:

isolate --cg -b 6 -o out -r err -M meta -t 10 -p --cg-mem 256000 --share-net --full-env --dir=/etc/:rw --dir=/opt/:rw --run /opt/conda/bin/python3 a.py

P.S. I am using isolate inside a docker container which is using the anaconda3 as the base image. If I run the same program inside container without using isolate ( i.e. /opt/conda/bin/python3 a.py ) , I get the result as expected.

[bblackham]

What does your /etc/resolv.conf point to? In some setups, it's a symlink elsewhere - you'll need to mount the filesystem of that symlink too. Failing that, can you run the command prefixed with sudo strace -f -o strace.log and post the resulting strace.log ?

[rohanraarora]

Resolv.conf doesn't have any symlink. Here's the strace.log output. Thanks in advance.

[bblackham]

Apparently it's empty though. Is that expected?

13    open("/etc/resolv.conf", O_RDONLY|O_CLOEXEC) = 4
13    fstat(4, {st_mode=S_IFREG|0755, st_size=0, ...}) = 0

[rohanraarora]

@bblackham If I bash into the container and cat the resolv.conf file, I get a namespace and I am also able to run the python code successfully if I run it without isolate. It fetches the html and prints it. But /etc/resolv.conf appears to be empty if isolates tries to access it.

If I cat /etc/resolv.conf , I get:

# This file is included on the metadata iso
nameserver 192.168.65.1

and on runningls -la /etc/resolv.conf , I get: -rw-r--r-- 1 root root 68 Jan 22 17:10 /etc/resolv.conf

and if I run isolate --cg -b 6 -o out -r err -M meta -t 10 -p --cg-mem 256000 --share-net --full-env --dir=/etc/:rw --dir=/opt/:rw --run /bin/cat /etc/resolv.conf , I get empty output

Any idea?

[rohanraarora]

I have created a docker image and dockerfile to reproduce this issue. Here's the Dockerfile and docker image.

Steps to reproduce:

1. Bash into the container using docker run --privileged -i -t --rm rohanraarora/isolate_issue:latest /bin/bash

2. Print the contents of /etc/resolv.conf file using /bin/cat /etc/resolv.conf. It should print the namespace

3. Use isolate to output resolv.conf file into out.txt

   ISOLATE_PATH=`isolate --cg -b 0 --init`
   cd $ISOLATE_PATH/box/
   isolate --cg -b 0 -o out.txt -r err.txt -M meta.txt -t 10 -p --cg-mem 256000 --share-net --full-env --dir=/etc/:rw --run /bin/cat /etc/resolv.conf

4. out.txt is empty

[bblackham]

Thanks for the easy repro steps. Here is the problem:

# mount|grep etc
/dev/mapper/ubuntu--vg-root on /etc/resolv.conf type ext4 (rw,relatime,errors=remount-ro)

In the docker container, /etc/resolv.conf is a bind mount of the host resolv.conf. I didn’t even know you could bind mount files until now. I think the solution here is for isolate to support recursive bind mounts (but perhaps as a non-default option to avoid breaking any existing setups).

[rohanraarora]

Thanks for looking into the issue and is there any workaround for the time being?

[bblackham]

If you are willing to just modify the isolate source, this patch will make isolate's bind mounts recursive:

--- a/rules.c
+++ b/rules.c
@@ -396,7 +396,7 @@ apply_dir_rules(int with_defaults)
        }
       else
        {
-         mount_flags |= MS_BIND | MS_NOSUID;
+         mount_flags |= MS_BIND | MS_NOSUID | MS_REC;
          msg("Binding %s on %s (flags %lx)\n", out, in, mount_flags);

          /*

[rohanraarora]

@bblackham Yup this worked. Thanks a ton :)

[zopieux]

@gollux Seems like we need an option to recursively bind mount?

[gollux]

I do not see much reasons for not binding recursively in all cases. Do you?

[bblackham]

I can imagine cases where recursive mounts may not be desirable (e.g. /var without /var/run), so making it configurable could be useful. Apart from potentially breaking existing setups, I think recursive mounts would be a sensible default.

[zopieux]

Well, sure, make it a default if that's okay.