python3 failing with a Unknown schema error when I attempt to connect to a non-default unix domain socket.

[Dweller]

I'm currently running a snap lxd 5.21 on centos8 and a pip installed pylxd of version 2.3.1

I've a LXD container which runs a pylxd script and that connects to the hosts LXD server via a proxy.

Currently I have to use a LXD proxy to mirror the default location of the unix.socket to get this to work.

lxdsocket: bind: container connect: unix:/var/snap/lxd/common/lxd/unix.socket gid: "1001" listen: unix:/var/snap/lxd/common/lxd/unix.socket mode: "0660" type: proxy uid: "0"

As soon as I attempt to override the pylxd module the script bails with an unknown schema error. Looks like the requests module wants a unix: or http: prefix. However adding one doesn't help.

e.g. Client('/var/lxd/host.sock')

Is there a fix for this..

[simondeziel]

@Dweller if you can, please provide your pylxd script to allow us to easily replicate the issue. Thanks

[Dweller]

from pylxd import Client, exceptions

lxd = Client('/dev/lxd/sock')

'

Traceback (most recent call last):
  File "/tmp/lxd.py", line 3, in <module>
    lxd = Client('/dev/lxd/sock')
  File "/usr/lib/python3/dist-packages/pylxd/client.py", line 308, in __init__
    response = self.api.get()
  File "/usr/lib/python3/dist-packages/pylxd/client.py", line 158, in get
    response = self.session.get(self._api_endpoint, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 557, in get
    return self.request('GET', url, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 530, in request
    prep = self.prepare_request(req)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 458, in prepare_request
    p.prepare(
  File "/usr/lib/python3/dist-packages/requests/models.py", line 316, in prepare
    self.prepare_url(url, params)
  File "/usr/lib/python3/dist-packages/requests/models.py", line 390, in prepare_url
    raise MissingSchema(error)
requests.exceptions.MissingSchema: Invalid URL '/dev/lxd/sock/1.0': No schema supplied. Perhaps you meant http:///dev/lxd/sock/1.0?

[simondeziel]

It works for me when providing a valid path to a Unix socket:

$ python3
Python 3.10.12 (main, Jul 29 2024, 16:56:48) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pylxd import Client, exceptions
>>> lxd = Client("/var/snap/lxd/common/lxd/unix.socket")
>>> for i in lxd.instances.all():
...     print (i.name)
... 
autopkg
c1
c6
cs50-python
jammy-builder
lxd-core18
lxd-imagebuilder
noble-builder
wine-games

However, providing a path to a non-existent socket gives the same error you got:

>>> lxd = Client("/var/snap/lxd/common/lxd/unix.socket2")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/sdeziel/git/pylxd/pylxd/client.py", line 410, in __init__
    response = self.api.get()
  File "/home/sdeziel/git/pylxd/pylxd/client.py", line 206, in get
    response = self.session.get(self._api_endpoint, *args, **kwargs)
  File "/home/sdeziel/.local/lib/python3.10/site-packages/requests/sessions.py", line 602, in get
    return self.request("GET", url, **kwargs)
  File "/home/sdeziel/.local/lib/python3.10/site-packages/requests/sessions.py", line 575, in request
    prep = self.prepare_request(req)
  File "/home/sdeziel/.local/lib/python3.10/site-packages/requests/sessions.py", line 486, in prepare_request
    p.prepare(
  File "/home/sdeziel/.local/lib/python3.10/site-packages/requests/models.py", line 368, in prepare
    self.prepare_url(url, params)
  File "/home/sdeziel/.local/lib/python3.10/site-packages/requests/models.py", line 439, in prepare_url
    raise MissingSchema(
requests.exceptions.MissingSchema: Invalid URL '/var/snap/lxd/common/lxd/unix.socket2/1.0': No scheme supplied. Perhaps you meant https:///var/snap/lxd/common/lxd/unix.socket2/1.0?

[Dweller]

The socket exists and is openable.. Mine which is inside a LXD instance.. You'll note that the error message isn't that it can't open the socket its that it doesn't have a unix or http schema prefix.. Not that adding on of those helps..

Traceback (most recent call last):
  File "/home/atcore/lxd2.py", line 4, in <module>
    lxd = Client('/var/snap/lxd/common/lxd/unix.socket')
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/pylxd/client.py", line 308, in __init__
    response = self.api.get()
               ^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/pylxd/client.py", line 158, in get
    response = self.session.get(self._api_endpoint, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 602, in get
    return self.request("GET", url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 575, in request
    prep = self.prepare_request(req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 486, in prepare_request
    p.prepare(
  File "/usr/lib/python3/dist-packages/requests/models.py", line 368, in prepare
    self.prepare_url(url, params)
  File "/usr/lib/python3/dist-packages/requests/models.py", line 439, in prepare_url
    raise MissingSchema(
requests.exceptions.MissingSchema: Invalid URL '/var/snap/lxd/common/lxd/unix.socket/1.0': No scheme supplied. Perhaps you meant https:///var/snap/lxd/common/lxd/unix.socket/1.0?

atcore@avmanager-shards:~$ nc -U /dev/lxd/sock
GET / HTTP/1.1

HTTP/1.1 400 Bad Request: missing required Host header
Content-Type: text/plain; charset=utf-8
Connection: close

400 Bad Request: missing required Host headeratcore@avmanager-shards:~$

python3 lxd2.py
Traceback (most recent call last):
  File "/home/atcore/lxd2.py", line 4, in <module>
    lxd = Client('/dev/lxd/sock')
          ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/pylxd/client.py", line 308, in __init__
    response = self.api.get()
               ^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/pylxd/client.py", line 158, in get
    response = self.session.get(self._api_endpoint, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 602, in get
    return self.request("GET", url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 575, in request
    prep = self.prepare_request(req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 486, in prepare_request
    p.prepare(
  File "/usr/lib/python3/dist-packages/requests/models.py", line 368, in prepare
    self.prepare_url(url, params)
  File "/usr/lib/python3/dist-packages/requests/models.py", line 439, in prepare_url
    raise MissingSchema(
requests.exceptions.MissingSchema: Invalid URL '/dev/lxd/sock/1.0': No scheme supplied. Perhaps you meant https:///dev/lxd/sock/1.0?

[Dweller]

What strange is if I call Client() without an endpoint argument it works.. But if I specify the path of the endpoint /var/snap/lxd/common/lxd/unix.socket it doesn't. Yet internally I can see its using that path..

So it would appear that the processing of the endpoint vs the default value is different.

I've confirmed that the path is correct by cuting and pasting the one I was using in my code and using it in a command line curl command. So I know the permissions etc. are correct.

[Dweller]

I wrote some python to open a unix domain socket directly to the unix.socket and that works... So confirmed no issues with the permissions.

Apart from debuging the pylxd connection code I'm not sure whats left?

[simondeziel]

It just occurred to me that your pylxd script is being run inside an instance and you are using /dev/lxd/sock:

from pylxd import Client, exceptions

lxd = Client('/dev/lxd/sock')

Inside instances, this socket path belongs to dev-lxd which is not a socket that exposes the usual LXD API.