Unix socket support

[lantos1618]

Hey treeform, I just pushed OPT_UNIX_SOCK_PATH into libcurl.nim https://github.com/Araq/libcurl/commit/7a420498f60a31d99fc8513886ce36c4e8c3a4ae

Is it possible to get unix socket support in libcurl.nim?

I'm using something like this which works.

type CurlWriteFnHandle* = proc (   
                buffer: cstring,
                size: int,
                count: int,
                outstream: pointer): int {. gcsafe, locks: 0.}

proc defaultCurlWriteFn(
        buffer: cstring,
        size: int,
        count: int,
        outstream: pointer
    ): int =
    let outbuf = cast[ref string](outstream)
    outbuf[] &= buffer
    result = size * count

# can swap this to puppy if puppy supported sockets
proc request*(
            req: Request,
            path: string,
            httpMethod: HttpMethod,
            body: string,
            multipartData: MultipartData,
            useCurl: bool = false,
            headers: HttpHeaders = newHttpHeaders({"Accept": "application/json",
                "Content-Type": "application/json"}),
            curlWriteFn: CurlWriteFnHandle = defaultCurlWriteFn
        ): string =

        let useUnix = docker.baseUrl.startsWith("unix://")
        if useCurl or useUnix:
            let webData: ref string = new string
            let socketPath = ctx.baseUrl[7..ctx.baseUrl.high]
            let curl = easy_init()

            if useUnix:
                discard curl.easy_setopt(OPT_UNIX_SOCKET_PATH,
                        socketPath.cstring)
                discard curl.easy_setopt(OPT_URL, path.cstring)
            else:
                discard curl.easy_setopt(OPT_URL, (docker.baseUrl & "/" & path).cstring)

            discard curl.easy_setopt(OPT_CUSTOMREQUEST, ($httpMethod).cstring)
            discard curl.easy_setopt(OPT_POSTFIELDSIZE, body.len)
            discard curl.easy_setopt(OPT_POSTFIELDS, body)

            var headerChunk: Pslist
            headerChunk = headerChunk.slist_append("Accept: application/json")
            headerChunk = headerChunk.slist_append("Content-Type: application/json")
            discard curl.easy_setopt(OPT_HTTPHEADER, headerChunk);

            discard curl.easy_setopt(OPT_WRITEDATA, webData)
            discard curl.easy_setopt(OPT_WRITEFUNCTION, curlWriteFn)
            discard curl.easy_setopt(OPT_VERBOSE, 1)
            let ret = curl.easy_perform()

            if ret != E_OK:
                raise newException(Defect, $ret & $easy_strerror(ret))

            result = webData[]
            curl.easy_reset()

[treeform]

I have never used Unix sockets, therefore, I do not think it is a frequently used feature. Do Windows or macOS have Unix socket equivalents? I do not want to increase the size of the library when I do not see a clear benefit. I do not think I will be adding Unix sockets at this time. Sorry.

[lantos1618]

I have never used Unix sockets, therefore, I do not think it is a frequently used feature. Do Windows or macOS have Unix socket equivalents? I do not want to increase the size of the library when I do not see a clear benefit. I do not think I will be adding Unix sockets at this time. Sorry.

Macos does (example; docker macos uses simlink), windows supports it now. It's used quite heavily in unix programs, ( I think most daemons use it, sql, docker...)

Araq merged my change in libcurl.nim to support it but makes sense not to do it if I'm the only one asking. ended up patching std/net.nim std/asyncnet.nim to make it work and a custom httpclient.nim

Thanks for considering it though :)

[eclairevoyant]

MacOS and Linux have supported Unix sockets forever, and as of Windows 10, Windows actually supports Unix sockets too. It's a very common protocol for interprocess communication.

[theAkito]

UNIX sockets are a thing of the past. No need to hold onto it. The only reason they are not uncommon is because of how certain low-level Linux things sadly work.

If Linux would be modern, there would be no UNIX sockets.

[eclairevoyant]

Look, I'm not here to argue philosophical discussions. I don't even have a problem if the maintainer doesn't want to include Unix socket support in this library. I commented to point out that a) there is cross-platform support and b) it's not an infrequently used functionality. And I wouldn't bother commenting if I wasn't planning on using them - either way I'll likely end up implementing support for myself.

[lantos1618]

UNIX sockets are a thing of the past. No need to hold onto it. The only reason they are not uncommon is because of how certain low-level Linux things sadly work.

If Linux would be modern, there would be no UNIX sockets.

I don't think this this is exactly true, I believe the following still use unix sockets

• Docker
• PhpFpm
• Mysql
• X11
• Nginx
• PulseAudio (?)
• Systemd (?)

This was a while ago but if IRC I ended up finding that there isn't really anything that needed to be changed so that puppy supported sockets. The issue with the sockets ended up being a problem in the std/net library. It was just not dialling correctly.

https://github.com/nim-lang/Nim/issues/20062

[Zectbumo]

You why use unix sockets?

ChatGPT

Unix sockets, also known as IPC (Inter-Process Communication) sockets, provide a way for processes to communicate with each other on the same host. They are a feature of Unix-like operating systems, including Linux and BSD variants. Here are several reasons why Unix sockets are used:

Efficiency: Communication through Unix sockets is generally faster than using network sockets (TCP/IP), as it doesn't involve the network stack and therefore has lower overhead. Data is passed directly between processes, reducing latency.

File system based: Unix sockets use the file system as their address name space. This means they are accessed using pathnames like files, which can be a more intuitive way of managing IPC resources for some applications.

Security: Unix sockets can utilize the file system's permission model to restrict access to the socket file, providing a straightforward way to control which users or processes can communicate with a service.

Support for both stream and datagram communication: Unix sockets support both connection-oriented (stream) and connectionless (datagram) communication, giving developers the flexibility to choose the most appropriate communication model for their application.

Simplifies local inter-process communication: For local IPC, using Unix sockets can simplify the development and deployment of applications by avoiding the need to assign port numbers, manage network configurations, or deal with potential network-related issues.

Support for passing file descriptors: Unix sockets allow file descriptors to be passed between processes. This feature can be used to share access to open files or sockets, enabling patterns like the worker process model where a master process listens for connections and passes them to worker processes for handling.

Ecosystem and tool support: Given their long history and utility, Unix sockets are well-supported across a wide range of programming languages and frameworks, making them a reliable choice for IPC on Unix-like systems.

Avoids network stack bottlenecks: Since Unix sockets don't go through the network stack, they can avoid bottlenecks or limitations associated with it, such as port exhaustion or TCP/IP overhead, making them ideal for high-performance local communications.

In summary, Unix sockets offer a high-performance, secure, and developer-friendly mechanism for IPC in Unix-like environments. They are particularly useful for local server-client interactions, microservices running on the same host, and any application where minimizing communication latency is crucial.

Unix sockets can utilize the file system's permission model to restrict access to the socket file, providing a straightforward way to control which users or processes can communicate with a service. ^^^^^^^^^^^^ this you can't do with INET.