Using usql as a library (or subprocess)

[astoff]

It would be interesting if one could use usql with other frontends than a terminal, for example as a Jupyter kernel or as an IDE integration (I myself would like to create an Emacs database REPL).

After having a brief look at the code, I'm not sure this can be easily done at this point. Concretely, the following API would be needed:

1. (essential) A function that takes a command as a string and evaluates it, returning the result as a string.
2. (highly desirable) A function that takes a command as a string and a cursor position, and returns possible code completions.
3. (highly desirable) A function that takes a partially typed command and says if it's complete; in other words, whether typing that command in the terminal would execute something or create a continuation prompt.
4. (nice to have) A function that takes a command as a string and a cursor position, and returns some information about the thing at point (aka "hover" help).

Another option (perhaps even better) would be to add a few extra backslash commands to allow a frontend (IDE or similar program) to run usql as subprocess and send and receive data in a way that can be easily and reliably parsed.

• Function 1. is more or less covered by \i -.
• For function 2, one could add a new \complete, which could work as follows: the frontend process sends usql the line \complete <character offset> LF followed by the current user input, followed by EOF; then usql prints possible completions, one per line.
• Functions 3. and 4. can be covered by analogous backslash commands.

[kenshaw]

So: the usql code is very modular, as its constiuent packages don't have many interdependencies (thank you Go!). You could easily conjure up a different frontend that uses the Handler to do exactly what you've described. Please see the main.go and args.go, it's really a trivial amount of code involved.

With regards to completions and a \complete command, please note that completions in usql aren't complete (no pun intended). They currently only work on a few databases, and only complete some entity names. As such, I can't really see how the benefit of a \complete command. You can dig into the code and use the actual completer engine, as it's also modular and completely separable from the usql code.

[astoff]

@kenshaw: Thanks for the explanation. What you describe should work indeed.

Still regarding the use of usql as a library, perhaps you can help me with a question about the build tags (I'm new to Go).

The installation instructions involving go install -tags somedriver github.com/xo/usql work as expected. However, if I create a simple program that imports usql like the one below, the resulting binary is quite small and prints 0; that is, no drivers are linked.

package main

import (
    "fmt"
    "github.com/xo/usql/drivers"
)

func main() {
    available := drivers.Available()
    fmt.Printf("%v\n", len(available))
}

How am I supposed to pass build tags to usql when it is used as a dependency?

[kenshaw]

You don't need the build tags when building it as a library. That's a hack I whipped up to change the configuration at build time, but without needing to modify the source code.

You can manually import any of the drivers you want, via the following:

package main

import (
    "fmt"
    "sort"

    "github.com/xo/usql/drivers"
    _ "github.com/xo/usql/drivers/mysql"
    "golang.org/x/exp/maps"
)

func main() {
    available := drivers.Available()
    keys := maps.Keys(available)
    sort.Strings(keys)
    for _, key := range keys {
        fmt.Println(key)
    }
}

You can import any (or none) of the drivers you'd like to import. Good luck!

[astoff]

Thanks for the hints! My usql integration for Emacs is working; for the record, it can be found here.

As a friendly piece of feedback, I wanted to say again that you like the idea of supporting other types of UIs, it would be nice to provide a dedicated "eval" function. In my case, it was enough to hack into the usql read-eval-print loop and manipulate the strings it receives, but a different situation might have required to copy and adapt the whole of Handler.Run, which is a rather long function. Conceptually, it does both the "E" and "L" of REPL at once.

[kenshaw]

Handler.Run is the entire REPL. A third-party package (github.com/gohxs/readline) is used to actually do the line reading, and other packages within usql handle the bulk of the operations, but it's that Run func that the main hands off to after setting up the environment that does the majority of the the "work".