search

cosmos72 / gomacro

Interactive Go interpreter and debugger with REPL, Eval, generics and Lisp-like macros
Mozilla Public License 2.0
2.18k stars 94 forks source link
debugger eval generics golang interpreter macros repl

readme

gomacro - interactive Go interpreter and debugger with generics and macros

gomacro is an almost complete Go interpreter, implemented in pure Go. It offers both an interactive REPL and a scripting mode, and does not require a Go toolchain at runtime (except in one very specific case: import of a 3rd party package at runtime).

It has two dependencies beyond the Go standard library: github.com/peterh/liner and golang.org/x/tools/go/packages

Gomacro can be used as:

Installation

Prerequites

Supported platforms

Gomacro is pure Go, and in theory it should work on any platform supported by the Go compiler. The following combinations are tested and known to work:

How to install

The command

  go install github.com/cosmos72/gomacro@latest

downloads, compiles and installs gomacro and its dependencies

Current Status

Almost complete.

The main limitations and missing features are:

The documentation also contains the full list of features and limitations

Extensions

Compared to compiled Go, gomacro supports several extensions:

and slightly relaxed checks:

Examples

Some short, notable examples - to run them on non-Linux platforms, see Importing packages first.

plot mathematical functions

simple mandelbrot web server

Further examples are listed by Gophernotes

Importing packages

Gomacro supports the standard Go syntax import, including package renaming. Examples:

import "fmt"
import (
    "io"
    "net/http"
    r "reflect"
)

Third party packages - i.e. packages not in Go standard library - can also be imported with the same syntax.

Extension: unpublished packages can also be imported from a local filesystem directory (implemented on 2022-05-28). Supported syntaxes are:

import (
     "."                             // imports the package in current directory
     ".."                            // imports the package in parent directory
     "./some/relative/path"          // "./"  means relative to current directory
     "../some/other/relative/path"   // "../" means relative to parent directory
     "/some/absolute/path"           // "/"   means absolute
)

For an import to work, you usually need to follow its installation procedure: sometimes there are additional prerequisites to install, and the typical command go get PACKAGE-PATH may or may not be needed.

The next steps depend on the system you are running gomacro on:

Linux, Mac OS X and *BSD

If you are running gomacro on Linux, Mac OS X or *BSD, import will then just work: it will automatically download, compile and import a package. Example:

$ gomacro
[greeting message...]

gomacro> import ( "gonum.org/v1/gonum/floats"; "gonum.org/v1/plot" )
// debug: running "go get gonum.org/v1/plot gonum.org/v1/gonum/floats" ...
go: downloading gonum.org/v1/gonum v0.12.0
go: downloading gonum.org/v1/plot v0.12.0
[ more "go: downloading " messages for dependencies...]
go: added gonum.org/v1/gonum v0.12.0
go: added gonum.org/v1/plot v0.12.0
// debug: running "go mod tidy" ...
go: downloading golang.org/x/exp v0.0.0-20220827204233-334a2380cb91
go: downloading github.com/go-fonts/latin-modern v0.2.0
go: downloading rsc.io/pdf v0.1.1
go: downloading github.com/go-fonts/dejavu v0.1.0
// debug: compiling plugin "/home/max/go/src/gomacro.imports/gomacro_pid_44092/import_1" ...

gomacro> floats.Sum([]float64{1,2,3})
6       // float64

Note: internally, gomacro will compile and load a single Go plugin containing the exported declarations of all the packages listed in import ( ... ).

The command go mod tidy is automatically executed before compiling the plugin, and it tries - among other things - to resolve any version conflict due to different versions of the same package being imported directly (i.e. listed in import ( ... )) or indirectly (i.e. as a required dependency).

Go plugins are currently supported only on Linux and Mac OS X.

WARNING On Mac OS X, never execute strip gomacro: it breaks plugin support, and loading third party packages stops working.

Other systems

On all other systems as Windows, Android and *BSD you can still use import, but there are more steps: you need to manually download the package, and you also need to recompile gomacro after the import (it will tell you). Example:

$ go get gonum.org/v1/plot
$ gomacro
[greeting message...]

gomacro> import "gonum.org/v1/plot"
// warning: created file "/home/max/go/src/github.com/cosmos72/gomacro/imports/thirdparty/gonum_org_v1_plot.go", recompile gomacro to use it

Now quit gomacro, recompile and reinstall it:

gomacro> :quit
$ go install github.com/cosmos72/gomacro

Finally restart it. Your import is now linked inside gomacro and will work:

$ gomacro
[greeting message...]

gomacro> import "gonum.org/v1/plot"
gomacro> plot.New()
&{...} // *plot.Plot
<nil>  // error

Note: if you need several packages, you can first import all of them, then quit and recompile gomacro only once.

Generics

gomacro contains two alternative, experimental versions of Go generics:

The second version of generics "CTI" is enabled by default in gomacro.

They are in beta status, and at the moment only generic types and functions are supported. Syntax and examples:

// declare a generic type with two type arguments T and U
type Pair#[T,U] struct {
    First T
    Second U
}

// instantiate the generic type using explicit types for T and U,
// and create a variable of such type.
var pair Pair#[complex64, struct{}]

// equivalent:
pair := Pair#[complex64, struct{}] {}

// a more complex example, showing higher-order functions
func Transform#[T,U](slice []T, trans func(T) U) []U {
    ret := make([]U, len(slice))
    for i := range slice {
        ret[i] = trans(slice[i])
    }
    return ret
}
Transform#[string,int] // returns func([]string, func(string) int) []int

// returns []int{3, 2, 1} i.e. the len() of each string in input slice:
Transform#[string,int]([]string{"abc","xy","z"}, func(s string) int { return len(s) })

Contracts specify the available methods of a generic type. For simplicity, they do not introduce a new syntax or new language concepts: contracts are just (generic) interfaces. With a tiny addition, actually: the ability to optionally indicate the receiver type.

For example, the contract specifying that values of type T can be compared with each other to determine if the first is less, equal or greater than the second is:

type Comparable#[T] interface {
    // returns -1 if a is less than b
    // returns  0 if a is equal to b
    // returns  1 if a is greater than b
    func (a T) Cmp(b T) int
}

A type T implements Comparable#[T] if it has a method func (T) Cmp(T) int. This interface is carefully chosen to match the existing methods of *math/big.Float, *math/big.Int and *math/big.Rat. In other words, *math/big.Float, *math/big.Int and *math/big.Rat already implement it.

What about basic types as int8, int16, int32, uint... float*, complex* ... ? Gomacro extends them, automatically adding many methods equivalent to the ones declared on *math/big.Int to perform arithmetic and comparison, including Cmp which is internally defined as (no need to define it yourself):

func (a int) Cmp(b int) int {
    if a < b {
        return -1
    } else if a > b {
        return 1
    } else {
        return 0
    }
}

Thus the generic functions Min and Max can be written as

func Min#[T: Comparable] (a, b T) T {
    if a.Cmp(b) < 0 { // also <= would work
        return a
    }
    return b
}
func Max#[T: Comparable] (a, b T) T {
    if a.Cmp(b) > 0 { // also >= would work
        return a
    }
    return b
}

Where the syntax #[T: Comparable] or equivalently #[T: Comparable#[T]] indicates that T must satisfy the contract (implement the interface) Comparable#[T]

Such functions Min and Max will then work automatically for every type T that satisfies the contract (implements the interface) Comparable#[T]:\ all basic integers and floats, plus *math/big.Float, *math/big.Int and *math/big.Rat, plus every user-defined type T that has a method func (T) Cmp(T) int

If you do not specify the contract(s) that a type must satisfy, generic functions cannot access the fields and methods of a such type, which is then treated as a "black box", similarly to interface{}.

Two values of type T can be added if T has an appropriate method. But which name and signature should we choose to add values? Copying again from math/big, the method we choose is func (T) Add(T,T) T If receiver is a pointer, it will be set to the result - in any case, the result will also be returned. Similarly to Comparable, the contract Addable is then

type Addable#[T] interface {
    // Add two values a, b and return the result.
    // If recv is a pointer, it must be non-nil
    // and it will be set to the result
    func (recv T) Add(a, b T) T
}

With such a contract, a generic function Sum is quite straightforward:

func Sum#[T: Addable] (args ...T) T {
    // to create the zero value of T,
    // one can write 'var sum T' or equivalently 'sum := T()'
    // Unluckily, that's not enough for math/big numbers, which require
    // the receiver of method calls to be created with a function `New()`
    // Once math/big numbers have such method, the following
    // will be fully general - currently it works only on basic types.
    sum := T().New()

    for _, elem := range args {
        // use the method T.Add(T, T)
        //
        // as an optimization, relevant at least for math/big numbers,
        // also use sum as the receiver where result of Add will be stored
        // if the method Add has pointer receiver.
        //
        // To cover the case where method Add has instead value receiver,
        // also assign the returned value to sum
        sum = sum.Add(sum, elem)
    }
    return sum
}
Sum#[int]         // returns func(...int) int
Sum#[int] (1,2,3) // returns int(6)

Sum#[complex64]                 // returns func(...complex64) complex64
Sum#[complex64] (1.1+2.2i, 3.3) // returns complex64(4.4+2.2i)

Sum#[string]                         // returns func(...string) string
Sum#[string]("abc.","def.","xy","z") // returns "abc.def.xyz"

Partial and full specialization of generics is not supported in CTI generics, both for simplicity and to avoid accidentally providing Turing completeness at compile-time.

Instantiation of generic types and functions is on-demand.

Current limitations:

Debugger

Since version 2.6, gomacro also has an integrated debugger. There are three ways to enter it:

In all cases, execution will be suspended and you will get a debug> prompt, which accepts the following commands:\ step, next, finish, continue, env [NAME], inspect EXPR, list, print EXPR-OR-STATEMENT

Also,

Only interpreted statements can be debugged: expressions and compiled code will be executed, but you cannot step into them.

The debugger is quite new, and may have some minor glitches.

Why it was created

First of all, to experiment with Go :)

Second, to simplify Go code generation tools (keep reading for the gory details)

Problem: "go generate" and many other Go tools automatically create Go source code from some kind of description - usually an interface specifications as WSDL, XSD, JSON...

Such specification may be written in Go, for example when creating JSON marshallers/unmarshallers from Go structs, or in some other language, for example when creating Go structs from JSON sample data.

In both cases, a variety of external programs are needed to generate Go source code: such programs need to be installed separately from the code being generated and compiled.

Also, Go is currently lacking generics (read: C++-like templates) because of the rationale "we do not yet know how to do them right, and once you do them wrong everybody is stuck with them"

The purpose of Lisp-like macros is to execute arbitrary code while compiling, in particular to generate source code.

This makes them very well suited (although arguably a bit low level) for both purposes: code generation and C++-like templates, which are a special case of code generation - for a demonstration of how to implement C++-like templates on top of Lisp-like macros, see for example the project https://github.com/cosmos72/cl-parametric-types from the same author.

Building a Go interpreter that supports Lisp-like macros, allows to embed all these code-generation activities into regular Go source code, without the need for external programs (except for the interpreter itself).

As a free bonus, we get support for Eval()

LEGAL

Gomacro is distributed under the terms of Mozilla Public License 2.0 or any later version.