search

cosmos72 / gomacro

Interactive Go interpreter and debugger with REPL, Eval, generics and Lisp-like macros
Mozilla Public License 2.0
2.18k stars 93 forks source link

can not import 3rd party packages #146

Closed nikolaydubina closed 10 months ago

nikolaydubina commented 1 year ago 
nikolaydubina@Nikolays-MacBook-Pro Workspace % gomacro
// Welcome to gomacro. Type :help for help, :copy for copyright and license.
// This is free software with ABSOLUTELY NO WARRANTY.
gomacro> import ( "gonum.org/v1/gonum/floats"; "gonum.org/v1/plot" )
// debug: running "go get gonum.org/v1/gonum/floats gonum.org/v1/plot" ...
go: added git.sr.ht/~sbinet/gg v0.4.1
go: added github.com/ajstarks/svgo v0.0.0-20211024235047-1546f124cd8b
go: added github.com/go-fonts/liberation v0.3.1
go: added github.com/go-latex/latex v0.0.0-20230307184459-12ec69307ad9
go: added github.com/go-pdf/fpdf v0.8.0
go: added github.com/golang/freetype v0.0.0-20170609003504-e2365dfdc4a0
go: added golang.org/x/image v0.7.0
go: added golang.org/x/text v0.9.0
go: added gonum.org/v1/gonum v0.13.0
go: added gonum.org/v1/plot v0.13.0
// warning: package "gonum.org/v1/gonum/floats" exports zero constants, functions, types and variables
// warning: package "gonum.org/v1/plot" exports zero constants, functions, types and variables
gomacro>

go env

GO111MODULE=""
GOARCH="arm64"
GOBIN="//Users/nikolaydubina/go/bin"
GOCACHE="/Users/nikolaydubina/Library/Caches/go-build"
GOENV="/Users/nikolaydubina/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="arm64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/nikolaydubina/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/nikolaydubina/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_arm64"
GOVCS=""
GOVERSION="go1.20.4"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-O2 -g"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-O2 -g"
CGO_FFLAGS="-O2 -g"
CGO_LDFLAGS="-O2 -g"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/dt/1y99_j6s0yj7y151_dr026gh0000gn/T/go-build2524980313=/tmp/go-build -gno-record-gcc-switches -fno-common"
SpikeWong commented 1 year ago

faced same issue

SpikeWong commented 1 year ago

Maybe I found the cause, it seems that this library is not compatible with go 1.20. When I tried to run it with go 1.19, it worked

cosmos72 commented 1 year ago

confirmed, it seems imports are broken when gomacro is compiled with go >= 1.20 ouch.

cosmos72 commented 1 year ago

[UPDATE] I checked this bug by compiling and running gomacro with various Go toolchains. The results are:

So Debian is actively applying a patch that fixes this issue. Now I need to find such patch and check why it was not merged into the upstream Go toolchain at go.dev/dl/

larsks commented 1 year ago

@cosmos72 I just built go from http://deb.debian.org/debian/pool/main/g/golang-1.20/golang-1.20_1.20.7.orig.tar.gz without applying any of the debian patches (and then reinstalled gomacro using go install ...), and it seems to work:

$ gomacro
gomacro> import "context"
gomacro> import "github.com/docker/docker/client"
[...]
// debug: compiling plugin "/home/lars/go/src/gomacro.imports/gomacro_pid_460679/import_1" ...
gomacro> import "github.com/docker/docker/api/types"
// debug: running "go get github.com/docker/docker/api/types" ...
[...]
// debug: compiling plugin "/home/lars/go/src/gomacro.imports/gomacro_pid_460679/import_2" ...
gomacro> dc, _ := client.NewClientWithOpts(client.FromEnv)
gomacro> dc.ContainerList(context.Background(), types.ContainerListOptions{})
[...list of containers here...]
<nil>   // error

The patches included in the package in any case don't sound relevant:

0001-Disable-test-for-UserHomeDir.patch
0002-Fix-Lintian-warnings-about-wrong-interpreter-path.patch
0003-cmd-dist-increase-default-timeout-scale-for-arm.patch
0004-skip-userns-test-in-schroot-as-well.patch

I see the same behavior if I rebuild go from https://go.dev/dl/go1.20.7.src.tar.gz and then re-install gomacro.

cosmos72 commented 1 year ago

Thanks for the cross checks! Do I understand correctly that as last check you downloaded Go toolchain sources from https://go.dev/dl/go1.20.7.src.tar.gz, you compiled them (with some other Go toolchain), and using it to compile and run gomacro this bug disappeared?

larsks commented 1 year ago

I think so. I'm trying to retest while being sure to explicitly clear the build cache each time because I'm having some reproducibility issues.

larsks commented 1 year ago

The problem appears to be independent of the version of Go used to build gomacro; what matters is the version of Go in $PATH at runtime.

I built and installed gomacro using Go versions 1.19.4, 1.20, and 1.20.7 (installed from upstream release binaries from https://go.dev/dl/), and then tried a third party import with different versions of Go in $PATH.

Here's the test script:

#!/bin/sh

set -e

topdir=$PWD
for buildversion in 1.19.4 1.20 1.20.7; do
    (
    PATH=/usr/local/pkg/go-$buildversion/bin:$PATH

    {
    go env GOCACHE
    go env GOMODCACHE
    go env GOPATH
    } | sudo xargs rm -rf

    echo "installing gomacro with Go $(go version)"
    go install github.com/cosmos72/gomacro@latest
    GOMACRO="$(go env GOPATH)/bin/gomacro"

    for goversion in 1.19.4 1.20 1.20.7; do
        workdir=$(mktemp -d "$PWD/gotestXXXXXX")
        (
        cd "$workdir"
        GOCACHE=$workdir/go/cache
        GOMODCACHE=$workdir/go/mod
        export GOCACHE GOMODCACHE
        PATH=/usr/local/pkg/go-$goversion/bin:$PATH
        $GOMACRO <<-EOF > test.out 2> test.err
        import "github.com/docker/docker/client"
        EOF
        {
        if grep -q "exports zero constants" test.out; then
            echo "build: $(go version $GOMACRO) runtime: $goversion status: FAILED"
        else
            echo "build: $(go version $GOMACRO) runtime: $goversion status: SUCCESS"
        fi
        } | tee -a $topdir/results.txt
        )
        sudo rm -rf "$workdir"
    done
    )
done

And the results:

build: /home/lars/go/bin/gomacro: go1.19.4 runtime: 1.19.4 status: SUCCESS
build: /home/lars/go/bin/gomacro: go1.19.4 runtime: 1.20 status: SUCCESS
build: /home/lars/go/bin/gomacro: go1.19.4 runtime: 1.20.7 status: SUCCESS
build: /home/lars/go/bin/gomacro: go1.20 runtime: 1.19.4 status: SUCCESS
build: /home/lars/go/bin/gomacro: go1.20 runtime: 1.20 status: FAILED
build: /home/lars/go/bin/gomacro: go1.20 runtime: 1.20.7 status: SUCCESS
build: /home/lars/go/bin/gomacro: go1.20.7 runtime: 1.19.4 status: SUCCESS
build: /home/lars/go/bin/gomacro: go1.20.7 runtime: 1.20 status: SUCCESS
build: /home/lars/go/bin/gomacro: go1.20.7 runtime: 1.20.7 status: FAILED

Which are...weird. For anything more recent than 1.19.4, the import fails when the runtime version matches the build version.

cosmos72 commented 1 year ago

Thanks a lot for the extensive tests!

Wow, thats's even weirder than I thought, especially if you consider that gomacro tries to find and use the Go toolchain it was compiled with, befofe falling back to the one in $PATH

larsks commented 1 year ago

On the bright side, that means that I now have a functioning gomacro, which I really appreciate as a way of running quick tests. Thanks for your work!

cosmos72 commented 1 year ago

Commit 0c584cc732cfc29d8df0ffbb63a1190ec690abc6 updates the dependencies to newest golang.org/x/tools At least for me, it fixes this bug and I can now import third-party packages even when gomacro is compiled with Go toolchain >= 1.20.0

Hopefully, it will work for everybody else too :)

Please try it and report the outcome

cosmos72 commented 1 year ago

commit 3dc08f5f8feafb4e7b2f7b7c360ab6811010801a adds checks while importing third-party packages, in order to skip importing generic types and generic functions, which are not supported yet

cosmos72 commented 10 months ago

No further feedback received, closing after timeout as it's supposedly fixed.

Cactusinhand commented 2 months ago

The bug is still on my go1.23.0 windows/amd64: Undisciplined.PNG

cosmos72 commented 2 months ago

Hello @Cactusinhand, importing third-party packages is not well supported on Windows:

the procedure is cumbersome and requires recompiling gomacro after adding the package you want to import. For details, see https://github.com/cosmos72/gomacro/?tab=readme-ov-file#other-systems

You may have an easier time by running gomacro or gophernotes inside Docker or a Linux virtual machine. Maybe running them inside WSL may work too.