Conflicting default version definition

dsxack commented 2 years ago

Warnings when I build my application (build in Docker):

/usr/bin/ld.gold: warning: /tmp/go-link-3460601247/go.o: conflicting default version definition for github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Fragments []mypackage/ast.Fragment "parser:\"@@+\"" }_0]).Lex
/usr/bin/ld.gold: /tmp/go-link-3460601247/go.o: previous definition of github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Fragments []mypackage/ast.Fragment "parser:\"@@+\"" }_0]).ParseFromLexer.func1 here
/usr/bin/ld.gold: warning: /tmp/go-link-3460601247/go.o: conflicting default version definition for github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Fragments []mypackage/ast.Fragment "parser:\"@@+\"" }_0]).ParseFromLexer
/usr/bin/ld.gold: /tmp/go-link-3460601247/go.o: previous definition of github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Fragments []mypackage/ast.Fragment "parser:\"@@+\"" }_0]).ParseFromLexer.func1 here
/usr/bin/ld.gold: warning: /tmp/go-link-3460601247/go.o: conflicting default version definition for github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Fragments []mypackage/ast.Fragment "parser:\"@@+\"" }_0]).parseNodeFor
/usr/bin/ld.gold: /tmp/go-link-3460601247/go.o: previous definition of github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Fragments []mypackage/ast.Fragment "parser:\"@@+\"" }_0]).ParseFromLexer.func1 here
/usr/bin/ld.gold: warning: /tmp/go-link-3460601247/go.o: conflicting default version definition for github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Fragments []mypackage/ast.Fragment "parser:\"@@+\"" }_0]).rootParseable
/usr/bin/ld.gold: /tmp/go-link-3460601247/go.o: previous definition of github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Fragments []mypackage/ast.Fragment "parser:\"@@+\"" }_0]).ParseFromLexer.func1 here
/usr/bin/ld.gold: warning: /tmp/go-link-3460601247/go.o: conflicting default version definition for github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Fragments []mypackage/ast.Fragment "parser:\"@@+\"" }_0]).parseOne
/usr/bin/ld.gold: /tmp/go-link-3460601247/go.o: previous definition of github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Fragments []mypackage/ast.Fragment "parser:\"@@+\"" }_0]).ParseFromLexer.func1 here
/usr/bin/ld.gold: warning: /tmp/go-link-3460601247/go.o: conflicting default version definition for github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Fragments []mypackage/ast.Fragment "parser:\"@@+\"" }_0]).parseInto
/usr/bin/ld.gold: /tmp/go-link-3460601247/go.o: previous definition of github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Fragments []mypackage/ast.Fragment "parser:\"@@+\"" }_0]).ParseFromLexer.func1 here
/usr/bin/ld.gold: warning: /tmp/go-link-3460601247/go.o: conflicting default version definition for github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Fragments []mypackage/ast.Fragment "parser:\"@@+\"" }_0]).parse
/usr/bin/ld.gold: /tmp/go-link-3460601247/go.o: previous definition of github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Fragments []mypackage/ast.Fragment "parser:\"@@+\"" }_0]).ParseFromLexer.func1 here
/usr/bin/ld.gold: warning: /tmp/go-link-3460601247/go.o: conflicting default version definition for github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Fragments []mypackage/ast.Fragment "parser:\"@@+\"" }_0]).getElidedTypes
/usr/bin/ld.gold: /tmp/go-link-3460601247/go.o: previous definition of github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Fragments []mypackage/ast.Fragment "parser:\"@@+\"" }_0]).ParseFromLexer.func1 here
/usr/bin/ld.gold: warning: /tmp/go-link-3460601247/go.o: conflicting default version definition for github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Fragments []mypackage/ast.Fragment "parser:\"@@+\"" }_0]).Parse
/usr/bin/ld.gold: /tmp/go-link-3460601247/go.o: previous definition of github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Fragments []mypackage/ast.Fragment "parser:\"@@+\"" }_0]).ParseFromLexer.func1 here
/usr/bin/ld.gold: warning: /tmp/go-link-3460601247/go.o: conflicting default version definition for github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Fragments []mypackage/ast.Fragment "parser:\"@@+\"" }_0]).ParseString
/usr/bin/ld.gold: /tmp/go-link-3460601247/go.o: previous definition of github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Fragments []mypackage/ast.Fragment "parser:\"@@+\"" }_0]).ParseFromLexer.func1 here
/usr/bin/ld.gold: warning: /tmp/go-link-3460601247/go.o: conflicting default version definition for github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Fragments []mypackage/ast.Fragment "parser:\"@@+\"" }_0]).ParseBytes
/usr/bin/ld.gold: /tmp/go-link-3460601247/go.o: previous definition of github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Fragments []mypackage/ast.Fragment "parser:\"@@+\"" }_0]).ParseFromLexer.func1 here
/usr/bin/ld.gold: warning: /tmp/go-link-3460601247/go.o: conflicting default version definition for github.com/alecthomas/participle/v2.Build[go.shape.struct { Fragments []mypackage/ast.Fragment "parser:\"@@+\"" }_0]
/usr/bin/ld.gold: /tmp/go-link-3460601247/go.o: previous definition of github.com/alecthomas/participle/v2.Build[go.shape.struct { Fragments []mypackage/ast.Fragment "parser:\"@@+\"" }_0].func1 here

On our CI (with other docker image) it occurs error instead of warning:

/usr/local/go/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
[303](https://github.com/ory/hydra/-/jobs/3190081#L303)/usr/bin/ld: /tmp/go-link-2229009697/go.o:(*IND*+0x0): multiple definition of `github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Fragments []mypackage/ast.Fragment "parser:\"'
[304](https://github.com/ory/hydra/-/jobs/3190081#L304)/usr/bin/ld: /tmp/go-link-2229009697/go.o:(*IND*+0x0): multiple definition of `github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Fragments []mypackage/ast.Fragment "parser:\"'
[305](https://github.com/ory/hydra/-/jobs/3190081#L305)/usr/bin/ld: /tmp/go-link-2229009697/go.o:(*IND*+0x0): multiple definition of `github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Fragments []mypackage/ast.Fragment "parser:\"'
[306](https://github.com/ory/hydra/-/jobs/3190081#L306)/usr/bin/ld: /tmp/go-link-2229009697/go.o:(*IND*+0x0): multiple definition of `github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Fragments []mypackage/ast.Fragment "parser:\"'
[307](https://github.com/ory/hydra/-/jobs/3190081#L307)/usr/bin/ld: /tmp/go-link-2229009697/go.o:(*IND*+0x0): multiple definition of `github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Fragments []mypackage/ast.Fragment "parser:\"'
[308](https://github.com/ory/hydra/-/jobs/3190081#L308)/usr/bin/ld: /tmp/go-link-2229009697/go.o:(*IND*+0x0): multiple definition of `github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Fragments []mypackage/ast.Fragment "parser:\"'
[309](https://github.com/ory/hydra/-/jobs/3190081#L309)/usr/bin/ld: /tmp/go-link-2229009697/go.o:(*IND*+0x0): multiple definition of `github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Fragments []mypackage/ast.Fragment "parser:\"'
[310](https://github.com/ory/hydra/-/jobs/3190081#L310)/usr/bin/ld: /tmp/go-link-2229009697/go.o:(*IND*+0x0): multiple definition of `github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Fragments []mypackage/ast.Fragment "parser:\"'
[311](https://github.com/ory/hydra/-/jobs/3190081#L311)/usr/bin/ld: /tmp/go-link-2229009697/go.o:(*IND*+0x0): multiple definition of `github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Fragments []mypackage/ast.Fragment "parser:\"'
[312](https://github.com/ory/hydra/-/jobs/3190081#L312)/usr/bin/ld: /tmp/go-link-2229009697/go.o:(*IND*+0x0): multiple definition of `github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Fragments []mypackage/ast.Fragment "parser:\"'
[313](https://github.com/ory/hydra/-/jobs/3190081#L313)/usr/bin/ld: /tmp/go-link-2229009697/go.o:(*IND*+0x0): multiple definition of `github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Fragments []mypackage/ast.Fragment "parser:\"'
[314](https://github.com/ory/hydra/-/jobs/3190081#L314)/usr/bin/ld: /tmp/go-link-2229009697/go.o:(*IND*+0x0): multiple definition of `github.com/alecthomas/participle/v2.Build[go.shape.struct { Fragments []mypackage/ast.Fragment "parser:\"'
[315](https://github.com/ory/hydra/-/jobs/3190081#L315)collect2: error: ld returned 1 exit status

go env:

GO111MODULE=""
GOARCH="arm64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="arm64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_arm64"
GOVCS=""
GOVERSION="go1.18.1"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/app/go.mod"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build1758053692=/tmp/go-build -gno-record-gcc-switches"

participle version:

github.com/alecthomas/participle/v2 v2.0.0-beta.4

alecthomas commented 2 years ago

Something is misconfigured with your Go tool chain, this has nothing to do with Participle.

alecthomas commented 2 years ago

(I've never seen this before so i'm not sure what the issue is)

debugloop commented 1 year ago

I am experiencing the same thing, so I am not sure it is strictly Go tool chain related. Maybe some misconfiguration in combination with Participle causes this. I first noticed it when upgrading from 2.0.0-alpha8 to the current 2.0.0-beta.5, I did not have this issue before, using the same toolchain. I'll report my findings, if I can produce some...

Edit: This issue also does not affect me on 2.0.0-beta.1, so it might have been introduced between then and 2.0.0-beta.4 (OPs version). If it even is an issue of Participle, that is...

Edit 2: The issue occurs starting with 2.0.0-beta.2 for me, I'll be downgrading to 2.0.0-beta.1 for now.

alecthomas commented 1 year ago

Do you have a way to replicate it?

FWIW I'm using beta.5 in several places without any issues. Also, what version of Go are you on? Can you try upgrading to the latest version? This is definitely toolchain related, because there's no way pure Go code could result in a linker error under normal circumstances, but something in Participle may have triggered it. But perhaps it is also fixed in a more recent version of Go.

rstcruzo commented 1 year ago

I get multiple definition errors too. Odd part is that it builds fine on a m1 mac, I get the error in an ubuntu machine only.

...
/usr/bin/ld: /usr/local/go/src/runtime/internal/atomic/types.go:101: multiple definition of `github.com/alecthomas/participle/v2.Build[go.shape.struct { Pos github.com/alecthomas/participle/v2/lexer.Position; EndPos github.com/alecthomas/participle/v2/lexer.Position; Entities []rstcruzo/dns/parsers.Entity "'
/usr/bin/ld: /usr/local/go/src/runtime/internal/atomic/types.go:101: multiple definition of `github.com/alecthomas/participle/v2.Build[go.shape.struct { Pos github.com/alecthomas/participle/v2/lexer.Position; EndPos github.com/alecthomas/participle/v2/lexer.Position; Lines []rstcruzo/dns/parsers.Line "(Whitespace | Newline)* '
/usr/bin/ld: /usr/local/go/src/runtime/internal/atomic/types.go:101: multiple definition of `github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Pos github.com/alecthomas/participle/v2/lexer.Position; EndPos github.com/alecthomas/participle/v2/lexer.Position; Lines []rstcruzo/dns/parsers.Line "(Whitespace | Newline)* '
/usr/bin/ld: /usr/local/go/src/runtime/internal/atomic/types.go:101: multiple definition of `github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Pos github.com/alecthomas/participle/v2/lexer.Position; EndPos github.com/alecthomas/participle/v2/lexer.Position; Entities []rstcruzo/dns/parsers.Entity "'
/usr/bin/ld: $WORK/b001/exe/a.out.so: version node not found for symbol github.com/alecthomas/participle/v2.Build[go.shape.struct { Pos github.com/alecthomas/participle/v2/lexer.Position; EndPos github.com/alecthomas/participle/v2/lexer.Position; Lines []rstcruzo/dns/parsers.Line "(Whitespace | Newline)* @@ (Whitespace? Newline+ @@)* (Whitespace | Newline)*" }_0]
/usr/bin/ld: failed to set dynamic section sizes: bad value
collect2: error: ld returned 1 exit status

make: *** [Makefile:13: docker] Error 2

Same go version on both machines 1.18.3

Edit: Issue comes up only when building a plugin and having a @ in a struct tag. https://github.com/golang/go/issues/19529 may be related.

rstcruzo commented 1 year ago

@alecthomas it definitely is about the @ sign. I made a fork, replaced the @ with $ and it now builds correctly. I will test it more throughly later today.

Let me know if you’d merge it and I will create a PR. If you’d rather use another symbol let me know too.

alecthomas commented 1 year ago

I won't merge that change.

Have you tried a more recent Go compiler?

rstcruzo commented 1 year ago

I am using go 1.18, I have tried compiling it with go 1.20 too, same error.

alecthomas commented 1 year ago

Are you able to reproduce it on a fresh install? ie. if you run go clean -cache -modcache -testcache, what steps will reproduce it?

I would suggest filing a bug against https://github.com/golang/go. This is absolutely a Go toolchain issue, it's occurring at the linking stage, well after anything in Participle's source might be an issue. And even if it is something strange in Participle that is triggering this issue, the Go toolchain should not be failing in this way.

alecthomas commented 1 year ago

I'll file an issue.

alecthomas commented 1 year ago

Actually it would be best if you did, as you can answer all the questions in the bug submission.

rstcruzo commented 1 year ago

Yes, I'm able to reproduce it after running go clean -cache -modcache -testcache. All I do is build the plugin with:

go build -buildmode=plugin -o plugin.so *.go

and I get the error.

I'll file an issue. Thanks for the library :)

kckrinke commented 1 year ago

Just a note that I'm experiencing this problem as well. Warnings on arm64 and errors on amd64.

Downgrading to tag v2.0.0-beta.1 does in fact build correctly.

I've tried the example code in the golang bug report and that did compile as well so I'm at a loss for determining what the error actually is. The only suspect I can think of is the usage of generics in beta.2 onward, though I'm not saying it's participle that has the issue and it could in fact be golang failing participle in some fashion that none of the other libraries I'm using exhibit.

Reference note: tried go versions 1.18.10, 1.19.5 and 1.20, and all the beta versions of participle (including the main branch latest changes) - with all producing the same results - except beta.1 (with any version of go), which does work.

alecthomas commented 1 year ago

It dawned on me last night after looking more closely at the errors that this was likely caused by the switch in Participle to generics. I suspect what is happening is that Go is generating duplicate symbols for the concrete types under some circumstances.

alecthomas commented 1 year ago

Although the fact that it only occurs on some platforms indicates it might also be a limitation of the linker on those platforms. Perhaps they have a length limit on symbols that is being truncated...or perhaps the behaviour of the Go toolchain is slightly different.

kckrinke commented 1 year ago

In my experimentation, it's a very narrow (complicatedly specific) corner case where the issue crops up. Here's a rundown of the situation:

// TLDR; it could be a case of indirect dependency with a direct dependency in multiple packages where golang compiles duplicates, which the arm linker somehow allows/picks one and the x86 linkers throws errors not picking either

participle is being used by github.com/go-enjin/be/pkg/pageql
side-note: this is a website "enjin" project where developers are expected to write the main.go usage of an "enjin" and if needed can also provide custom features beyond what's available in stock
for the official go-enjin website https://github.com/go-enjin/website/ the conflicting default version definition problem is not present, nor is it present in any of the other implementations being worked on
the issue is present in a very specific case scenario involving Atlassian Jira cloud applications
there is a series of dependency features required for implementing an Atlassian app using go-enjin
the first is a forked version of github.com/go-enjin/github-com-craftamap-atlas-gonnect which has enjin-specific tooling and integrations, which also imports github.com/go-enjin/be (which includes participle as a dependency)
the second is a custom enjin feature github.com/go-enjin/features-gonnectian which uses atlas-gonnect to provide a simple means of implementing an Atlassian Jira cloud app using go-enjin, note that this also imports github.com/go-enjin/be which depends upon participle, as well as importing participle indirectly via the atlas-gonnect usage
the third may likely have nothing to do with the participle issue, mentioning it only because it too indirectly and directly depends upon participle and that's the github.com/go-enjin/features-consoles-gonnectian enjin feature which provides a very simple curses UI for enabling or disabling the debug states of client application installations
in my testing / side-project Jira cloud applications, this situation results in warnings on arm and compile errors on x86
in all the other non-jira-projects and sites, the issue is not present

One deduction from all of this is that it may be an issue within golang itself (v1.18 - v1.20 tested) and how generics are being used in participle beta.2 and onwards. Specifically, when go is running through the globals / init() phase, where all the participle.MustBuild calls are made, the compiler is including things twice: once for the indirect dependency and then again for the direct (or reversed).

I could totally be wrong with this but perhaps it sheds some additional light or perspective for reproducing the problem.

// it should be noted that the entire project has downgraded to beta.1 so the errors are no longer present, I can however rig my local environment for testing new versions of participle when needed.

imuxin commented 1 year ago

Build successfully in macos, but failed in linux. Here is the error log.

# command-line-arguments
/usr/local/bin/go/1.20.4/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
/usr/bin/ld: /tmp/go-link-51844258/go.o:(*IND*+0x0): multiple definition of `github.com/alecthomas/participle/v2.Build[go.shape.struct { Use github.com/imuxin/ksql/pkg/parser.UseStat "parser:\" '
/usr/bin/ld: /tmp/go-link-51844258/go.o:(*IND*+0x0): multiple definition of `github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Use github.com/imuxin/ksql/pkg/parser.UseStat "parser:\" '
/usr/bin/ld: /tmp/go-link-51844258/go.o:(*IND*+0x0): multiple definition of `github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Use github.com/imuxin/ksql/pkg/parser.UseStat "parser:\" '
/usr/bin/ld: /tmp/go-link-51844258/go.o:(*IND*+0x0): multiple definition of `github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Use github.com/imuxin/ksql/pkg/parser.UseStat "parser:\" '
/usr/bin/ld: /tmp/go-link-51844258/go.o:(*IND*+0x0): multiple definition of `github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Use github.com/imuxin/ksql/pkg/parser.UseStat "parser:\" '
/usr/bin/ld: /tmp/go-link-51844258/go.o:(*IND*+0x0): multiple definition of `github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Use github.com/imuxin/ksql/pkg/parser.UseStat "parser:\" '
/usr/bin/ld: /tmp/go-link-51844258/go.o:(*IND*+0x0): multiple definition of `github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Use github.com/imuxin/ksql/pkg/parser.UseStat "parser:\" '
/usr/bin/ld: /tmp/go-link-51844258/go.o:(*IND*+0x0): multiple definition of `github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Use github.com/imuxin/ksql/pkg/parser.UseStat "parser:\" '
/usr/bin/ld: /tmp/go-link-51844258/go.o:(*IND*+0x0): multiple definition of `github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Use github.com/imuxin/ksql/pkg/parser.UseStat "parser:\" '
/usr/bin/ld: /tmp/go-link-51844258/go.o:(*IND*+0x0): multiple definition of `github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Use github.com/imuxin/ksql/pkg/parser.UseStat "parser:\" '
/usr/bin/ld: /tmp/go-link-51844258/go.o:(*IND*+0x0): multiple definition of `github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Use github.com/imuxin/ksql/pkg/parser.UseStat "parser:\" '
/usr/bin/ld: /tmp/go-link-51844258/go.o:(*IND*+0x0): multiple definition of `github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Use github.com/imuxin/ksql/pkg/parser.UseStat "parser:\" '
/usr/bin/ld: /tmp/go-link-51844258/go.o:(*IND*+0x0): multiple definition of `github.com/alecthomas/participle/v2.(*Parser[go.shape.struct { Use github.com/imuxin/ksql/pkg/parser.UseStat "parser:\" '
collect2: error: ld returned 1 exit status

FYI: And I tried passing -ldflags "-extldflags=-Wl,--allow-multiple-definition" flag to go build command, error disappeared.

alecthomas / participle

Conflicting default version definition #251