search

minio / simdjson-go

Golang port of simdjson: parsing gigabytes of JSON per second
Apache License 2.0
1.81k stars 86 forks source link

Build error with gccgo (10.2.0) #21

Closed rnsanchez closed 3 years ago

rnsanchez commented 4 years ago

When trying to build the current (ef1a2b46ad3790965aad804c5b82bab2033cacce) revision:

$ go version
go version go1.14.4 gccgo (GCC) 10.2.0 linux/amd64

$ go get -v github.com/minio/simdjson-go
github.com/minio/simdjson-go
# github.com/minio/simdjson-go
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s: Assembler messages:
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:6: Error: no such instruction: `text ·_finalize_structurals(SB),$0-48'
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:8: Error: junk `(FP)' after expression
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:8: Error: too many memory references for `movq'
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:9: Error: junk `(FP)' after expression
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:9: Error: too many memory references for `movq'
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:10: Error: junk `(FP)' after expression
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:10: Error: too many memory references for `movq'
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:11: Error: junk `(FP)' after expression
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:11: Error: too many memory references for `movq'
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:12: Error: junk `(FP)' after expression
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:12: Error: too many memory references for `movq'
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:14: Error: junk `(SB)' after expression
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:16: Error: too many memory references for `movq'
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:19: Error: no such instruction: `text ·__finalize_structurals(SB),$0'
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:21: Error: too many memory references for `andn'
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:22: Error: too many memory references for `or'
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:23: Error: too many memory references for `movq'
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:24: Error: too many memory references for `or'
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:25: Error: junk `(AX*1)' after expression
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:25: Error: too many memory references for `lea'
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:26: Error: too many memory references for `or'
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:28: Error: too many memory references for `movq'
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:30: Error: too many memory references for `andn'
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:31: Error: too many memory references for `and'
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:32: Error: too many memory references for `or'
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:34: Error: too many memory references for `or'
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:35: Error: too many memory references for `and'
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:38: Error: no such instruction: `text ·__finalize_structurals_avx512(SB),$0'
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:40: Error: too many memory references for `kmovq'
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:41: Error: too many memory references for `kmovq'
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:42: Error: too many memory references for `kmovq'
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:43: Error: too many memory references for `andn'
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:44: Error: too many memory references for `or'
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:45: Error: too many memory references for `movq'
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:46: Error: too many memory references for `or'
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:47: Error: junk `(AX*1)' after expression
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:47: Error: too many memory references for `lea'
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:48: Error: too many memory references for `or'
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:50: Error: too many memory references for `movq'
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:52: Error: too many memory references for `andn'
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:53: Error: too many memory references for `and'
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:54: Error: too many memory references for `or'
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:56: Error: too many memory references for `or'
../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s:57: Error: too many memory references for `and'
$ cat ../../../../../Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s
//+build !noasm !appengine gc
// AUTO-GENERATED BY C2GOASM -- DO NOT EDIT

#include "common.h"

TEXT ·_finalize_structurals(SB), $0-48

    MOVQ structurals_in+0(FP), DI
    MOVQ whitespace+8(FP), SI
    MOVQ quote_mask+16(FP), DX
    MOVQ quote_bits+24(FP), CX
    MOVQ prev_iter_ends_pseudo_pred+32(FP), R8

    CALL ·__finalize_structurals(SB)

    MOVQ AX, structurals+40(FP)
    RET

TEXT ·__finalize_structurals(SB), $0

    ANDNQ DI, DX, DI             // andn    rdi, rdx, rdi
    ORQ  CX, DI                  // or    rdi, rcx
    MOVQ DI, AX                  // mov    rax, rdi
    ORQ  SI, AX                  // or    rax, rsi
    LEAQ (AX)(AX*1), R9          // lea    r9, [rax + rax]
    ORQ  (R8), R9                // or    r9, qword [r8]
    SHRQ $63, AX                 // shr    rax, 63
    MOVQ AX, (R8)                // mov    qword [r8], rax
    NOTQ SI                      // not    rsi
    ANDNQ SI, DX, AX             // andn    rax, rdx, rsi
    ANDQ R9, AX                  // and    rax, r9
    ORQ  DI, AX                  // or    rax, rdi
    NOTQ CX                      // not    rcx
    ORQ  DX, CX                  // or    rcx, rdx
    ANDQ CX, AX                  // and    rax, rcx
    RET

TEXT ·__finalize_structurals_avx512(SB), $0

    KMOVQ K_WHITESPACE,  SI
    KMOVQ K_QUOTEBITS,   CX
    KMOVQ K_STRUCTURALS, DI
    ANDNQ DI, DX, DI             // andn    rdi, rdx, rdi
    ORQ  CX, DI                  // or    rdi, rcx
    MOVQ DI, AX                  // mov    rax, rdi
    ORQ  SI, AX                  // or    rax, rsi
    LEAQ (AX)(AX*1), R9          // lea    r9, [rax + rax]
    ORQ  (R8), R9                // or    r9, qword [r8]
    SHRQ $63, AX                 // shr    rax, 63
    MOVQ AX, (R8)                // mov    qword [r8], rax
    NOTQ SI                      // not    rsi
    ANDNQ SI, DX, AX             // andn    rax, rdx, rsi
    ANDQ R9, AX                  // and    rax, r9
    ORQ  DI, AX                  // or    rax, rdi
    NOTQ CX                      // not    rcx
    ORQ  DX, CX                  // or    rcx, rdx
    ANDQ CX, AX                  // and    rax, rcx
    RET

CPU:

Model name:                      Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz

Any more info I could provide?

fwessels commented 4 years ago

Hmm, this is weird, it looks like the golang assembler is not being invoked on the .s (assembler) file.

Could it be that the gcc assembler (gas) is being invoked on this file?

rnsanchez commented 4 years ago

I have only GCC Go installed, so I think so (per strace):

6316  execve("/usr/bin/gccgo", ["/usr/bin/gccgo", "-xassembler-with-cpp", "-I", "/tmp/go-build589333021/b001/", "-c", "-o", "/tmp/go-build589333021/b001/finalize_structurals_amd64.o", "-D", "GOOS_linux", "-D", "GOARCH_amd64", "-D",                 .."GOPKGPATH=github.x2ecom..z2fminio..z2fsimdjson..z2dgo", "-m64", "/home/rnsanchez/Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s"], 0xc00014f080 /* 84 vars */) = 0 
6317  execve("/usr/libexec/gcc/x86_64-slackware-linux/10.2.0/cc1", ["/usr/libexec/gcc/x86_64-slackware-linux/10.2.0/cc1", "-E", "-lang-asm", "-quiet", "-I", "/tmp/go-build589333021/b001/", "-D", "GOOS_linux", "-D", "GOARCH_amd64", "-D",            .."GOPKGPATH=github.x2ecom..z2fminio..z2fsimdjson..z2dgo", "/home/rnsanchez/Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s", "-m64", "-mtune=generic", "-march=x86-64", "-fno-directives-only", "-o", "/tmp/ccjI0P9L.s"],             ..0x1c40c70 /* 86 vars */) = 0 
6317  +++ exited with 0 +++ 
6316  --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=6317, si_uid=1000, si_status=0, si_utime=0, si_stime=0} --- 
6318  execve("/usr/local/bin/as", ["as", "-I", "/tmp/go-build589333021/b001/", "--64", "-o", "/tmp/go-build589333021/b001/finalize_structurals_amd64.o", "/tmp/ccjI0P9L.s"], 0x1c40c70 /* 86 vars */) = -1 ENOENT (No such file or directory)
6318  execve("/bin/as", ["as", "-I", "/tmp/go-build589333021/b001/", "--64", "-o", "/tmp/go-build589333021/b001/finalize_structurals_amd64.o", "/tmp/ccjI0P9L.s"], 0x1c40c70 /* 86 vars */) = -1 ENOENT (No such file or directory)
6318  execve("/usr/bin/as", ["as", "-I", "/tmp/go-build589333021/b001/", "--64", "-o", "/tmp/go-build589333021/b001/finalize_structurals_amd64.o", "/tmp/ccjI0P9L.s"], 0x1c40c70 /* 86 vars */) = 0 
6318  +++ exited with 1 +++
fwessels commented 4 years ago

Yes, it is invoking gcc on the .s file (see "/home/rnsanchez/Go/src/github.com/minio/simdjson-go/finalize_structurals_amd64.s", "-m64", "-mtune=generic", "-march=x86-64", "-fno-directives-only").

No clue as to why that would happen, I'll ask around to see if somebody here might have a clue.

In the meantime, since you are on Intel, why are you not using the regular compiler?

rnsanchez commented 4 years ago

Not sure I follow; GCC is my regular compiler. :-)

If you mean as to why I don't use google-golang (not sure if it still called that way), it creates a bit of havoc, since I need to match gcc-go in production.

rnsanchez commented 4 years ago

I've tried rebuilding golang/snappy for an strace output, and it too uses as albeit successfully:

15404 +++ exited with 0 +++
15403 --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=15404, si_uid=1000, si_status=0, si_utime=0, si_stime=0} ---
15405 execve("/usr/local/bin/as", ["as", "-I", "/tmp/go-build210707090/b001/", "--64", "-o", "/tmp/go-build210707090/b001/_buildid.o", "/tmp/ccdyNblT.s"], 0x22b51a0 /* 87 vars */) = -1 ENOENT (No such file or directory)
15405 execve("/bin/as", ["as", "-I", "/tmp/go-build210707090/b001/", "--64", "-o", "/tmp/go-build210707090/b001/_buildid.o", "/tmp/ccdyNblT.s"], 0x22b51a0 /* 87 vars */) = -1 ENOENT (No such file or directory)
15405 execve("/usr/bin/as", ["as", "-I", "/tmp/go-build210707090/b001/", "--64", "-o", "/tmp/go-build210707090/b001/_buildid.o", "/tmp/ccdyNblT.s"], 0x22b51a0 /* 87 vars */) = 0
15405 +++ exited with 0 +++

Just let me know how I can help.

klauspost commented 4 years ago

GCC is not compatible with any Go assembly. It will have to fall back to pure Go. Ie. be disabled.