rogers0 commented 4 years ago

two tests pass on 64-bit systems, but fail on 32-bit systems, such as i386, armhf.

error log:

=== RUN   Test_AsString
    Test_AsString: conversion_test.go:143: 
            Error Trace:    conversion_test.go:143
            Error:          Not equal: 
                            expected: "6273346999"
                            actual  : "6.273346999e+09"

                            Diff:
                            --- Expected
                            +++ Actual
                            @@ -1 +1 @@
                            -6273346999
                            +6.273346999e+09
            Test:           Test_AsString
--- FAIL: Test_AsString (0.00s)

=== RUN   TestNewCollection
--- FAIL: TestNewCollection (0.00s)
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
    panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x804a78c]

goroutine 21 [running]:
testing.tRunner.func1.1(0x83569a0, 0x863c860)
    /usr/lib/go-1.14/src/testing/testing.go:940 +0x27c
testing.tRunner.func1(0x8ce01e0)
    /usr/lib/go-1.14/src/testing/testing.go:943 +0x349
panic(0x83569a0, 0x863c860)
    /usr/lib/go-1.14/src/runtime/panic.go:969 +0x122
runtime/internal/atomic.Xadd64(0x8c94f84, 0x1, 0x0, 0xffffffff, 0xf)
    /usr/lib/go-1.14/src/runtime/internal/atomic/asm_386.s:105 +0xc
github.com/viant/toolbox/data.(*CompactedSlice).Add(0x8c94f60, 0x8c4ce70)
    /build/1st/golang-github-viant-toolbox-0.32.2/obj-i686-linux-gnu/src/github.com/viant/toolbox/data/compacted.go:146 +0x72
github.com/viant/toolbox/data.TestNewCollection(0x8ce01e0)
    /build/1st/golang-github-viant-toolbox-0.32.2/obj-i686-linux-gnu/src/github.com/viant/toolbox/data/compacted_test.go:14 +0x30d
testing.tRunner(0x8ce01e0, 0x83b8410)
    /usr/lib/go-1.14/src/testing/testing.go:991 +0xb4
created by testing.(*T).Run
    /usr/lib/go-1.14/src/testing/testing.go:1042 +0x2ad
FAIL    github.com/viant/toolbox/data   0.032s

Reference:

adranwit commented 4 years ago

I've patched converter test, However, with the second issue, it looks like using int64 data type cause panic, If there is a way to detect test architecture I can disable that test ?

var initSize = len(s.fieldNames)
if initSize < len(data) {
    initSize = len(data)
}

line 146# atomic.AddInt64(&s.size, 1)

rogers0 commented 4 years ago

for detecting CPU, maybe you can refer:

https://github.com/klauspost/cpuid

rogers0 commented 4 years ago

tried latest 0.32.3 on i386, and there're two fails:

# github.com/viant/toolbox_test [github.com/viant/toolbox.test]
src/github.com/viant/toolbox/converter_test.go:409:9: constant 1668069210749 overflows int
src/github.com/viant/toolbox/converter_test.go:521:8: constant 1668069210749 overflows int
FAIL    github.com/viant/toolbox [build failed]
# github.com/viant/toolbox/data/udf [github.com/viant/toolbox/data/udf.test]
src/github.com/viant/toolbox/data/udf/conversion_test.go:142:17: constant 6273346999 overflows int
FAIL    github.com/viant/toolbox/data/udf [build failed]
# github.com/viant/toolbox/storage_test [github.com/viant/toolbox/storage.test]
src/github.com/viant/toolbox/storage/file_info_test.go:13:3: constant 2147484141 overflows int
src/github.com/viant/toolbox/storage/file_info_test.go:14:3: constant 2147484159 overflows int
src/github.com/viant/toolbox/storage/file_info_test.go:15:3: constant 2147484128 overflows int
FAIL    github.com/viant/toolbox/storage [build failed]

another one:

=== RUN   TestCompactedSlice_SortedRange
--- FAIL: TestCompactedSlice_SortedRange (0.00s)
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
        panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x804a78c]

goroutine 8 [running]:
testing.tRunner.func1.1(0x83569a0, 0x863c860)
        /usr/lib/go-1.14/src/testing/testing.go:940 +0x27c
testing.tRunner.func1(0xa4a6320)
        /usr/lib/go-1.14/src/testing/testing.go:943 +0x349
panic(0x83569a0, 0x863c860)
        /usr/lib/go-1.14/src/runtime/panic.go:969 +0x122
runtime/internal/atomic.Xadd64(0xa466fe4, 0x1, 0x0, 0xfffffff, 0x0)
        /usr/lib/go-1.14/src/runtime/internal/atomic/asm_386.s:105 +0xc
github.com/viant/toolbox/data.(*CompactedSlice).Add(0xa466fc0, 0xa413540)
        /build/golang-github-viant-toolbox-0.32.3/obj-i686-linux-gnu/src/github.com/viant/toolbox/data/compacted.go:146 +0x72
github.com/viant/toolbox/data.TestCompactedSlice_SortedRange(0xa4a6320)
        /build/golang-github-viant-toolbox-0.32.3/obj-i686-linux-gnu/src/github.com/viant/toolbox/data/compacted_test.go:211 +0x12a6 
testing.tRunner(0xa4a6320, 0x83b8404)
        /usr/lib/go-1.14/src/testing/testing.go:991 +0xb4
created by testing.(*T).Run
        /usr/lib/go-1.14/src/testing/testing.go:1042 +0x2ad
FAIL    github.com/viant/toolbox/data   0.145s
FAIL    github.com/viant/toolbox/data/udf [build failed]

adranwit commented 4 years ago

Updated test with skip for non 64 bit arch

rogers0 commented 4 years ago

@adranwit Thanks for your update!

For latest 0.33.0, the result is:

# github.com/viant/toolbox/data/udf [github.com/viant/toolbox/data/udf.test]
src/github.com/viant/toolbox/data/udf/conversion_test.go:151:17: constant 2323232323223 overflows int
# github.com/viant/toolbox/storage_test [github.com/viant/toolbox/storage.test]
src/github.com/viant/toolbox/storage/file_info_test.go:13:3: constant 2147484141 overflows int
src/github.com/viant/toolbox/storage/file_info_test.go:14:3: constant 2147484159 overflows int
src/github.com/viant/toolbox/storage/file_info_test.go:15:3: constant 2147484128 overflows int
# github.com/viant/toolbox_test [github.com/viant/toolbox.test]
src/github.com/viant/toolbox/converter_test.go:527:8: constant 1668069210749 overflows int

another one:

=== RUN   TestNewCollection
--- FAIL: TestNewCollection (0.00s)
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
    panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x804a78c]

goroutine 6 [running]:
testing.tRunner.func1.1(0x83594e0, 0x8642880)
    /usr/lib/go-1.14/src/testing/testing.go:940 +0x27c
testing.tRunner.func1(0x94ac1e0)
    /usr/lib/go-1.14/src/testing/testing.go:943 +0x349
panic(0x83594e0, 0x8642880)
    /usr/lib/go-1.14/src/runtime/panic.go:969 +0x122
runtime/internal/atomic.Xadd64(0x9428fb4, 0x1, 0x0, 0xffffffff, 0xffffff)
    /usr/lib/go-1.14/src/runtime/internal/atomic/asm_386.s:105 +0xc
github.com/viant/toolbox/data.(*CompactedSlice).Add(0x9428f90, 0x9451e70)
    /build/1st/golang-github-viant-toolbox-0.33.0/obj-i686-linux-gnu/src/github.com/viant/toolbox/data/compacted.go:146 +0x72
github.com/viant/toolbox/data.TestNewCollection(0x94ac1e0)
    /build/1st/golang-github-viant-toolbox-0.33.0/obj-i686-linux-gnu/src/github.com/viant/toolbox/data/compacted_test.go:17 +0x31b
testing.tRunner(0x94ac1e0, 0x83bbe50)
    /usr/lib/go-1.14/src/testing/testing.go:991 +0xb4
created by testing.(*T).Run
    /usr/lib/go-1.14/src/testing/testing.go:1042 +0x2ad
FAIL    github.com/viant/toolbox/data   0.036s

adranwit commented 4 years ago

Looks like 64 detection, I've added to the test does not work, if cpuid.CPU.Family < 64 { return }

could you please print me the following on the machine the test failed ?

fmt.Println("Name:", cpuid.CPU.BrandName)
fmt.Println("PhysicalCores:", cpuid.CPU.PhysicalCores)
fmt.Println("ThreadsPerCore:", cpuid.CPU.ThreadsPerCore)
fmt.Println("LogicalCores:", cpuid.CPU.LogicalCores)
fmt.Println("Family", cpuid.CPU.Family, "Model:", cpuid.CPU.Model)
fmt.Println("Features:", cpuid.CPU.Features)
fmt.Println("Cacheline bytes:", cpuid.CPU.CacheLine)
fmt.Println("L1 Data Cache:", cpuid.CPU.Cache.L1D, "bytes")
fmt.Println("L1 Instruction Cache:", cpuid.CPU.Cache.L1D, "bytes")
fmt.Println("L2 Cache:", cpuid.CPU.Cache.L2, "bytes")
fmt.Println("L3 Cache:", cpuid.CPU.Cache.L3, "bytes")

rogers0 commented 4 years ago

For debian, I used the patch: https://salsa.debian.org/go-team/packages/golang-github-viant-toolbox/-/blob/debian/sid/debian/patches/0001-Skip-some-tests-fail-at-git-pbuilder-environment.patch

This most important part is:

-   if cpuid.CPU.CacheLine < 64 {
-       return
+   if (32 << uintptr(^uintptr(0)>>63)) < 64 {
+       t.Skip()

rogers0 commented 4 years ago

For debian, I used the patch: https://salsa.debian.org/go-team/packages/golang-github-viant-toolbox/-/blob/debian/sid/debian/patches/0001-Skip-some-tests-fail-at-git-pbuilder-environment.patch

Let me clarify. I do not mean you should merge this patch to upstream repository. I just show you how I workaround current errors from reproducible builds. Thanks for your working!

adranwit commented 4 years ago

Thanks for the suggestion, updated tests

adranwit commented 4 years ago

Have you run test with the latest code ?

rogers0 commented 4 years ago

Sorry for the late reply. The test code still build fails under 32-bit systems, so I have to disable it.

https://salsa.debian.org/go-team/packages/golang-github-viant-toolbox/-/blob/debian/sid/debian/rules#L17

But I still think some code need to be updated:

https://salsa.debian.org/go-team/packages/golang-github-viant-toolbox/-/blob/debian/sid/debian/patches/0002-Skip-some-tests-fail-at-git-pbuilder-environment.patch

I'll send you a PR later.

adranwit commented 4 years ago

Removed 32 bit test

rogers0 commented 4 years ago

Except PR #39 there're a few other issues. (since I set ignore all 32-bit go test errors, so currently reproducible builds all "passed")

https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/golang-github-viant-toolbox.html

All current log based on latest, + PR #39 patch:

test fails on i386 (x86 32-bit)

=== RUN   TestConverter_AsInt
TestConverter_AsInt: converter_test.go:702: 
        Error Trace:    converter_test.go:702
        Error:          Should be true
        Test:           TestConverter_AsInt
--- FAIL: TestConverter_AsInt (0.00s)

https://tests.reproducible-builds.org/debian/rbuild/unstable/i386/golang-github-viant-toolbox_0.33.0-11.rbuild.log.gz

panic: runtime error on i386 (x86 32-bit) and armhf (arm 32-bit)


=== RUN   TestLogger
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x804ab9b]

goroutine 165 [running]: runtime/internal/atomic.Load64(0xa86b2ec, 0xbfc1ad5d, 0xd905ff2) /usr/lib/go-1.14/src/runtime/internal/atomic/asm_386.s:194 +0xb github.com/viant/toolbox.(LogStream).isFrequencyFlushNeeded(0xa86b2c0, 0x0) /build/1st/golang-github-viant-toolbox-0.33.0/obj-i686-linux-gnu/src/github.com/viant/toolbox/file_logger.go:138 +0x69 github.com/viant/toolbox.(LogStream).manageWritesInBatch(0xa86b2c0) /build/1st/golang-github-viant-toolbox-0.33.0/obj-i686-linux-gnu/src/github.com/viant/toolbox/file_logger.go:166 +0x23b github.com/viant/toolbox.(FileLogger).NewLogStream.func1(0xa86b2c0) /build/1st/golang-github-viant-toolbox-0.33.0/obj-i686-linux-gnu/src/github.com/viant/toolbox/file_logger.go:219 +0x21 created by github.com/viant/toolbox.(FileLogger).NewLogStream /build/1st/golang-github-viant-toolbox-0.33.0/obj-i686-linux-gnu/src/github.com/viant/toolbox/file_logger.go:218 +0xfe FAIL github.com/viant/toolbox 0.239s

https://tests.reproducible-builds.org/debian/rbuild/unstable/armhf/golang-github-viant-toolbox_0.33.0-11.rbuild.log.gz

- build error on 32-bit systems:

github.com/viant/toolbox/data/udf [github.com/viant/toolbox/data/udf.test]

src/github.com/viant/toolbox/data/udf/conversion_test.go:142:17: constant 2323232323223 overflows int

github.com/viant/toolbox/storage_test [github.com/viant/toolbox/storage.test]

src/github.com/viant/toolbox/storage/file_info_test.go:13:3: constant 2147484141 overflows int src/github.com/viant/toolbox/storage/file_info_test.go:14:3: constant 2147484159 overflows int src/github.com/viant/toolbox/storage/file_info_test.go:15:3: constant 2147484128 overflows int


https://tests.reproducible-builds.org/debian/rbuild/unstable/i386/golang-github-viant-toolbox_0.33.0-11.rbuild.log.gz

Thank you!

viant / toolbox

go tests fail on 32-bit systems #37

github.com/viant/toolbox/data/udf [github.com/viant/toolbox/data/udf.test]

github.com/viant/toolbox/storage_test [github.com/viant/toolbox/storage.test]