Test failures on big-endian system (s390x)

[gibmat]

I've recently packaged this library for Debian, and when its tests are run on a big-endian system (s390x), several of the tests fail. My initial guess is that since the mmvdump test files were created on a little-endian system, they are being read improperly on the big-endian system.

    cd _build && go test -vet=off -v -p 10 github.com/performancecopilot/speed github.com/performancecopilot/speed/bytewriter github.com/performancecopilot/speed/mmvdump
error initializing config. maybe PCP isn't installed properly
=== RUN   TestMmvFileLocation
--- PASS: TestMmvFileLocation (0.00s)
=== RUN   TestTocCountAndLength
--- PASS: TestTocCountAndLength (0.00s)
=== RUN   TestMapping
--- PASS: TestMapping (0.00s)
=== RUN   TestWritingSingletonMetric
    client_test.go:373: Incomplete/Partially Written TOC
--- FAIL: TestWritingSingletonMetric (0.03s)
=== RUN   TestUpdatingSingletonMetric
    client_test.go:427: Cannot extract dump from the writer buffer
--- FAIL: TestUpdatingSingletonMetric (0.02s)
=== RUN   TestWritingInstanceMetric
    client_test.go:539: Incomplete/Partially Written TOC
--- FAIL: TestWritingInstanceMetric (0.06s)
=== RUN   TestUpdatingInstanceMetric
    client_test.go:582: cannot get dump, error: Incomplete/Partially Written TOC
    client_test.go:342: expected 1 metrics, got 0
    client_test.go:346: expected 2 values, got 0
    client_test.go:301: expected a metric of name met.1
    client_test.go:486: expected 2 instances, got 0
    client_test.go:493: expected an instance domain of name met
    client_test.go:500: expected an instance domain at 216
    client_test.go:500: expected an instance domain at 136
    client_test.go:612: cannot get dump, error: Incomplete/Partially Written TOC
    client_test.go:342: expected 1 metrics, got 0
    client_test.go:346: expected 2 values, got 0
    client_test.go:301: expected a metric of name met.1
    client_test.go:486: expected 2 instances, got 0
    client_test.go:493: expected an instance domain of name met
    client_test.go:500: expected an instance domain at 136
    client_test.go:500: expected an instance domain at 216
--- FAIL: TestUpdatingInstanceMetric (0.24s)
=== RUN   TestStringValueWriting
    client_test.go:638: Incomplete/Partially Written TOC
--- FAIL: TestStringValueWriting (0.09s)
=== RUN   TestWritingDifferentSemantics
    client_test.go:705: cannot create dump: Incomplete/Partially Written TOC
    client_test.go:342: expected 8 metrics, got 0
    client_test.go:346: expected 12 values, got 0
    client_test.go:284: expected a metric of name m.2
    client_test.go:284: expected a metric of name m.3
    client_test.go:284: expected a metric of name m.4
    client_test.go:301: expected a metric of name m.5
    client_test.go:301: expected a metric of name m.6
    client_test.go:301: expected a metric of name m.7
    client_test.go:301: expected a metric of name m.8
    client_test.go:284: expected a metric of name m.1
    client_test.go:486: expected 2 instances, got 0
    client_test.go:493: expected an instance domain of name m
    client_test.go:500: expected an instance domain at 136
    client_test.go:500: expected an instance domain at 216
--- FAIL: TestWritingDifferentSemantics (0.13s)
=== RUN   TestWritingDifferentUnits
    client_test.go:758: cannot get dump: Incomplete/Partially Written TOC
--- FAIL: TestWritingDifferentUnits (0.13s)
=== RUN   TestWritingDifferentTypes
    client_test.go:794: cannot get dump: Incomplete/Partially Written TOC
--- FAIL: TestWritingDifferentTypes (0.24s)
=== RUN   TestMMV2MetricWriting
    client_test.go:817: cannot create dump, error: Incomplete/Partially Written TOC
--- FAIL: TestMMV2MetricWriting (0.63s)
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
    panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x155e8c]

goroutine 215 [running]:
testing.tRunner.func1.2({0x177be0, 0x2a6190})
    /usr/lib/go-1.19/src/testing/testing.go:1396 +0x2e8
testing.tRunner.func1()
    /usr/lib/go-1.19/src/testing/testing.go:1399 +0x3fc
panic({0x177be0, 0x2a6190})
    /usr/lib/go-1.19/src/runtime/panic.go:884 +0x240
github.com/performancecopilot/speed.TestMMV2MetricWriting(0xc018521a00)
    /tmp/autopkgtest-lxc.jhd0e6dy/downtmp/autopkgtest_tmp/_build/src/github.com/performancecopilot/speed/client_test.go:820 +0x2bc
testing.tRunner(0xc018521a00, 0x1a8d30)
    /usr/lib/go-1.19/src/testing/testing.go:1446 +0x128
created by testing.(*T).Run
    /usr/lib/go-1.19/src/testing/testing.go:1493 +0x448
FAIL    github.com/performancecopilot/speed 1.614s
=== RUN   TestWriteInt32
--- PASS: TestWriteInt32 (0.00s)
=== RUN   TestWriteInt64
--- PASS: TestWriteInt64 (0.00s)
=== RUN   TestWriteString
--- PASS: TestWriteString (0.00s)
=== RUN   TestOffset
--- PASS: TestOffset (0.00s)
=== RUN   TestMemoryMappedWriter
--- PASS: TestMemoryMappedWriter (0.01s)
PASS
ok      github.com/performancecopilot/speed/bytewriter  0.023s
=== RUN   TestMmvDump1
    mmvdump_test.go:17: Incomplete/Partially Written TOC
--- FAIL: TestMmvDump1 (0.05s)
=== RUN   TestInputs
    mmvdump_test.go:67: Incomplete/Partially Written TOC
--- FAIL: TestInputs (0.03s)
FAIL
FAIL    github.com/performancecopilot/speed/mmvdump 0.098s
FAIL

[natoscott]

@gibmat yeah, I remember noticing that on Fedora awhile back too. I don't know the code base (or golang itself) well enough to fix it, but I suspect you're right and it needs separate big endian and little endian MMV dumps to verify with.

[gibmat]

Do you know how those test files were created? I do have access to the Debian porter boxes, so I can help debug/fix this in a s390x environment.

[natoscott]

Not 100% sure but from poking around the code just now I can see similar sorts of MMV files would be created by some of the ./examples/* content. For instance ./examples/simple/main.go seems to create the metric listed in test1.mmv ...

nathans@fedora:~/git/speed$ /var/lib/pcp/pmdas/mmv/mmvdump ./mmvdump/testdata/test1.mmv 
MMV file   = ./mmvdump/testdata/test1.mmv
Version    = 1
Generated  = 1468770536
TOC count  = 3
Cluster    = 127
Process    = 29956
Flags      = 0x2 (process)

TOC[0]: toc offset 40, metrics offset 88 (1 entries)
  [725/88] simple.counter
       type=32-bit int (0x0), sem=counter (0x1), pad=0x0
       units=count
       (no indom)
       shorttext=A Simple Metric
       helptext=This is a simple counter metric to demonstrate the speed API

TOC[1]: offset 56, values offset 192 (1 entries)
  [725/192] simple.counter = 42

TOC[2]: offset 72, string offset 224 (2 entries)
  [1/224] A Simple Metric
  [2/480] This is a simple counter metric to demonstrate the speed API
nathans@fedora:~/git/speed$