Bug: HTTP check fails when http content is too large with "Body: Error: bufio.Scanner: token too long"

[rmetzler]

Describe the bug

I have some very big XMLs I need to verify

• the HTTP status code is 200
• it starts with the xml declaration

How To Reproduce

http:

  https://example.com/sitemap_0.xml:
    status: 200
    allow-insecure: false
    no-follow-redirects: false
    timeout: 5000

    # all documents should return XML with example.com in it
    body:
      - '<?xml version="1.0" encoding="UTF-8" standalone="yes"?>'

Expected Behavior

goss should say this is ok

Actual Behavior

$ goss -gossfile=src/goss/sitemap/bug.yaml v --format=documentation
HTTP: https://example.com/sitemap_0.xml: status: matches expectation: [200]
https://example.com/sitemap_0.xml: Body: Error: bufio.Scanner: token too long

Failures/Skipped:

https://example.com/sitemap_0.xml: Body: Error: bufio.Scanner: token too long

Total Duration: 0.582s
Count: 2, Failed: 1, Skipped: 0

Environment:

• Version of goss: 7b6cd899d94b4f0857b5ff3289b5a8d26edf1b09
• OS/Distribution version: I build and run on MacOS

[aelsabbahy]

Since you're compiling locally, can you try making the following change: https://github.com/aelsabbahy/goss/blob/master/resource/validate.go#L29

Change that line to:

    maxScanTokenSize = 10240 * 1024

If that works for you, feel free to submit a PR. I don't think a 10mb token size is too unreasonable.

[rmetzler]

hmm... this feels like a cheap workaround. It will probably do for now, but I hope to find time fixing this properly.

[aelsabbahy]

Sure, I'd love your thoughts on the proper fix.

[aelsabbahy]

Feel free to open up a new ticket with a proper fix. I'm guessing it would use bufio.Reader: https://golang.org/pkg/bufio/#Scanner