index out of range error when scanning large files

[simonmckenzie]

Hi,

I'm using the BitBucket coderefs pipe, and it's been working fine until recently, when it started failing with this error:

Status: Downloaded newer image for launchdarkly/ld-find-code-refs-bitbucket-pipeline:2.1.0
INFO: 2021/01/21 04:02:44 bitbucket-pipelines.go:24: Setting Bitbucket Pipelines env vars
INFO: 2021/01/21 04:02:44 coderefs.go:31: absolute directory path: /opt/atlassian/pipelines/agent/build
INFO: 2021/01/21 04:02:44 git.go:48: git branch: master
INFO: 2021/01/21 04:06:57 coderefs.go:137: sending 196 code references across 389 flags and 29 files to LaunchDarkly for project: $LAUNCH_DARKLY_PROJ_KEY
INFO: 2021/01/21 04:06:58 coderefs.go:166: checking if 220 flags without references were removed in the last 10 commits
panic: runtime error: index out of range [65533] with length 58951
goroutine 1 [running]:
github.com/sergi/go-diff/diffmatchpatch.(*DiffMatchPatch).DiffCharsToLines(0xc000b9f390, 0xc000330090, 0x1, 0x2, 0xc00220c000, 0xe647, 0xec00, 0x25800, 0xc000330090, 0x1)
    /Users/arnold/.gvm/pkgsets/go1.15/global/pkg/mod/github.com/sergi/go-diff@v1.1.0/diffmatchpatch/diff.go:452 +0x328
github.com/go-git/go-git/v5/utils/diff.DoWithTimeout(0xc00fadc000, 0xb9058c, 0x0, 0x0, 0x34630b8a000, 0x0, 0x0, 0x1)
    /Users/arnold/.gvm/pkgsets/go1.15/global/pkg/mod/github.com/go-git/go-git/v5@v5.1.0/utils/diff/diff.go:37 +0x1b8
github.com/go-git/go-git/v5/utils/diff.Do(...)
    /Users/arnold/.gvm/pkgsets/go1.15/global/pkg/mod/github.com/go-git/go-git/v5@v5.1.0/utils/diff/diff.go:22
github.com/go-git/go-git/v5/plumbing/object.filePatchWithContext(0xa27f20, 0xc0000b2030, 0xc001541e00, 0xa25ee0, 0xc0014f5cc0, 0x0, 0x0)
    /Users/arnold/.gvm/pkgsets/go1.15/global/pkg/mod/github.com/go-git/go-git/v5@v5.1.0/plumbing/object/patch.go:68 +0x23e
github.com/go-git/go-git/v5/plumbing/object.getPatchContext(0xa27f20, 0xc0000b2030, 0x0, 0x0, 0xc00000a5a0, 0x3c, 0x3c, 0x3c, 0x0, 0x0)
    /Users/arnold/.gvm/pkgsets/go1.15/global/pkg/mod/github.com/go-git/go-git/v5@v5.1.0/plumbing/object/patch.go:38 +0xce
github.com/go-git/go-git/v5/plumbing/object.Changes.PatchContext(...)
    /Users/arnold/.gvm/pkgsets/go1.15/global/pkg/mod/github.com/go-git/go-git/v5@v5.1.0/plumbing/object/change.go:158
github.com/go-git/go-git/v5/plumbing/object.Changes.Patch(0xc00000a5a0, 0x3c, 0x3c, 0x3c, 0x3c, 0x0)
    /Users/arnold/.gvm/pkgsets/go1.15/global/pkg/mod/github.com/go-git/go-git/v5@v5.1.0/plumbing/object/change.go:150 +0x5f
github.com/launchdarkly/ld-find-code-refs/internal/git.Client.FindExtinctions(0xc000024154, 0x24, 0xc00018e030, 0x6, 0xc0000b47b0, 0x28, 0xc0000220cc, 0x7, 0xc0000c2000, 0xdc, ...)
    /Users/arnold/code/go/src/github.com/launchdarkly/ld-find-code-refs/internal/git/git.go:145 +0x665
github.com/launchdarkly/ld-find-code-refs/pkg/coderefs.Scan(0xc000024010, 0x28, 0x9936a2, 0x1c, 0x0, 0x0, 0x0, 0x0, 0x9878cf, 0x6, ...)
    /Users/arnold/code/go/src/github.com/launchdarkly/ld-find-code-refs/pkg/coderefs/coderefs.go:167 +0x10df
main.main()
    /Users/arnold/code/go/src/github.com/launchdarkly/ld-find-code-refs/build/package/bitbucket-pipelines/bitbucket-pipelines.go:20 +0x558

Here is the pipeline yml that I'm using:

pipelines:
  branches:
    master:
      - step:
          name: Refresh LaunchDarkly code references
          services:
            - docker
          script:
            - pipe: launchdarkly/ld-find-code-refs-pipe:2.1.0
              environment:
                LD_ACCESS_TOKEN: $LAUNCH_DARKLY_ACCESS_TOKEN
                LD_PROJ_KEY: $LAUNCH_DARKLY_PROJ_KEY
definitions:
  services:
    docker:
      memory: 1536

I'm afraid I can't really give much more detail than this, as reproducing it involves the content of my codebase. Please tell me if there's anything I can do from my side to aid diagnosis.

[atrakh]

Hi @simonmckenzie,

Thank you for reporting this issue. This appears to be a bug in a transitive dependency used by one of the ancillary features of code references (flag extinctions).

I'll begin an investigation to remediate this issue. For the time being, you can work around this error by disabled extinctions by setting the lookback arg (LD_LOOKBACK as an env var) to 0.

If you have been able to pin this issue down to a particular change in your codebase and are comfortable sharing nonsensitive details about that, please feel free to contact me at our private support email - support@launchdarkly.com.

[simonmckenzie]

Thanks for looking into this, @atrakh.

I'm not sure whether this helps, but the merge that broke the pipeline was one that removed some React bundles from source control. These bundles were large .js files, and would have contained a lot of references to flags.

[atrakh]

Thanks for the additional context @simonmckenzie,

This issue appears to correlate well with a known bug diffing large files in a transitive dependency. We're working on producing a new version with a fix.

If you don't expect your React bundles to contain code references (or duplicative of the base source files), another way to resolve this issue for the time being would be by adding the bundle directory to your .ldignore file

I'll leave this issue open until we release a fix.

Cheers!

[simonmckenzie]

Hi @atrakh, Will there be a new release coming soon? I see that the update to go-diff has been released...

[atrakh]

Hi @simonmckenzie, I’m not working at LaunchDarkly any more, but I paged the team and it sounds like they’re aware of the latest update to go-diff. You should get an update on this issue soon!

[vlymar]

Hey @simonmckenzie, apologies for the radio silence. We'll be looking into pulling in the updated version of go-diff soon.

[InTheCloudDan]

This is still blocked until it gets merged into the git-go repo. Here is the original issue @atrakh opened https://github.com/go-git/go-git/issues/254

[chenrui333]

Is there any update on this issue?

[chenrui333]

This is still an issue.

[jazanne]

@chenrui333 v2.10.0 changes how extinction scanning works and should fix this issue.