burndown with granularity != sampling panics

bobheadxi commented 5 years ago

There doesn't seem to be anything in the documentation explicitly saying that granularity can't be different from burndown, so could this be a bug introduced by #245 ? I read through the burndown code and where it panics (when grouping histories), but can't seem to decipher what might be going on.

I'm using V10:

❯ hercules version
Version: 10
Git:     bf6626c0210a1446d6b494c0af25736c83bc01c0

sampling != burndown, default tick size


hercules --sampling=7 --burndown https://github.com/bobheadxi/calories.git       
finalizing...2019/03/25 14:14:08 Failed to run the pipeline on [https://github.com/bobheadxi/calories.git]
panic: runtime error: index out of range

goroutine 1 [running]: gopkg.in/src-d/hercules.v10/leaves.(BurndownAnalysis).groupSparseHistory(0xc0001d6700, 0xc000b75410, 0xffffffffffffffff, 0x20300000000000, 0x65fffff, 0xc000da34c0, 0x4017a9a) /Users/robertlin/go/src/gopkg.in/src-d/hercules.v10/leaves/burndown.go:1509 +0x681 gopkg.in/src-d/hercules.v10/leaves.(BurndownAnalysis).Finalize(0xc0001d6700, 0x4d82580, 0xc000104600) /Users/robertlin/go/src/gopkg.in/src-d/hercules.v10/leaves/burndown.go:504 +0x55 gopkg.in/src-d/hercules.v10/internal/core.(Pipeline).Run(0xc000da3cc8, 0xc00048b400, 0x61, 0x80, 0x0, 0x0, 0x0) /Users/robertlin/go/src/gopkg.in/src-d/hercules.v10/internal/core/pipeline.go:852 +0x6d1 main.glob..func3(0x57c9820, 0xc000a5f5c0, 0x1, 0x3) /Users/robertlin/go/src/gopkg.in/src-d/hercules.v10/cmd/hercules/root.go:270 +0x865 gopkg.in/src-d/hercules.v10/vendor/github.com/spf13/cobra.(Command).execute(0x57c9820, 0xc0000e6010, 0x3, 0x3, 0x57c9820, 0xc0000e6010) /Users/robertlin/go/src/gopkg.in/src-d/hercules.v10/vendor/github.com/spf13/cobra/command.go:766 +0x2ae gopkg.in/src-d/hercules.v10/vendor/github.com/spf13/cobra.(Command).ExecuteC(0x57c9820, 0xc0000e99e0, 0xc0001bbf88, 0x400821f) /Users/robertlin/go/src/gopkg.in/src-d/hercules.v10/vendor/github.com/spf13/cobra/command.go:852 +0x2c0 gopkg.in/src-d/hercules.v10/vendor/github.com/spf13/cobra.(Command).Execute(...) /Users/robertlin/go/src/gopkg.in/src-d/hercules.v10/vendor/github.com/spf13/cobra/command.go:800 main.main() /Users/robertlin/go/src/gopkg.in/src-d/hercules.v10/cmd/hercules/root.go:510 +0x32


* sampling == burndown works fine

❯ hercules --sampling=7 --granularity=7 --burndown https://github.com/bobheadxi/calories.git hercules: version: 10 hash: bf6626c0210a1446d6b494c0af25736c83bc01c0 repository: https://github.com/bobheadxi/calories.git begin_unix_time: 1509843192 end_unix_time: 1522909723 commits: 97 run_time: 34 Burndown: granularity: 7 sampling: 7 tick_size: 24h0m0s "project": |-


* sampling == burndown, with custom tick size works fine as well

❯ hercules --tick-size=7 --sampling=7 --granularity=7 --burndown https://github.com/bobheadxi/calories.git hercules: version: 10 hash: bf6626c0210a1446d6b494c0af25736c83bc01c0 repository: https://github.com/bobheadxi/calories.git begin_unix_time: 1509843192 end_unix_time: 1522909723 commits: 97 run_time: 40 Burndown: granularity: 7 sampling: 7 tick_size: 7h0m0s "project": |-

bobheadxi commented 5 years ago

Just tried it with v9 and can confirm it was an issue back then, so this might not be related to #245 :

❯ hercules version
Version: 9
Git:     c8fd37d1bcc1163e829d20a8e818b36e4930dcc5

❯ hercules --sampling=7 --burndown https://github.com/bobheadxi/calories.git                
finalizing...2019/03/25 14:24:44 Failed to run the pipeline on [https://github.com/bobheadxi/calories.git]
panic: runtime error: index out of range

goroutine 1 [running]:
gopkg.in/src-d/hercules.v9/leaves.(*BurndownAnalysis).groupSparseHistory(0xc00104eb40, 0xc000f2e2d0, 0xffffffffffffffff, 0xc000def468, 0x40384d4, 0xc000000180, 0x200000003)
        /Users/robertlin/go/src/gopkg.in/src-d/hercules.v9/leaves/burndown.go:1471 +0x67e
gopkg.in/src-d/hercules.v9/leaves.(*BurndownAnalysis).Finalize(0xc00104eb40, 0x4d7ae00, 0xc000f0bd10)
        /Users/robertlin/go/src/gopkg.in/src-d/hercules.v9/leaves/burndown.go:489 +0x55
gopkg.in/src-d/hercules.v9/internal/core.(*Pipeline).Run(0xc000defcc8, 0xc0004a3000, 0x61, 0x80, 0x0, 0x0, 0x0)
        /Users/robertlin/go/src/gopkg.in/src-d/hercules.v9/internal/core/pipeline.go:852 +0x6d1
main.glob..func3(0x57b7800, 0xc000a68b40, 0x1, 0x3)
        /Users/robertlin/go/src/gopkg.in/src-d/hercules.v9/cmd/hercules/root.go:270 +0x865
gopkg.in/src-d/hercules.v9/vendor/github.com/spf13/cobra.(*Command).execute(0x57b7800, 0xc0000320d0, 0x3, 0x3, 0x57b7800, 0xc0000320d0)
        /Users/robertlin/go/src/gopkg.in/src-d/hercules.v9/vendor/github.com/spf13/cobra/command.go:766 +0x2ae
gopkg.in/src-d/hercules.v9/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0x57b7800, 0xc000a98350, 0xc000199f88, 0x400787f)
        /Users/robertlin/go/src/gopkg.in/src-d/hercules.v9/vendor/github.com/spf13/cobra/command.go:852 +0x2c0
gopkg.in/src-d/hercules.v9/vendor/github.com/spf13/cobra.(*Command).Execute(...)
        /Users/robertlin/go/src/gopkg.in/src-d/hercules.v9/vendor/github.com/spf13/cobra/command.go:800
main.main()
        /Users/robertlin/go/src/gopkg.in/src-d/hercules.v9/cmd/hercules/root.go:510 +0x32

vmarkovtsev commented 5 years ago

Sampling and burndown are not bound together and can be arbitrary. According to the stack trace, there is an invalid commit "from the future". Actually, groupSparseHistory is trivial compared to addBurndownMatrix so it could be much worse :smile:

You should debug how this record "from the future" appeared.

bobheadxi commented 5 years ago

Hm, the same issue occurs with a burndown on Hercules itself on V9 of the CLI:

❯ hercules version
Version: 9
Git:     c8fd37d1bcc1163e829d20a8e818b36e4930dcc5

❯ hercules --sampling=7 --burndown https://github.com/src-d/hercules.git
finalizing...2019/03/25 14:36:50 Failed to run the pipeline on [https://github.com/src-d/hercules.git]
panic: runtime error: index out of range

goroutine 1 [running]:
gopkg.in/src-d/hercules.v9/leaves.(*BurndownAnalysis).groupSparseHistory(0xc00538ab40, 0xc00538ee40, 0xffffffffffffffff, 0xc00667f468, 0x40384d4, 0xc000000180, 0x200000003)
        /Users/robertlin/go/src/gopkg.in/src-d/hercules.v9/leaves/burndown.go:1471 +0x67e
gopkg.in/src-d/hercules.v9/leaves.(*BurndownAnalysis).Finalize(0xc00538ab40, 0x4d7ae00, 0xc00538ab40)
        /Users/robertlin/go/src/gopkg.in/src-d/hercules.v9/leaves/burndown.go:489 +0x55
gopkg.in/src-d/hercules.v9/internal/core.(*Pipeline).Run(0xc00667fcc8, 0xc0051f0000, 0x33a, 0x400, 0x0, 0x0, 0x0)
        /Users/robertlin/go/src/gopkg.in/src-d/hercules.v9/internal/core/pipeline.go:852 +0x6d1
main.glob..func3(0x57b7800, 0xc000a55530, 0x1, 0x3)
        /Users/robertlin/go/src/gopkg.in/src-d/hercules.v9/cmd/hercules/root.go:270 +0x865
gopkg.in/src-d/hercules.v9/vendor/github.com/spf13/cobra.(*Command).execute(0x57b7800, 0xc0000c4010, 0x3, 0x3, 0x57b7800, 0xc0000c4010)
        /Users/robertlin/go/src/gopkg.in/src-d/hercules.v9/vendor/github.com/spf13/cobra/command.go:766 +0x2ae
gopkg.in/src-d/hercules.v9/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0x57b7800, 0xc000079650, 0xc00017df88, 0x400787f)
        /Users/robertlin/go/src/gopkg.in/src-d/hercules.v9/vendor/github.com/spf13/cobra/command.go:852 +0x2c0
gopkg.in/src-d/hercules.v9/vendor/github.com/spf13/cobra.(*Command).Execute(...)
        /Users/robertlin/go/src/gopkg.in/src-d/hercules.v9/vendor/github.com/spf13/cobra/command.go:800
main.main()
        /Users/robertlin/go/src/gopkg.in/src-d/hercules.v9/cmd/hercules/root.go:510 +0x32

vmarkovtsev commented 5 years ago

Yep, crashes for me too. Any combination with granularity > sampling crashes. Can it be a feature, not a bug :thinking:

src-d / hercules

burndown with granularity != sampling panics #260