search

merklecounty / rget

download URLs and verify the contents against a publicly recorded cryptographic log
https://merklecounty.com
Apache License 2.0
205 stars 17 forks source link

`rget github publish-release-sums` panics #24

Open luxas opened 5 years ago

luxas commented 5 years ago

While running the publish-release-sums command; first I thought the command had hanged, because after the Server gracefully stopped log text, nothing was outputted in a couple of minutes. Some debug message to say e.g. Publishing shasums to the Github release... or similar would have helped.

After this, it paniced: (probably just a retry need to be added; due to a request failing)

$ rget github publish-release-sums <redacted>
2019/08/07 08:29:32 You will now be taken to your browser for authentication or open the url below in a browser.
2019/08/07 08:29:32 <redacted>
2019/08/07 08:29:32 If you are opening the url manually on a different machine you will need to curl the result url on this machine manually.
2019/08/07 08:29:34 Authentication will be cancelled in 120 seconds
2019/08/07 08:29:35 Shutting down server...
Server gracefully stopped
panic: read tcp 10.37.71.52:43166->52.216.96.75:443: read: connection reset by peer

goroutine 1 [running]:
go.merklecounty.com/rget/rgethash.readerDigest(0x1021bc0, 0xc0005c46c0, 0xc0005c46c0, 0x1021bc0, 0xc0005c46c0, 0xc00033e240, 0x0)
    /home/brandon/src/github.com/merklecounty/rget/rgethash/sgethash.go:121 +0x12d
go.merklecounty.com/rget/rgethash.(*URLSumList).AddURL(0xc000195a60, 0xc000358240, 0x53, 0x0, 0x0)
    /home/brandon/src/github.com/merklecounty/rget/rgethash/sgethash.go:57 +0x13e
go.merklecounty.com/rget/rget/github.publishReleaseSumsMain(0x1784ac0, 0xc0001ec150, 0x1, 0x1)
    /home/brandon/src/github.com/merklecounty/rget/rget/github/publish-release-sums.go:90 +0x433
github.com/spf13/cobra.(*Command).execute(0x1784ac0, 0xc0001ec110, 0x1, 0x1, 0x1784ac0, 0xc0001ec110)
    /home/brandon/go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:830 +0x2ae
github.com/spf13/cobra.(*Command).ExecuteC(0x1784840, 0xc000199f68, 0xc83f3e, 0x1784840)
    /home/brandon/go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:914 +0x2fc
github.com/spf13/cobra.(*Command).Execute(...)
    /home/brandon/go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:864
go.merklecounty.com/rget/rget/cmd.Execute()
    /home/brandon/src/github.com/merklecounty/rget/rget/cmd/root.go:64 +0x32
main.main()
    /home/brandon/src/github.com/merklecounty/rget/rget/main.go:20 +0x20
philips commented 5 years ago

Yes, it can take quite some time due to connection speed or size of binaries on GitHub. Unfortunately this command has to download all of the files, and then run the cryptographic digest over them.

It would be nice to print over each file as the download is started. I will try and do that in the next release.

As for the panic yes it seems like a place where there is need to put a retry loop.