search

algorand / go-algorand

Algorand's official implementation in Go.
https://developer.algorand.org/
Other
1.36k stars 474 forks source link

delete #2131

Closed ghost closed 2 years ago

ghost commented 3 years ago

During the voyage of life, remember to keep an eye out for a fair wind; batten down during a storm; hail all passing ships; and fly your colors proudly.

tsachiherman commented 3 years ago

@braderhart

All the issues you've mentioned above are on our boilerplate and being handled in one way or another. Some of the issues can't be easily solved, and might take a while before being addressed.

Note that none of the issues here would ever have a high priority since everything on the algorand toolchains works correctly ( from functional standpoint ). These build changes would primarily makes 3rd party integration slightly easier, which is nice to have, but not a blocker.

Regarding the VRF - it was implemented inside libsodium, so that it can take advantage of some of the construct available there, which would improve the performance. This is required in order to get to the advertised performance goals. ( nevertheless, I'm not saying that there aren't other ways to achieve the same goals )

The Go's standard crypto/ed25519, is a good implementation - but too slow for our purposes.

As for your comment regarding updating the libsodium upstream - there was a PR to the libsodium, which included our VRF implementation. That PR was rejected. At this time, no newer version of libsodium have any relevant changes, and therefore - we didn't update it. If you can point to a specific change in the libsodium that would benefit the Algorand platform, please open a concrete ticket.

tmc commented 3 years ago

@tsachiherman is there an issue/branch/etc. benchmarking the pure go implementation and the libsodium fork? I'm sure some crypto and performance interested folks would love to examine the delta and try to figure out how a pure go approach could be improved.

edit: I see https://github.com/algorand/go-algorand/blob/041e1f92d9c190bdc6d6c78b1dd04ef19b8ec03b/crypto/vrf_test.go#L92, I'll start pulling threads there.

brianolson commented 3 years ago

iirc, I tested Go crypto/ed25519 vs libsodium and the speed difference was about 2:1. The Algorand Go SDK uses the pure go implementation to make that a cleaner build. I think a client might be expected to manage dozens or hundreds of ed25519 operations per second, but algod will be doing thousands per second and needs the fastest code. CPU profiling always turns up ed25519 operations as a substantial chunk of CPU time.

winder commented 3 years ago

@braderhart I share your desire to simplify the shell scripts. There are a number of improvements that could be made to make the project more approachable for developers like yourself. That said, if you review these scripts I think you'll find that they're not as complicated as the number of scripts may suggest. There are some nice productivity features in them for things like automatically managing a local devnet node. The scripts also tie into some of our standard deployment tools, like update.sh which allow for continuous testing of the production deployment procedures.

If you prefer to skip the nonstandard helpers then make buildsrc may be what you're looking for. The actual compilation process starts there. It uses go install like you would expect.

The libsodium concerns are valid, but changes to this code must be taken very seriously. The code has already been audited so we cannot change it without good reason. As Tsachi mentioned we are working to improve things, if you would like to review some of that progress you can see it here: #2193.

And thanks for the feedback!

ghost commented 1 year ago

@winder, others, can someone please recreate the original issue text (whilst leaving the last text intact)?

Would look somehow like this:

During the voyage of life, remember to keep an eye out for a fair wind; batten down during a storm; hail all passing ships; and fly your colors proudly.

((Original Issue Text Follows))

Summary

This all started when I just wanted to get started using the CLI tools, and the instructions on the website suggested that I run a script on my computer. After following the script, it was just too convoluted for me to reliably trust that it would even be a good idea. I have built multiple Go applications from source, including those that use C libraries.

When building go-algorand from source, it is currently expected that people run a Makefile that is c [...] Would love to see this become a priority, as I think once it is done that it could be more easily adopted into Linux package repos and allow developers/engineers to get familiar with the tools a lot quicker and safely.