Remove AWS SDK dependency by default (split the fetcher package)

[kokizzu]

It seems it causes the binary very large by default (23MB with overseer, 19MB without overseer)

[kokizzu]

measurements:

goweight . | head -n 20
  8.7 MB github.com/go-redis/redis/v7
  7.3 MB github.com/aws/aws-sdk-go/service/s3
  3.9 MB net/http
  3.8 MB runtime
  3.3 MB github.com/gocql/gocql
  2.8 MB github.com/aws/aws-sdk-go/aws/endpoints
  2.5 MB github.com/valyala/fasthttp
  1.9 MB go.mongodb.org/mongo-driver/mongo
  1.8 MB net
  1.8 MB crypto/tls
  1.5 MB go.mongodb.org/mongo-driver/x/mongo/driver/operation
  1.5 MB golang.org/x/sys/unix
  1.4 MB reflect
  1.2 MB github.com/xdg/stringprep
  1.2 MB go.mongodb.org/mongo-driver/bson/bsoncodec
  1.2 MB go.mongodb.org/mongo-driver/x/mongo/driver
  1.1 MB go.mongodb.org/mongo-driver/bson/bsonrw
  1.0 MB github.com/klauspost/compress/zstd
  998 kB math/big
  916 kB go.mongodb.org/mongo-driver/x/mongo/driver/topology

[jpillora]

Will fix this. I can't remove S3 fetcher because that would break existing programs, so I'm in the process of swapping out the AWS SDK for a simple request signing approach.

On Sat, 28 Mar 2020 at 19:01, Kiswono Prayogo notifications@github.com wrote:

measurements:

goweight . | head -n 20 8.7 MB github.com/go-redis/redis/v7 7.3 MB github.com/aws/aws-sdk-go/service/s3 3.9 MB net/http 3.8 MB runtime 3.3 MB github.com/gocql/gocql 2.8 MB github.com/aws/aws-sdk-go/aws/endpoints 2.5 MB github.com/valyala/fasthttp 1.9 MB go.mongodb.org/mongo-driver/mongo 1.8 MB net 1.8 MB crypto/tls 1.5 MB go.mongodb.org/mongo-driver/x/mongo/driver/operation 1.5 MB golang.org/x/sys/unix 1.4 MB reflect 1.2 MB github.com/xdg/stringprep 1.2 MB go.mongodb.org/mongo-driver/bson/bsoncodec 1.2 MB go.mongodb.org/mongo-driver/x/mongo/driver 1.1 MB go.mongodb.org/mongo-driver/bson/bsonrw 1.0 MB github.com/klauspost/compress/zstd 998 kB math/big 916 kB go.mongodb.org/mongo-driver/x/mongo/driver/topology

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/jpillora/overseer/issues/44#issuecomment-605411020, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAE2X466FJRZWJQ4RCA4RGLRJWVHDANCNFSM4LVN6BFA .

[kokizzu]

oh nevermind then, just ignore my PR XD

[jpillora]

Please test 6877a7e73f6a4bebbfe1999fcbf84e52a3c1e6ca

[jpillora]

$ ..../overseer goweight .
  4.0 MB net/http
  3.9 MB runtime
  1.8 MB crypto/tls
  1.8 MB net
  1.4 MB reflect
  1.0 MB math/big
  869 kB syscall
  865 kB crypto/x509
  670 kB encoding/json
  621 kB vendor/golang.org/x/net/dns/dnsmessage
  545 kB vendor/golang.org/x/text/unicode/norm
  509 kB time
  469 kB fmt
  425 kB internal/reflectlite
  414 kB os
  368 kB compress/flate
  327 kB crypto/elliptic
  327 kB github.com/jpillora/s3
  311 kB vendor/golang.org/x/crypto/cryptobyte
  310 kB encoding/asn1
  292 kB strings
  291 kB vendor/golang.org/x/text/unicode/bidi
  290 kB strconv
  261 kB os/exec
  238 kB vendor/golang.org/x/net/idna
  234 kB mime

[kokizzu]

It works fine :3 thanks a bunch!

Before:
21983101 Mar 28 23:03 ApiServer*
After
18857547 Mar 28 23:22 ApiServer*

goweight . | head -n 20
  3.9 MB net/http
  3.8 MB runtime
  3.3 MB github.com/gocql/gocql
  3.0 MB gopkg.in/redis.v5
  2.5 MB github.com/valyala/fasthttp
  1.9 MB go.mongodb.org/mongo-driver/mongo
  1.8 MB net
  1.8 MB crypto/tls
  1.5 MB go.mongodb.org/mongo-driver/x/mongo/driver/operation
  1.5 MB golang.org/x/sys/unix
  1.4 MB reflect
  1.2 MB github.com/xdg/stringprep
  1.2 MB go.mongodb.org/mongo-driver/bson/bsoncodec
  1.2 MB go.mongodb.org/mongo-driver/x/mongo/driver
  1.1 MB go.mongodb.org/mongo-driver/bson/bsonrw
  1.0 MB github.com/klauspost/compress/zstd
  998 kB math/big
  916 kB go.mongodb.org/mongo-driver/x/mongo/driver/topology
  847 kB syscall
  828 kB go.mongodb.org/mongo-driver/x/bsonx