Description

I'd like to use aptly to publish packages to S3. The upload-artifacts.sh script example provided on Github fails with http code 307 and my S3 bucket is empty.

Logs in the journal:

Apr 01 21:04:42 ip-172-31-20-168 aptly[24785]: [GIN] 2020/04/01 - 21:04:42 | 200 |   22.195651ms |    86.220.91.60 | POST     /api/repos/myrepo/file/tmp.4gUVrNh68HIobix
Apr 01 21:04:42 ip-172-31-20-168 aptly[24785]: [GIN-debug] redirecting request 307: /api/publish/s3:foobar:./ --> /api/publish/s3:foobar:.

I'm not sure what aptly_repository aptly_published values should be. Should they match specific options set in /etc/aptly.conf configuration file?

Environment

# dpkg -l aptly
ii  aptly          1.3.0+ds1-2.2~deb10u1 amd64        Swiss army knife for Debia
lines 1-6/6 (END)

Publishing with the CLI

Publishing to S3 via the CLI is successful and the files are uploaded to S3. I guess communication with AWS S3 is OK.

gondaz commented 4 years ago

I bumped into the same issue, but along with a 307 error (possibly because of an unneeded trailing slash "/" in the end of the request) there is another problem crops up.

When I try either to publish a local repo to S3 or update an already published repo via API I get the following error response: Trying to publish a repo to S3: # curl --user <user>:<password> -X POST -H 'Content-Type: application/json' --data '{"SourceKind": "local", "Sources": [{"Name": "local_repo"}], "Architectures": ["amd64"], "Distribution": "bionic"}' https://example.com/api/publish/s3:bucket_name:local And as a result:

апр 03 13:09:35 apt-repo aptly[1208]: &{%!s(int=1) published S3 storage publishes not configured}
апр 03 13:09:35 apt-repo aptly[1208]: /home/travis/gopath/src/github.com/aptly-dev/aptly/context/context.go:70 (0xb48ae7)
апр 03 13:09:35 apt-repo aptly[1208]: /home/travis/gopath/src/github.com/aptly-dev/aptly/context/context.go:343 (0xb4b024)
апр 03 13:09:35 apt-repo aptly[1208]: /home/travis/gopath/src/github.com/aptly-dev/aptly/deb/publish.go:491 (0x8fbbe0)
апр 03 13:09:35 apt-repo aptly[1208]: /home/travis/gopath/src/github.com/aptly-dev/aptly/api/publish.go:216 (0xb50f6c)
апр 03 13:09:35 apt-repo aptly[1208]: /home/travis/gopath/src/github.com/aptly-dev/aptly/vendor/github.com/gin-gonic/gin/context.go:104 (0x98a952)
апр 03 13:09:35 apt-repo aptly[1208]: /home/travis/gopath/src/github.com/aptly-dev/aptly/api/router.go:47 (0xb5be16)
апр 03 13:09:35 apt-repo aptly[1208]: /home/travis/gopath/src/github.com/aptly-dev/aptly/vendor/github.com/gin-gonic/gin/context.go:104 (0x98a952)
апр 03 13:09:35 apt-repo aptly[1208]: /home/travis/gopath/src/github.com/aptly-dev/aptly/vendor/github.com/gin-gonic/gin/logger.go:38 (0x99a187)
апр 03 13:09:35 apt-repo aptly[1208]: /home/travis/gopath/src/github.com/aptly-dev/aptly/vendor/github.com/gin-gonic/gin/context.go:104 (0x98a952)
апр 03 13:09:35 apt-repo aptly[1208]: /home/travis/gopath/src/github.com/aptly-dev/aptly/vendor/github.com/gin-gonic/gin/recovery.go:45 (0x99ae09)
апр 03 13:09:35 apt-repo aptly[1208]: /home/travis/gopath/src/github.com/aptly-dev/aptly/vendor/github.com/gin-gonic/gin/context.go:104 (0x98a952)
апр 03 13:09:35 apt-repo aptly[1208]: /home/travis/gopath/src/github.com/aptly-dev/aptly/vendor/github.com/gin-gonic/gin/logger.go:79 (0x99a306)
апр 03 13:09:35 apt-repo aptly[1208]: /home/travis/gopath/src/github.com/aptly-dev/aptly/vendor/github.com/gin-gonic/gin/context.go:104 (0x98a952)
апр 03 13:09:35 apt-repo aptly[1208]: /home/travis/gopath/src/github.com/aptly-dev/aptly/vendor/github.com/gin-gonic/gin/gin.go:332 (0x992335)
апр 03 13:09:35 apt-repo aptly[1208]: /home/travis/gopath/src/github.com/aptly-dev/aptly/vendor/github.com/gin-gonic/gin/gin.go:296 (0x991b73)
апр 03 13:09:35 apt-repo aptly[1208]: /home/travis/.gimme/versions/go1.12.6.linux.amd64/src/net/http/server.go:2774 (0x721f87)
апр 03 13:09:35 apt-repo aptly[1208]: /home/travis/.gimme/versions/go1.12.6.linux.amd64/src/net/http/server.go:1878 (0x71db70)
апр 03 13:09:35 apt-repo aptly[1208]: /home/travis/.gimme/versions/go1.12.6.linux.amd64/src/runtime/asm_amd64.s:1337 (0x45a810)
апр 03 13:09:35 apt-repo aptly[1208]: 

aptly[1208]: [GIN] 2020/04/03 - 12:43:15 | 500 |   19.558747ms  |  PUT     /api/publish/s3:bucket_name:local

The same error while updating a published repo. Meanwhile, a CLI commands working properly, a repo can be published to S3 successfully: # aptly publish repo local_repo s3:bucket_name:local/

My config for S3 is as following:

  "S3PublishEndpoints": {
  "bucket_name": {
    "region": "us-east-1",
    "endpoint": "example.com",
    "bucket": "bucket_name",
    "acl": "public-read",
    "awsAccessKeyID": "{key_id}",
    "awsSecretAccessKey": "{secret_key}"
    }
  }

I assume there is a bug in API, so I would be endlessly thankful if someone helps me to fix this.

turbinenreiter commented 4 years ago

I have the same problem. I'm not sure if this is a bug or if I just don't make right request.

This works: aptly publish -distribution=testing repo reponame s3:test:reponame

But trying to do the same via API does not work: curl -X POST -H 'Content-Type: application/json' --data '{"SourceKind": "local", "Sources": [{"Name": "reponame"}], "Architectures": ["amd64"], "Distribution": "testing"}' http://blabla/api/publish/s3:test

There is no error message, but also no files uploaded to the bucket.

aptly-dev / aptly

Publish API returning 307 and files not uploaded on S3 #917

Description

Environment

Publishing with the CLI