API: uploading of the WebExtension: 301, `HTTP error before end of send, stop sending`

[vitaly-zdanevich]

According to this page I am trying to upload my WebExtension through the REST API. My code is:

curl https://addons.mozilla.org/api/v3/addons \
    --form "upload=@$FIREFOX_FILEPATH" \
    --form "version=$VERSION" \
    -H "Authorization: JWT $JWT" \
    -v

I am getting:

*   Trying 34.208.172.96...
* TCP_NODELAY set
* Connected to addons.mozilla.org (34.208.172.96) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: businessCategory=Private Organization; 1.3.6.1.4.1.311.60.2.1.3=US; 1.3.6.1.4.1.311.60.2.1.2=California; serialNumber=C2543436; C=US; ST=California; L=Mountain View; O=Mozilla Foundation; OU=Cloud Services; CN=addons.mozilla.org
*  start date: Sep 28 00:00:00 2017 GMT
*  expire date: Oct  4 12:00:00 2019 GMT
*  subjectAltName: host "addons.mozilla.org" matched cert's "addons.mozilla.org"
*  issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=DigiCert SHA2 Extended Validation Server CA
*  SSL certificate verify ok.
> POST /api/v3/addons HTTP/1.1
> Host: addons.mozilla.org
> User-Agent: curl/7.54.0
> Accept: */*
> Authorization: JWT eyJhbGciOiAiSXXXXXXXXXidHlwIjogIkpXVCJ9.eyJpc3MXXXXXXXXXXEwOjk5NyIsImp0aSI6IjgwMjIwY2U1ZWYwMThkMzg1N2EwOTRiNDBlZjc2ZWFjNzFjZDZjMTMiLCJpYXQiOiIxNTE4MDkwMDUyIiwiZXhwIjoiMTUxODA5MDE1MSJ9.KL+0ay1XXXXXXX4b1EibBWee0=
> Content-Length: 69837
> Expect: 100-continue
> Content-Type: multipart/form-data; boundary=------------------------681abe707e5861c6
> 
< HTTP/1.1 100 Continue
< HTTP/1.1 301 MOVED PERMANENTLY
< Content-Security-Policy: script-src https://ssl.google-analytics.com/ga.js https://www.google.com/recaptcha/ https://www.gstatic.com/recaptcha/ https://addons.cdn.mozilla.net; style-src 'self' 'unsafe-inline' https://addons.cdn.mozilla.net; default-src 'self'; frame-src 'self' https://www.google.com/recaptcha/; child-src 'self' https://www.google.com/recaptcha/; img-src 'self' data: blob: https://ssl.google-analytics.com https://addons.cdn.mozilla.net https://static.addons.mozilla.net https://sentry.prod.mozaws.net; media-src https://videos.cdn.mozilla.net; object-src 'none'; connect-src 'self' https://sentry.prod.mozaws.net; font-src 'self' https://addons.cdn.mozilla.net; form-action 'self' https://developer.mozilla.org; base-uri 'self' https://addons.mozilla.org; report-uri /__cspreport__
< Content-Type: text/html; charset=utf-8
< Date: Thu, 08 Feb 2018 11:40:53 GMT
< ETag: "d41d8cd98f00b204e9800998ecf8427e"
< Location: https://addons.mozilla.org/api/v3/addons/
< Public-Key-Pins: max-age=5184000; includeSubDomains; pin-sha256="WoiWRyIOVNa9ihaBciRSC7XHjliYS9VwUGOIud4PB18="; pin-sha256="r/mIkG3eEpVdm+u/ko/cwxzOMo1bk4TyHIlByibiA5E="
< Server: nginx
< Set-Cookie: multidb_pin_writes=y; expires=Thu, 08-Feb-2018 11:41:08 GMT; Max-Age=15; Path=/
< strict-transport-security: max-age=31536000
< Vary: User-Agent
< x-content-type-options: nosniff
< X-Frame-Options: DENY
< x-xss-protection: 1; mode=block
< Content-Length: 0
< Connection: keep-alive
* HTTP error before end of send, stop sending
< 
* Closing connection 0
* TLSv1.2 (OUT), TLS alert, Client hello (1):

I successfully checked my JWT at jwt.io.

[diox]

The docs do not say that you can omit the trailing slash - do include it! That's why you are getting a redirect, in fact.

[vitaly-zdanevich]

Now I get 500 :(

[diox]

Without more information I suspect you're hitting https://github.com/mozilla/addons/issues/4989 because your JWT is incorrect (make sure the fields that should be integers, like iat and exp, are passed as integers and not as strings)

[vitaly-zdanevich]

Can my JWT be incorrect if I am getting greenlight from jwt.io?

[diox]

Probably because it's less strict than our own implementation, which does not like strings for things that should be NumericDate per JWT spec. As mentioned in mozilla/addons#4989 we have a bug that causes us to return a 500 in that case, but it's still invalid. Don't use strings for those properties.

[vitaly-zdanevich]

Ok, now iat and exp as numbers, getting 400 with another strange message: {"error":"Duplicate add-on ID found."} How it can be possible?

I can successfully upload next version through web UI.

[diox]

Again it's difficult to say without more information, but it probably means you are not using the right API - you are probably using POST /api/v3/addons/, which creates a new add-on, instead of PUT /api/v3/addons/[string:addon-id]/versions/[string:version]/, which creates a new version. See the docs for more information.

[vitaly-zdanevich]

But this api is POST, for Uploading without an ID.

[vitaly-zdanevich]

My code:

AMO_JWT_ISSUER=user:1321XXXXXX:XXX
AMO_SECRET=503084ad4069208XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
JWT_HEADER_BASE64=`printf %s '{"alg": "HS256", "typ": "JWT"}' | base64`

random() {
    dd if=/dev/urandom bs=20 count=1 2>/dev/null | openssl sha1
}
UNIXTIME=`date +%s`
PAYLOAD_JSON=$(python3 - <<EOF
import json
print(
    json.dumps({
        'iss': "${AMO_JWT_ISSUER}",
        'jti': "$(random)",
        'iat': $UNIXTIME,
        'exp': $(($UNIXTIME+99))
    })
)
EOF
)
JWT_PAYLOAD_BASE64=`printf %s $PAYLOAD_JSON | base64`
FOR_SIGN="$JWT_HEADER_BASE64.$JWT_PAYLOAD_BASE64"
JWT_SIGNATURE_BASE64=`printf %s $FOR_SIGN | openssl dgst -binary -sha256 -hmac $AMO_SECRET | base64`
JWT=$JWT_HEADER_BASE64.$JWT_PAYLOAD_BASE64.$JWT_SIGNATURE_BASE64

curl "https://addons.mozilla.org/api/v3/addons/" \
    --form "upload=@$FIREFOX_FILEPATH" \
    --form "version=$VERSION" \
    -H "Authorization: JWT $JWT" \
    -v

Also it would be helpful for future developers to add some code examples to the documentation, in pure shell, Python3, node.

[vitaly-zdanevich]

@diox something wrong with my code? I cannot find any error above...

[diox]

Sorry, I don't have time to debug what's wrong with your code. We do have a node module implementing this API: https://github.com/mozilla/sign-addon/ and this is used by a CLI client: https://developer.mozilla.org/en-US/Add-ons/WebExtensions/Getting_started_with_web-ext