Uploading to devicefarm using the pre-signed URL gives signature mismatch

[jedi4ever]

I'm trying to upload to the presigned URL returned by the devicefarm CreateUpload call. I get an URL and I can PUT to it but it keeps complaining:

The request signature we calculated does not match the signature you provided. Check your key and signing method

• The file is read correctly
• I'm able to list and do createupload with my credentials which have for testing no restrictions.
• I've tried with and without headers, content-type with various spelling

Thank you for fixing/proving a sample to upload an ipa/apk to devicefarm.

 svc := devicefarm.New(&aws.Config{Region: aws.String("us-west-2")})

 name := "a.ipa"
myarn := "arn:aws:devicefarm:us-west-2:110440800955:project......"

uploadReq := &devicefarm.CreateUploadInput{
   Name:        aws.String(name),
   ProjectARN:  aws.String(myarn),
   Type:        aws.String("IOS_APP"),
   ContentType: aws.String("application/octet-stream"),
 }

resp, err := svc.CreateUpload(uploadReq)

  req, err := http.NewRequest("PUT", *resp.Upload.URL, data)
  req.Header.Set("Content-Type", "application/octet-stream")

  if err != nil {
    log.Fatal(err)
  }

  client := &http.Client{}
  res, err := client.Do(req)

  if err != nil {
    log.Fatal(err)
  }
  defer res.Body.Close()

[cristim]

Hi,

I had similar issues when uploading to S3, needed for CloudFormation custom resources written in golang.

The problem is caused by net/http, which un-escapes the special characters from the response URL(in my case colons and pipe characters), which is breaking the signature mechanism.

My workaround was to undo the un-escaping in the URL, using Opaque URLs:

req, err := http.NewRequest("PUT", event.ResponseURL, strings.NewReader(string(responseBody)))

req.URL.Opaque = strings.Replace(URL.Path, ":", "%3A", -1)
req.URL.Opaque = strings.Replace(req.URL.Opaque, "|", "%7C", -1)

req.Header.Set("content-length", strconv.Itoa(len(responseBody)))

client := &http.Client{}

resp, err := client.Do(req)

if err != nil {
    fmt.Println("Failed to set CloudFormation state", err.Error())
}

defer resp.Body.Close()

I hope this helps you, but if any of you guys can think of a better way, I'm all ears :-)

-Cristi

[cristim]

Apparently answering by email is not so well supported by Github, and looks like I can't properly format the answer's code anymore.

[jedi4ever]

Thanks cristim! You put me on the right track with the opaque trick. I've had to add the raw_query too

  fileToUpload := "a.ipa"
  file, err := os.Open(fileToUpload)

  if err != nil {
    fmt.Println(err)
    os.Exit(1)
  }

  defer file.Close()

  fileInfo, _ := file.Stat()
  var fileSize int64 = fileInfo.Size()

  buffer := make([]byte, fileSize)

  // read file content to buffer
  file.Read(buffer)

  fileBytes := bytes.NewReader(buffer) // convert to io.ReadSeeker type

 req, err := http.NewRequest("PUT", upload_url, fileBytes)

// Splitting the url by string function because golang URL parsing tries to be too clever
 strippedUrl := strings.Split(strings.Replace(upload_url, "https://prod-us-west-2-uploads.s3-us-west-2.amazonaws.com/", "/", -1), "?")
// The actual path
 req.URL.Opaque = strippedUrl[0]
// the part with AWS_KEY , Expire, ...
 req.URL.RawQuery = strippedUrl[1]

 req.Header.Set("Content-Type", "application/octet-stream")
 req.Header.Add("Content-Length", strconv.FormatInt(fileSize, 10))

client := &http.Client{}

    resp, err := client.Do(req)

    if err != nil {
        fmt.Println("Failed to set Upload To Devicefarm", err.Error())
    }

    defer resp.Body.Close()

[jedi4ever]

For more info on using the devicefarm API - I've made a devicefarm-cli.

The upload part can be found at https://github.com/jedi4ever/devicefarm-cli/blob/master/devicefarm-cli.go#L443