service/iot: Add MQTT over websockets client

dhubler commented 8 years ago

It only has Topic Publish.

This is most likely because SDK needs to use websocket protocol to get bidirectional communication.

Even if this is closed as "Will not implement", this should stand as notice API is incomplete compared with JS and Java SDKs feature set.

Also Note in source file service/iotdataplane/service.go We see comment:

// AWS IoT-Data enables secure, bi-directional communication between Internet-connected
// things (such as sensors, actuators, embedded devices, or smart appliances)

While AWS IoT API is bi-direction capable, the AWS IoT Golang SDK is not bi-directional

xibz commented 8 years ago

Hello @dhubler, thank you for reaching out to us. Those are service specific SDKs, the JS and Java SDK, and currently are only supported by those two SDKs. This would be a great feature and I will mark this as a feature request.

dhubler commented 8 years ago

As a work around, If anyone is interested, I followed instructions in http://docs.aws.amazon.com/iot/latest/developerguide/protocols.html#http ported code to golang and using paho MQTT over websockets protocol from Golang SDK.

func AwsIotWsUrl(accessKey string, secretKey string, region string, endpoint string) string {
    host := fmt.Sprintf("%s.iot.%s.amazonaws.com", endpoint, region)

    // according to docs, time must be within 5min of actual time (or at least according to AWS servers)
    now := time.Now().UTC()

    dateLong := now.Format("20060102T150405Z")
    dateShort := dateLong[:8]
    serviceName := "iotdevicegateway"
    scope := fmt.Sprintf("%s/%s/%s/aws4_request", dateShort, region, serviceName)
    alg := "AWS4-HMAC-SHA256"
    q := [][2]string{
        {"X-Amz-Algorithm", alg},
        {"X-Amz-Credential", accessKey + "/" + scope},
        {"X-Amz-Date", dateLong},
        {"X-Amz-SignedHeaders", "host"},
    }
    query := awsQueryParams(q)

    signKey := awsSignKey(secretKey, dateShort, region, serviceName)
    stringToSign := awsSignString(accessKey, secretKey, query, host, dateLong, alg, scope)
    signature := fmt.Sprintf("%x", awsHmac(signKey, []byte(stringToSign)))

    return fmt.Sprintf("wss://%s/mqtt?%s&X-Amz-Signature=%s", host, query, signature)
}

func awsQueryParams(q [][2]string) string {
    var buff bytes.Buffer
    var i int
    for _, param := range q {
        if i != 0 {
            buff.WriteRune('&')
        }
        i++
        buff.WriteString(param[0])
        buff.WriteRune('=')
        buff.WriteString(url.QueryEscape(param[1]))
    }
    return buff.String()
}

func awsSignString(accessKey string, secretKey string, query string, host string, dateLongStr string, alg string, scopeStr string) string {
    emptyStringHash := "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
    req := strings.Join([]string{
        "GET",
        "/mqtt",
        query,
        "host:" + host,
        "", // separator
        "host",
        emptyStringHash,
    }, "\n")
    return strings.Join([]string{
        alg,
        dateLongStr,
        scopeStr,
        awsSha(req),
    }, "\n")
}

func awsHmac(key []byte, data []byte) []byte {
    h := hmac.New(sha256.New, key)
    h.Write(data)
    return h.Sum(nil)
}

func awsSignKey(secretKey string, dateShort string, region string, serviceName string) []byte {
    h := awsHmac([]byte("AWS4"+secretKey), []byte(dateShort))
    h = awsHmac(h, []byte(region))
    h = awsHmac(h, []byte(serviceName))
    h = awsHmac(h, []byte("aws4_request"))
    return h
}

func awsSha(in string) string {
    h := sha256.New()
    fmt.Fprintf(h, "%s", in)
    return fmt.Sprintf("%x", h.Sum(nil))
}

jeanbza commented 7 years ago

+1 this, we're really hurting not being able to pubsub from iot topics

aidansteele commented 6 years ago

FWIW I made a library to fill the gap until the AWS SDK provides support for this: https://github.com/glassechidna/awsiot

sess := session.Must(session.NewSessionWithOptions(sessOpts))
iot := awsiot.New(sess)
theUrl, _ := iot.WebsocketUrl("a1kxjqeyezkt7") // can be used with the eclipse paho mqtt library

swt2c commented 6 years ago

Just a note, as it took me a while to figure this out. It seems that @dhubler and @aidansteele solutions don't work (anymore?). IOT seems to want the X-Amz-Security-Token parameter, but it cannot be part of the canonical query parameters. It has to be added on after signing.

aidansteele commented 6 years ago

@swt2c I'll check that out. I had it working a couple of weeks ago, haven't tried since.

noliva commented 6 years ago

@swt2c @aidansteele Did you got this working? i've used @dhubler solution and appended the securityToken.. but i still get a 403 ..

swt2c commented 6 years ago

Yes, I did get it working at the time. You included the security token outside of the signature?

noliva commented 6 years ago

@swt2c yes, did you use @dhubler example of the library by @aidansteele ?

swt2c commented 6 years ago

Yes, but unfortunately, I'm not using it anymore and don't have access to the code I was using.

dhubler commented 6 years ago

One thing to know even if you get this working, websocket connection will reset every 24 hours. Ultimately using straight MQTT and TLS was most reliable.

On Mon, Jul 2, 2018, 8:36 AM Scott Talbert notifications@github.com wrote:

Yes, but unfortunately, I'm not using it anymore and don't have access to the code I was using.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/aws/aws-sdk-go/issues/820#issuecomment-401845527, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAgfip3PIZCg3P6uWC6cA8Jl1WVDpk7ks5uCj3ugaJpZM4Jwdvx .

ytwig commented 5 years ago

The solution provided by @aidansteele is still working great. Tested just now

Pitasi commented 4 years ago

I just had a hard time to get this working and want to share what I ended up doing.

aidansteele's solution worked on my laptop but not inside a lambda
dhubler's solution didn't work on my laptop but I got it working on my lambda (ensure IAM roles allows iot actions) editing it a little:

// usage:
// addr, err := AwsIotWsUrl(sess, "xxxxx-ats.iot.eu-west-1.amazonaws.com ")

func AwsIotWsUrl(p client.ConfigProvider, endpoint string) (string, error) {
    serviceName := "iotdevicegateway"
    config := p.ClientConfig(serviceName)
    region := *config.Config.Region
    creds, err := config.Config.Credentials.Get()
    if err != nil {
        return "", err
    }
    accessKey := creds.AccessKeyID
    secretKey := creds.SecretAccessKey
    sessionToken := creds.SessionToken

    // according to docs, time must be within 5min of actual time (or at least according to AWS servers)
    now := time.Now().UTC()

    dateLong := now.Format("20060102T150405Z")
    dateShort := dateLong[:8]
    scope := fmt.Sprintf("%s/%s/%s/aws4_request", dateShort, region, serviceName)
    alg := "AWS4-HMAC-SHA256"
    q := [][2]string{
        {"X-Amz-Algorithm", alg},
        {"X-Amz-Credential", accessKey + "/" + scope},
        {"X-Amz-Date", dateLong},
        {"X-Amz-SignedHeaders", "host"},
    }
    query := awsQueryParams(q)

    signKey := awsSignKey(secretKey, dateShort, region, serviceName)
    stringToSign := awsSignString(accessKey, secretKey, query, endpoint, dateLong, alg, scope)
    signature := fmt.Sprintf("%x", awsHmac(signKey, []byte(stringToSign)))

    return fmt.Sprintf("wss://%s/mqtt?%s&X-Amz-Signature=%s&X-Amz-Security-Token=%s", endpoint, query, signature, url.QueryEscape(sessionToken)), nil
}

func awsQueryParams(q [][2]string) string {
    var buff bytes.Buffer
    var i int
    for _, param := range q {
        if i != 0 {
            buff.WriteRune('&')
        }
        i++
        buff.WriteString(param[0])
        buff.WriteRune('=')
        buff.WriteString(url.QueryEscape(param[1]))
    }
    return buff.String()
}

func awsSignString(accessKey string, secretKey string, query string, host string, dateLongStr string, alg string, scopeStr string) string {
    emptyStringHash := "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
    req := strings.Join([]string{
        "GET",
        "/mqtt",
        query,
        "host:" + host,
        "", // separator
        "host",
        emptyStringHash,
    }, "\n")
    return strings.Join([]string{
        alg,
        dateLongStr,
        scopeStr,
        awsSha(req),
    }, "\n")
}

func awsHmac(key []byte, data []byte) []byte {
    h := hmac.New(sha256.New, key)
    h.Write(data)
    return h.Sum(nil)
}

func awsSignKey(secretKey string, dateShort string, region string, serviceName string) []byte {
    h := awsHmac([]byte("AWS4"+secretKey), []byte(dateShort))
    h = awsHmac(h, []byte(region))
    h = awsHmac(h, []byte(serviceName))
    h = awsHmac(h, []byte("aws4_request"))
    return h
}

func awsSha(in string) string {
    h := sha256.New()
    fmt.Fprintf(h, "%s", in)
    return fmt.Sprintf("%x", h.Sum(nil))
}

ffoysal commented 3 years ago

this kind of subscription does not work over webscoket. using goland with paho mqtt library Steps I have done:

create aws iam role

create iot role-alias pointing to the IAM role

{
  "Effect": "Allow",
  "Action": [
    "iot:Subscribe"
  ],
  "Resource": [
    "arn:aws:iot:region:********:topicfilter/$aws/things/${iot:Connection.Thing.ThingName}/jobs/notify-next",
    "arn:aws:iot:region:*****:topicfilter/$aws/things/${iot:Connection.Thing.ThingName}/jobs/start-next/accepted",
    "arn:aws:iot:region:******:topicfilter/$aws/things/${iot:Connection.Thing.ThingName}/jobs/start-next/rejected",
    "arn:aws:iot:region:******:topicfilter/$aws/things/${iot:Connection.Thing.ThingName}/jobs/*/update/accepted",
    "arn:aws:iot:region:*******:topicfilter/$aws/things/${iot:Connection.Thing.ThingName}/jobs/*/update/rejected"
  ]
}

create a policy with role-alias and having

{
  "Effect": "Allow",
  "Action": "iot:AssumeRoleWithCertificate",
  "Resource": "arn:aws:iot:REGION:***********:rolealias/my-thing-role-alias"
}

I can subscribe to the jop topic but dont get any events from it.

If IAM role has something like this

        {
            "Effect": "Allow",
            "Action": [
                "iot:Subscribe"
            ],
            "Resource": [
                "arn:aws:iot:region:*****:topicfilter/$aws/things/${iot:ClientId}/jobs/notify-next",
                "arn:aws:iot:region:*****:topic/$aws/things/${iot:ClientId}/jobs/notify",
                "arn:aws:iot:region:*******:topic/$aws/things/${iot:ClientId}/jobs/get/accepted",
                "arn:aws:iot:region:******:topic/$aws/things/${iot:ClientId}/jobs/*/get/accepted",
                "arn:aws:iot:region:*******:topic/$aws/things/${iot:ClientId}/jobs/get/rejected",
                "arn:aws:iot:region:********:topic/$aws/things/${iot:ClientId}/jobs/*/get/rejected",
                "arn:aws:iot:region:********:topicfilter/$aws/things/${iot:ClientId}/jobs/start-next/accepted",
                "arn:aws:iot:region:********:topicfilter/$aws/things/${iot:ClientId}/jobs/start-next/rejected",
                "arn:aws:iot:region:*********:topicfilter/$aws/things/${iot:ClientId}/jobs/*/update/accepted",
                "arn:aws:iot:region:*********:topicfilter/$aws/things/${iot:ClientId}/jobs/*/update/rejected"
            ]
        }

Then subscribed and get evens from those topics.

vudh1 commented 2 years ago

Hi, our team has discussed and we decide that this feature is not implemented as this is a more cross SDK feature.

github-actions[bot] commented 2 years ago

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.

yeraslan-96 commented 2 years ago

Anyone got this working thought MQTT over Websockets, still struggling with it...

yeraslan-96 commented 2 years ago

As a work around, If anyone is interested, I followed instructions in http://docs.aws.amazon.com/iot/latest/developerguide/protocols.html#http ported code to golang and using paho MQTT over websockets protocol from Golang SDK.

func AwsIotWsUrl(accessKey string, secretKey string, region string, endpoint string) string {
    host := fmt.Sprintf("%s.iot.%s.amazonaws.com", endpoint, region)

    // according to docs, time must be within 5min of actual time (or at least according to AWS servers)
    now := time.Now().UTC()

    dateLong := now.Format("20060102T150405Z")
    dateShort := dateLong[:8]
    serviceName := "iotdevicegateway"
    scope := fmt.Sprintf("%s/%s/%s/aws4_request", dateShort, region, serviceName)
    alg := "AWS4-HMAC-SHA256"
    q := [][2]string{
        {"X-Amz-Algorithm", alg},
        {"X-Amz-Credential", accessKey + "/" + scope},
        {"X-Amz-Date", dateLong},
        {"X-Amz-SignedHeaders", "host"},
    }
    query := awsQueryParams(q)

    signKey := awsSignKey(secretKey, dateShort, region, serviceName)
    stringToSign := awsSignString(accessKey, secretKey, query, host, dateLong, alg, scope)
    signature := fmt.Sprintf("%x", awsHmac(signKey, []byte(stringToSign)))

    return fmt.Sprintf("wss://%s/mqtt?%s&X-Amz-Signature=%s", host, query, signature)
}

func awsQueryParams(q [][2]string) string {
    var buff bytes.Buffer
    var i int
    for _, param := range q {
        if i != 0 {
            buff.WriteRune('&')
        }
        i++
        buff.WriteString(param[0])
        buff.WriteRune('=')
        buff.WriteString(url.QueryEscape(param[1]))
    }
    return buff.String()
}

func awsSignString(accessKey string, secretKey string, query string, host string, dateLongStr string, alg string, scopeStr string) string {
    emptyStringHash := "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
    req := strings.Join([]string{
        "GET",
        "/mqtt",
        query,
        "host:" + host,
        "", // separator
        "host",
        emptyStringHash,
    }, "\n")
    return strings.Join([]string{
        alg,
        dateLongStr,
        scopeStr,
        awsSha(req),
    }, "\n")
}

func awsHmac(key []byte, data []byte) []byte {
    h := hmac.New(sha256.New, key)
    h.Write(data)
    return h.Sum(nil)
}

func awsSignKey(secretKey string, dateShort string, region string, serviceName string) []byte {
    h := awsHmac([]byte("AWS4"+secretKey), []byte(dateShort))
    h = awsHmac(h, []byte(region))
    h = awsHmac(h, []byte(serviceName))
    h = awsHmac(h, []byte("aws4_request"))
    return h
}

func awsSha(in string) string {
    h := sha256.New()
    fmt.Fprintf(h, "%s", in)
    return fmt.Sprintf("%x", h.Sum(nil))
}

With which parameters I can call the AwsIotWsUrl function? I can not call that function

aws / aws-sdk-go

service/iot: Add MQTT over websockets client #820

⚠️COMMENT VISIBILITY WARNING⚠️