search

aws-amplify / amplify-js

A declarative JavaScript library for application development using cloud services.
https://docs.amplify.aws/lib/q/platform/js
Apache License 2.0
9.42k stars 2.12k forks source link

Aws Amplify PubSub Connection issue after latest version update #10829

Closed saharJ95 closed 2 months ago

saharJ95 commented 1 year ago

Before opening, please confirm:

JavaScript Framework

React

Amplify APIs

PubSub

Amplify Categories

No response

Environment information

``` # Put output below this line ```

Describe the bug

  • Package.json "aws-amplify": "^5.0.8", "@aws-amplify/pubsub": "^5.0.8"

After the Package update, unable to make an MQTT connection. No errors were found while adding a plugin. No errors were found while subscribing to topics also with no success. image image

Expected behavior

MQTT connection should be made should receive data from topic

Reproduction steps

  1. Update @aws-amplify libraries
  2. Add plugin
  3. Call subscription method of PubSub.

Code Snippet

// Put your code below this line.

image

Log output

``` // Put your logs below this line ```

aws-exports.js

No response

Manual configuration

No response

Additional configuration

No response

Mobile Device

No response

Mobile Operating System

No response

Mobile Browser

No response

Mobile Browser Version

No response

Additional information and screenshots

No response

nadetastic commented 1 year ago

Hi @saharJ95, thank you for opening this issue. I have been able to consistently reproduce it, and I am discussing with the team.

One note is when I checked for the Hub events, I see ConnectionStateChange where connectionState: Connecting only when the page loads/refreshed, which may suggest that a connection isn't established.

Hub.listen('pubsub', (data) => {
     const { payload } = data;
     console.log(payload)
});

Will follow up once I have additional info, in the meantime - could you let me know what version you are upgrading from?

saharJ95 commented 1 year ago

Okay, Thanks for your update... Currently I am using this version

"@aws-amplify/pubsub": "^4.5.3", "aws-amplify": "^4.3.35",

grahamhar commented 1 year ago

I'm using 5.0.7 and saw the reported issue when updating to to 5.0.8

briankoksal commented 1 year ago

I've tried everything getting the Amplify PubSub working in my project but I think my problem may be related to this issue also. This is what I see with debug logging on:

Screen Shot 2022-12-31 at 7 07 29 PM
HAK-CODE commented 1 year ago

Hi, @grahamhar How are you making IoT pubsub connection using any version of 5.x.x? We are using code as attached in this issue but unable to do so. Thanks

saharJ95 commented 1 year ago

I'm using 5.0.7 and saw the reported issue when updating to to 5.0.8

Hey, @grahamhar I tried version 5.0.0 of amplify and pubsub BUT the issue is the same for me unable to subscribe. Can you please tell me if there are any changes regarding the implementation ?

grahamhar commented 1 year ago

Hi, @grahamhar

How are you making IoT pubsub connection using any version of 5.x.x? We are using code as attached in this issue but unable to do so.

Thanks I can't share the full code as it is in a private repo, I will try to describe what I do. I also made a few code changes so will retry with the latest 5 version. I have cognito auth configured with a role allowing connect to iot for both the auth and unauthorised IAM roles. Then in the component that has the subscription in the component did mount I setup the pubsub.subscribe inside the then of a auth.current credentials call.

If I recall correctly you may need to assign the thing policy to the cognito identity pool ID for each user. My unauth users just work.

If I get time I will try to extract the code snippet into a gist

HAK-CODE commented 1 year ago

Thanks, We followed same steps defined here PubSub and is working before version 5 but not for any version of 5 or above. As you mentioned its working for you for version 5.0.7

nadetastic commented 1 year ago

Hi @saharJ95 - after doing some additional testing on this, I am not able to reproduce this on a project setup with npx create-react-app but consistently seeing the issue on a sample project setup with vite. Any additional information regarding how your project is setup would be helpful here.

cc. @grahamhar @briankoksal @HAK-CODE

HAK-CODE commented 1 year ago

We are using npx

stocaaro commented 1 year ago

Hello @saharJ95,

I've been working with @nadetastic to try to figure out why this setup isn't able to form a connection to IoT. They had a sample app that reproduces what you are seeing, where my attempt to reproduce ended with an app that is able to connect. The difference between our apps is how we resolved the issues where global is missing when using Vite.

I had followed the instructions from the Amplify UI troubleshooting steps, and my app overwrote global in index.html.

The other workaround recommend configuring global: {}, in vite.config.js.

When global is setup in the second way in vite.config.js, I see the error, but when I remove this and configure it in the index.html way recommended in the Amplify UI steps, then the subscription connects as expected.

Can you test this to see if it works in your application?

Thanks, Aaron

saharJ95 commented 1 year ago

Hi @stocaaro, I am using npx create-react-app
It works fine below 5.x.x. We are not using Vite, so the above solution is not helpful for me.

HAK-CODE commented 1 year ago

@stocaaro for any version of 5.x.x its not even giving any error output, what causing this issue.

stocaaro commented 1 year ago

Sorry for the misunderstanding. I'm not sure how I got off on that tangent with Vite.

Re-reading the above, is this a new application that you are trying to get working for the first time or an existing app where it stops working when upgrading to 5.x.x?

Above, @nadetastic provided details on watching the connection state by subscribe to Hub updates. It looks like @briankoksal had done this and turned on error logging. The error appears to be the type of thing I would expect when I miss a step in the IAM/IoT Policy setup steps. Is this consistent with the errors you are seeing @saharJ95 and @HAK-CODE ?

Working through the PubSub Getting Started guide, I am able to get a CRA based sample app working. Are there any steps or details in you setup that might be different in a way that would help me reproduce the issue you're seeing?

HAK-CODE commented 1 year ago

Hi @stocaaro, Thanks for your response as we migrated from 4.5.3 to 5.0.8 (also tested for 5.0.7) so steps remains same and code snippet is attached @saharJ95

  1. Configuration as code attached above
  2. Add Iot Policy as mentioned here Iot Policy
  3. Subscribe to topic(s) Subscribe So in short we haven’t changed any code assuming interface remains same, one thing weird is we are not getting errors in console as not even a network call that shows a mqqt attempt made.

If you could share your code snippet, we can reverify if we are missing and new or modified code. thanks

nadetastic commented 1 year ago

@HAK-CODE here's a sample project using CRA that works as expected for me [1].

Can you also verify the following:

[1] https://github.com/nadetastic/cra-amplified [2] https://docs.amplify.aws/lib/pubsub/getting-started/q/platform/js/#step-3-allow-the-amazon-cognito-authenticated-role-to-access-iot-services

HAK-CODE commented 1 year ago

Hi @nadetastic , Just a dummy question where should i add user identity id, or security token to make access to AWS Iot core as for now i am unable to make it work. Thanks

briankoksal commented 1 year ago

Sorry for the misunderstanding. I'm not sure how I got off on that tangent with Vite.

Re-reading the above, is this a new application that you are trying to get working for the first time or an existing app where it stops working when upgrading to 5.x.x?

Above, @nadetastic provided details on watching the connection state by subscribe to Hub updates. It looks like @briankoksal had done this and turned on error logging. The error appears to be the type of thing I would expect when I miss a step in the IAM/IoT Policy setup steps. Is this consistent with the errors you are seeing @saharJ95 and @HAK-CODE ?

Working through the PubSub Getting Started guide, I am able to get a CRA based sample app working. Are there any steps or details in you setup that might be different in a way that would help me reproduce the issue you're seeing?

Just to follow up on my issue. I recreated my Amplify react project with 5.0.8 and followed the PubSub example and was able to subscribe to the MQTT events successfully. My issue was due to permissions like you said. I did not follow step 3 of the pubsub guide correctly the first time around. Thanks for your help.

nadetastic commented 1 year ago

@HAK-CODE there are two things you need to do in order to have permissions:

First, you would need to create an IoT policy through the IoT core console as described at [1], then attach this newly created policy to the Cognito Identity that your app uses. The Amplify documentation shows how you can retrieve the Identity ID, then use the AWS CLI to attach it the created policy to the Cognito Identity [2]

Then, for the AuthRoles and UnAuthRoles created by Amplify, you need to give them IoT permissions as described here [3].

[1] https://docs.amplify.aws/lib/pubsub/getting-started/q/platform/js/#step-1-create-iam-policies-for-aws-iot [2] https://docs.amplify.aws/lib/pubsub/getting-started/q/platform/js/#step-2-attach-your-policy-to-your-amazon-cognito-identity [3] https://docs.amplify.aws/lib/pubsub/getting-started/q/platform/js/#step-3-allow-the-amazon-cognito-authenticated-role-to-access-iot-services

Let me know if you would like for me to further clarify.

nadetastic commented 1 year ago

@briankoksal glad you were able to get unblocked!

saharJ95 commented 1 year ago

Hi @nadetastic, I tried the sample project with my configs and it works fine but It not working in my project with amplify 5.X.x. I tried to copy your codes but unfortunately, results are the same for me

image This is my index.js file

image File where I am using PubSub

After debugging... I think there is an issue in adding AWSIoTProvider pluggable... As you can see PubSub.Subscribe(..) is not giving any error/success while publishing the message I am getting Could not find provider named AWSIoTProvider error

image

or pubSub object is null in my case after adding AWSIoTProvider image

I am unable to understand the issue because it works completely fine with "@aws-amplify/pubsub": "^4.5.3", "aws-amplify": "^4.3.35",

nadetastic commented 1 year ago

@saharJ95 could you try removing your node_modules and re-installing with the new version of aws-amplify?

I've come across a few cases recently with issues when upgrading, and doing this was the fix

saharJ95 commented 1 year ago

I tried deleting modules and re-installing nothing changed.

HAK-CODE commented 1 year ago

I am having same issue after upgrade node version = v18.13.0

package.json

{
  "name": "pragmatic-dashboard",
  "version": "6.4.0",
  "private": true,
  "dependencies": {
    "@aws-amplify/pubsub": "^5.0.8",
    "@casl/ability": "^5.4.3",
    "@casl/react": "2.3.0",
    "@elastic/charts": "^46.11.0",
    "@elastic/datemath": "^5.0.3",
    "@elastic/eui": "^57.0.0",
    "@emotion/react": "^11.7.1",
    "@fullcalendar/core": "5.10.1",
    "@fullcalendar/daygrid": "5.10.1",
    "@fullcalendar/interaction": "5.10.1",
    "@fullcalendar/list": "5.10.1",
    "@fullcalendar/react": "5.10.1",
    "@fullcalendar/timegrid": "5.10.1",
    "@fullcalendar/timeline": "5.10.1",
    "@hookform/resolvers": "2.8.5",
    "@reduxjs/toolkit": "^1.8.1",
    "@rjsf/core": "^4.2.0",
    "@types/react-transition-group": "^4.4.4",
    "@uppy/react": "2.1.2",
    "animate.css": "4.1.1",
    "antd": "^4.17.3",
    "apexcharts": "^3.36.3",
    "aws-amplify": "^5.0.8",
    "babel-plugin-formatjs": "^10.3.20",
    "bootstrap": "^5.1.3",
    "bs-stepper": "1.7.0",
    "chart.js": "^3.7.1",
    "chroma-js": "^2.1.2",
    "classnames": "^2.3.1",
    "cleave.js": "1.6.0",
    "draft-js": "0.11.7",
    "draftjs-to-html": "0.9.1",
    "file-saver": "2.0.5",
    "flat": "^5.0.2",
    "flatpickr": "^4.6.13",
    "history": "^5.3.0",
    "html-to-draftjs": "1.5.0",
    "intl-messageformat": "9.10.0",
    "jquery": "3.6.0",
    "jsonwebtoken": "8.5.1",
    "leaflet": "1.7.1",
    "lodash": "^4.17.21",
    "mathjs": "^11.5.0",
    "moment": "^2.29.3",
    "nouislider": "^15.5.0",
    "nouislider-react": "3.4.1",
    "object-hash": "^2.2.0",
    "patch-package": "^6.5.0",
    "postcss-rtl": "1.7.3",
    "prismjs": "1.25.0",
    "prop-types": "^15.7.2",
    "react": "^18.2.0",
    "react-apexcharts": "^1.4.0",
    "react-chartjs-2": "^4.0.0",
    "react-contexify": "5.0.0",
    "react-copy-to-clipboard": "5.0.4",
    "react-country-flag": "3.0.2",
    "react-custom-scroll": "^4.4.0",
    "react-data-table-component": "7.4.5",
    "react-dom": "18.2.0",
    "react-draft-wysiwyg": "1.14.7",
    "react-dropdown-tree-select": "^2.7.1",
    "react-feather": "~2.0.9",
    "react-flatpickr": "3.10.7",
    "react-hook-form": "7.22.1",
    "react-intl": "5.23.0",
    "react-json-view": "^1.21.3",
    "react-jsonschema-form": "^1.8.1",
    "react-leaflet": "3.2.2",
    "react-paginate": "8.1.0",
    "react-perfect-scrollbar": "1.5.8",
    "react-player": "2.9.0",
    "react-rating": "2.0.5",
    "react-redux": "7.2.6",
    "react-router-dom": "^6.3.0",
    "react-scripts": "^5.0.1",
    "react-scroll-up": "1.3.7",
    "react-select": "5.2.1",
    "react-shepherd": "3.3.6",
    "react-slidedown": "2.4.6",
    "react-sortablejs": "^6.0.0",
    "react-toastify": "^8.1.0",
    "react-transition-group": "^4.4.2",
    "react-world-flags": "^1.5.1",
    "reactstrap": "9.0.1",
    "recharts": "2.1.8",
    "redux": "4.1.2",
    "redux-debounced": "0.5.0",
    "redux-thunk": "2.4.1",
    "reselect": "^4.1.5",
    "rxjs": "^7.4.0",
    "sass": "^1.45.0",
    "screenfull": "6.0.0",
    "sortablejs": "^1.15.0",
    "styled-components": "5.3.3",
    "sweetalert2": "11.3.0",
    "sweetalert2-react-content": "4.2.0",
    "swiper": "7.3.2",
    "uppy": "2.3.1",
    "webpack": "5.65.0",
    "wnumb": "1.2.0",
    "xlsx": "0.17.4",
    "yarn": "1.22.17",
    "yup": "0.32.11"
  },
  "scripts": {
    "postinstall": "patch-package",
    "start": "react-scripts start",
    "build": "react-app-rewired build",
    "test": "react-app-rewired test",
    "eject": "react-app-rewired eject",
    "lint": "eslint src/**/*.js",
    "lint:fix": "eslint src/**/*.js --fix",
    "lint-staged": "lint-staged"
  },
  "eslintConfig": {
    "extends": "react-app"
  },
  "devDependencies": {
    "@types/sortablejs": "^1.10.7",
    "crypto-browserify": "^3.12.0",
    "eslint": "^8.4.1",
    "eslint-plugin-import": "^2.25.3",
    "eslint-plugin-react": "^7.27.1",
    "lint-staged": "^12.1.2",
    "postcss": "^7.0.39",
    "pre-commit": "^1.2.2",
    "process": "^0.11.10",
    "react-app-alias": "^2.2.2",
    "react-app-rewire-postcss": "^3.0.2",
    "react-app-rewired": "^2.2.1",
    "stream-browserify": "^3.0.0",
    "util": "^0.12.4"
  },
  "browserslist": [
    ">0.2%",
    "not dead",
    "not op_mini all"
  ],
  "homepage": "",
  "lint-staged": {
    "src/**/*.js": "eslint"
  },
  "pre-commit": "lint-staged"
}

index.js

import ReactDOM from 'react-dom/client';
import { Auth, Amplify } from 'aws-amplify';
import { AWSIoTProvider } from '@aws-amplify/pubsub/lib/Providers';

Amplify.configure({
  aws_project_region: 'us-east-2',
  aws_cognito_identity_pool_id:
    'us-east-2:dummyuuid',
  aws_cognito_region: 'us-east-2',
  aws_user_pools_id: 'us-east-2_dummy',
  aws_user_pools_web_client_id: 'dummy'
});

Amplify.addPluggable(
  new AWSIoTProvider({
    aws_pubsub_region: 'us-east-2',
    aws_pubsub_endpoint:
      'wss://dummyendpoint-ats.iot.us-east-2.amazonaws.com/mqtt'
  })
);

console.log(Amplify);
console.log(Amplify.PubSub);

async function signIn () {
  try {
    const user = await Auth.signIn('{some@gmail.com}', 'somedummypassword');
    console.log(user);
  } catch (error) {
    console.log('error signing in', error);
  }
}
const root = ReactDOM.createRoot(document.getElementById('root'));

root.render(
  'hello'
)
signIn()
Screenshot 2023-01-13 at 7 15 11 PM

Tried different combination but it seems AWS Iot provider is not adding as pluggable.

stocaaro commented 1 year ago

@saharJ95 / @HAK-CODE ,

In other cases where PubSub hasn't existed to receive the addPluggable call configuring IoT, I've found that this is caused by tree shaking by the build.

As an experiment, can you import PubSub and do something like console.log(PubSub) before calling addPluggable?

This isn't a pattern I would recommend, but it testing like this would tell us if the build system is doing something extra smart that is causing the behavior you're observing.

Thanks, Aaron

saharJ95 commented 1 year ago

Hey @nadetastic , @stocaaro , Thank you for your help. The Plugin AWS-SDK was conflicting with our changes. After deleting that package PubSub works fine. Thanks,

nadetastic commented 1 year ago

Hi @saharJ95 I am glad you were able to resolve this and thank you for sharing your findings.

I will re-open this issue so that I may follow up with the other folks facing this issue (@HAK-CODE )

zacharyrogers1 commented 1 year ago

Sorry for the misunderstanding. I'm not sure how I got off on that tangent with Vite.

Re-reading the above, is this a new application that you are trying to get working for the first time or an existing app where it stops working when upgrading to 5.x.x?

Above, @nadetastic provided details on watching the connection state by subscribe to Hub updates. It looks like @briankoksal had done this and turned on error logging. The error appears to be the type of thing I would expect when I miss a step in the IAM/IoT Policy setup steps. Is this consistent with the errors you are seeing @saharJ95 and @HAK-CODE ?

Working through the PubSub Getting Started guide, I am able to get a CRA based sample app working. Are there any steps or details in you setup that might be different in a way that would help me reproduce the issue you're seeing?

This was my exact problem in my reactJS app + vite. Once I updated my index.html + vite.config as suggested I was able to connect, publish, and subscribe. Thank you.

stocaaro commented 1 year ago

Anyone still facing this issue and need help troubleshooting?

alevaylin commented 1 year ago

Hello @saharJ95,

I've been working with @nadetastic to try to figure out why this setup isn't able to form a connection to IoT. They had a sample app that reproduces what you are seeing, where my attempt to reproduce ended with an app that is able to connect. The difference between our apps is how we resolved the issues where global is missing when using Vite.

I had followed the instructions from the Amplify UI troubleshooting steps, and my app overwrote global in index.html.

The other workaround recommend configuring global: {}, in vite.config.js.

When global is setup in the second way in vite.config.js, I see the error, but when I remove this and configure it in the index.html way recommended in the Amplify UI steps, then the subscription connects as expected.

Can you test this to see if it works in your application?

Thanks, Aaron

Hi @stocaaro, I have the same issue, When I try first configuration, I was able to connect, but after a few seconds connection interrupts and gives errorcode: 8. I'm using vite and amplify 5.0.11. Do you have any advice?

stocaaro commented 1 year ago

Hello @alevaylin,

Error code 8 means the remote MQTT server sent you an on_close message ending the connection. Usually means that something isn't correctly configured with the permissions. Would you follow the guidance from this post above to try to isolate down why the server is disconnecting?

If you are able to get any messages sent or received before it is closed, then it's possibly different from the permissions configuration issues we often see. In this case, please document what you're seeing.

Thanks, Aaron

imflamboyant commented 1 year ago

Hey guys, some of these comments and examples were super useful to me to get something working in my Angular 15 app. I downgraded Amplify dependencies to version 4 and now I can publish and receive messages using the MqttOverWSProvider. NOTE: I am using custom Lambda authorizer for IoT Core and not Cognito Identity. My first question would be: does AWSIoTProvider work with custom authorizer? By observing debug logs when listening to Hub events I noticed that it always tries to connect using the Cognito Identity (and fails in my case as I don't use one).

Additional note: If I switch to Amplify version 5 for dependencies, I get nothing working. By observing the Hub event logs, I concluded that the mqtt client is not instantiated.

Any help would be appreciated.

stocaaro commented 1 year ago

Hello @imflamboyant ,

Glad the above was useful. I don't have any experience getting IoT Core connections working with non-cognito authorizers. I would imagine the IAM permissions translate pretty directly, but I'm not sure what would be put in the IoT policy to allow connections.

Regarding behavior that was working in V4 that is no longer working in V5, would you be able to create a new issue and document your application specs in detail along with the input and output that you're observing?

Thanks!

torablien commented 1 year ago

Hey folks, I think I may be running into the same issue that is being discussed here. I am able to successfully publish and subscribe from MQTT test client within the IoT Core console but am getting nondescript errors on a simple example when using Amplify's JS pubsub.

Specifically, on subscribe, I am seeing

[DEBUG] 36:20.70 MqttOverWSProvider - Creating new MQTT client 9644aeff-7dca-430a-b163-b9d07a227b85
[DEBUG] 36:20.189 MqttOverWSProvider - Error forming connection Error: AMQJS0011E Invalid state not connected.
    at ClientImpl.subscribe (paho-mqtt.js:1016:1)
    at Client.subscribe (paho-mqtt.js:2482:1)
    at MqttOverWSProvider.ts:245:1
    at Map.forEach (<anonymous>)
    at AWSIoTProvider.<anonymous> (MqttOverWSProvider.ts:240:1)
    at step (tslib.es6.js:100:1)
    at Object.next (tslib.es6.js:81:1)
    at fulfilled (tslib.es6.js:71:1)

and on publishing, I get [DEBUG] 36:25.28 MqttOverWSProvider - Publishing to topic(s) failed with no additional detail

Setup I started with a totally fresh project/app and am using these are references:

I used npx create-react-app (not vite).

I am on 

    "aws-amplify": "^5.0.22",
    "@aws-amplify/pubsub": "^5.1.5",
    "@aws-amplify/ui-react": "^4.4.1",

I created a new Amplify app, created a single user, and added AWSIoTFullAccess to all the roles (e.g. Auth, Unauth) associated with the Amplify app from the IAM console.

In IoT Core, I created an "allow_all" policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "*",
      "Resource": "arn:aws:iot:us-west-1:<MY-ACCOUNT-ID>:*"
    }
  ]
}

and I attached the policy to my Cognito user aws iot attach-policy --policy-name 'allow_all' --target '<MY_COGNITO_IDENTITY_ID>

I double checked that I used the right Cognito ID by logging it to console

  Auth.currentCredentials().then((info) => {
  console.log("Current identity", info.identityId);
  });

For reference, here is the full code:

import { Amplify, PubSub, Auth } from "aws-amplify";
import { withAuthenticator } from '@aws-amplify/ui-react';
import React from 'react';
import awsconfig from './aws-exports';
// import '@aws-amplify/ui/dist/style.css';
import { AWSIoTProvider } from '@aws-amplify/pubsub';
// import { AWSIoTProvider } from '@aws-amplify/pubsub/lib/Providers';
import 'bootstrap/dist/css/bootstrap.min.css';
import logo from './logo.svg';

Amplify.Logger.LOG_LEVEL = "VERBOSE";
Amplify.configure(awsconfig);

var SUB_TOPIC = "neil_m5stack_dev/pub";
var PUB_TOPIC = "neil_m5stack_dev/sub";

// Apply plugin with configuration
Amplify.addPluggable(
  new AWSIoTProvider({
    aws_pubsub_region: "us-west-1",
    aws_pubsub_endpoint:
      "wss://<MY-ENDPOINT>.iot.us-west-1.amazonaws.com/mqtt",

  })
);

async function ProcessMessage(payload) {
  console.log('Message received', payload);
  let topic=payload.value[Object.getOwnPropertySymbols(payload.value)[0]];
  let time=payload.value.time;
  let sensor_a0=payload.value.sensor_a0;
  let scrollBox = document.getElementById('incomingMsg');
  scrollBox.innerHTML += "<b>NEW MESSAGE: </b><br></br> Topic: " + topic + "<br></br> Time: " + time + "<br></br> Sensor_a0: " + sensor_a0 + "<br></br>";
  scrollBox.scrollTop = scrollBox.scrollHeight;
}

async function SendMessage() {
  let payload=document.getElementById('msg').value;
  document.getElementById('msg').value='';
  console.log("Publish to ", PUB_TOPIC, payload);
  try {
  await PubSub.publish(PUB_TOPIC, { msg: payload });
  document.getElementById('returnMsg').innerHTML = '<h3 style="color: green">Message sent!</h3>';
  let sentMsgBox = document.getElementById('sentMsg');
  sentMsgBox.innerHTML += payload + "<br></br>";
  sentMsgBox.scrollTop = sentMsgBox.scrollHeight;
  } catch (error) {
    console.log("Error publishing ", PUB_TOPIC, payload, error);
  }
}

function App() {
  Auth.currentCredentials().then((info) => {
  console.log("Current identity", info.identityId);
  });
  subscribe();
  return (
    <div className="App">
      <div className="mt-5 row" style={{"backgroundColor": "black", "alignItems": "center", "justifyContent": "center"}}>
        <img src={logo} style={{"height": "5vmin"}} alt="logo"></img>
        <h1 style={{"color": "white"}}>IoT Messaging</h1>
        <a href="https://aws.amazon.com/what-is-cloud-computing" className="pl-4"><img src="https://d0.awsstatic.com/logos/powered-by-aws-white.png" alt="Powered by AWS Cloud Computing"></img></a>
      </div>
      <div className="row">
        <div id="publisher" className="col ml-5 mt-5 mb-5 mr-3" style={{"borderStyle": "solid", "borderWidth": "2px"}}>
          <h2>Publisher</h2>
          <p>The box below can be used to publish messages back to your devices by publishing to the topic <b>{PUB_TOPIC}</b></p>
          <h5>Message: </h5>
          <input type="text" className="form-control" id="msg" name="msg" placeholder="Enter Message Here"></input>
          <br></br>
          <button className="btn btn-primary" onClick={SendMessage}>Send Message</button>
          <div id='returnMsg'></div>
          <br></br>
          <h3>Sent Messages:</h3>
          <p>Your sent messages will appear here</p>
          <div id='sentMsg' className="overflow-auto mb-5 border" syle={{"maxHeight": "220px"}}></div>
        </div>
        <br></br><br></br>
        <div id="subscriber" className="col mt-5 mr-5 mb-5" style={{"borderStyle": "solid", "borderWidth": "2px"}}>
          <h2>Subscriber</h2>
          <p>New messages from your device(s) that publish to the topic <b>{SUB_TOPIC}</b> will appear in the box below</p>
          <div id="incomingMsg" className="overflow-auto border" style={{"maxHeight": "200px"}}></div>
        </div>
      </div>
    </div>
  );
}

function subscribe() {
  PubSub.subscribe(SUB_TOPIC).subscribe({
    next: data => ProcessMessage(data),
    error: error => console.error(error),
    close: () => console.log('Done'),
  });
}

export default withAuthenticator(App, true);

Would any of you have any ideas or suggestions on how to proceed? Thanks!

stocaaro commented 1 year ago

I've followed the steps you described on a fresh app and used the code sample you provided, which resulted in a working application (with a bit of missing css).

You've double checked all the account permission settings I would normally suggest you double check. I might triple check that the identity id is on the policy in the IoT Core > Policy "name" > Targets list. You shouldn't need to grant IoT access to the UnAuth role, but there's nothing about that which would cause this to not work.

I was hoping that stepping through this with your example code would turn something up, but since it worked in my example case, I'm not sure why its not working for you.

image
stocaaro commented 1 year ago

I will note that I had to grant my admin IAM user access to AWSIoTFullAccess before I could run aws iot attach-policy --policy-name .... Without this grant, I was getting

User: arn:aws:iam::XXXXXXX:user/amplify-dev is not authorized to perform: iot:AttachPolicy on resource: us-west-1:YYYYYY because no identity-based policy allows the iot:AttachPolicy action

Not sure if you saw an error when attaching the iot policy, but if that didn't work, it could explain the behavior.

torablien commented 1 year ago

Thank you @stocaaro for trying. Just to confirm, you then do see that "Test" message coming to that topic (e.g. if you subscribe to that topic in the MQTT test client in IoT Core console)? I ask because that sample app will say on the page "Message sent!" regardless of whether it actually did (incl. if console errors) so I wanted to double check.

As you suggested, I went to IoT Core > Policies > "allow_all" > Targets and confirmed that I see the one entry there which is my Cognito identity. I also confirmed that I did add AWSIoTFullAccess to my IAM user before doing any of this and can also still see that role attached to my IAM user right now.

Any chance you have any suggestions on something else I could try or some way I could get some more logs on why things are failing? I'm not exactly sure what the error I get on subscribing means (AMQJS0011E Invalid state not connected.) but perhaps that could indicate the underlying issue? Thanks!

stocaaro commented 1 year ago

@torablien,

Correct. When I subscribe to the topic using using the IoT test client and send a message using the page I created with your code, then I see it in the test client. (See screen cap below). I also confirmed the websocket is open and active and I can see the content sent as I was expecting.

In terms of other things to check, I don't have any clear suggestions. Your comment says you have checked everything about the setup process and confirmed that it is correctly configured. Since its not working for you, but it works on my end, then something must be misconfigured. Short of double checking everything, I'm not sure what to recommend.

It might be worth running through the pubsub guide on a new app, following only the getting started guide and getting an example app working that you can compare with this apps config for inconsistencies.

image
torablien commented 1 year ago

OK, I started completely fresh again and followed only this getting started guide with only a publisher. Unfortunately it still does not seem to work. I wrote down every single step I did in case I somehow still missed something.

  1. Create fresh new directory
  2. npx create-react-app pubsubtest and cd pubsubtest
  3. amplify init (all defaults)
  4. npm install aws-amplify
  5. npm install @aws-amplify/ui-react aws-amplify
  6. amplify add auth (all defaults)
  7. amplify push
  8. Add Amplify and withAuthenticator to App.js: 
    
import { Amplify, Auth, PubSub } from 'aws-amplify';
import { AWSIoTProvider } from '@aws-amplify/pubsub';
import awsExports from './aws-exports';
import { withAuthenticator } from '@aws-amplify/ui-react';
Amplify.configure(awsExports);

// Apply plugin with configuration Amplify.addPluggable( new AWSIoTProvider({ aws_pubsub_region: 'us-west-1', aws_pubsub_endpoint: 'wss:/.iot.us-west-1.amazonaws.com/mqtt' }) ); . . . export default withAuthenticator(App, true);

9. `npm start`
10. Create user
11. Verify email
12. Create a test policy named "testIoTPolicy" in IoT Core with allow all permissions.
```json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "*",
      "Resource": "arn:aws:iot:us-west-1:<my-aws-account-id-with-no-hyphens>:*"
    }
  ]
}
  1. Log current identity at the top of function App() in App.js. 
    Auth.currentCredentials().then((info) => {
console.log("Current identity", info.identityId);
});
  2. npm start
  3. Login with created and verified user
  4. Attach the policy to my Cognito ID aws iot attach-policy --policy-name 'testIoTPolicy' --target '<the-value-of-info.identityId-just-logged>' (no errors)
  5. Go to CloudFormation in AWS Console
  6. Select the newly created root Stack (not marked as "Nested")
  7. Select the Resources tab
  8. Click on the Physical ID link in the AuthRole row.
  9. Click Add Permissions -> Attach Policies
  10. Search for AWSIoT
  11. Check mark AWSIoTDataAccess and AWSIoTConfigAccess
  12. Click Add Permissions
  13. In App.js, add the ability to publishes to "testTopic" 
    
var PUB_TOPIC = "testTopic/pub";

async function SendMessage() { let payload=document.getElementById('msg').value; document.getElementById('msg').value=''; console.log("Publishing to ", PUB_TOPIC, payload); await PubSub.publish(PUB_TOPIC, { msg: payload }); let sentMsgBox = document.getElementById('sentMsg'); sentMsgBox.innerHTML += payload + "

"; sentMsgBox.scrollTop = sentMsgBox.scrollHeight; }

26. Go to the MQTT test client in the IoT Core console
27. Subscribe to topic testTopic/pub
28. `npm start`
29. Publish a message (from localhost:3000)

Nothing shows up in the test client. 
<img width="1732" alt="Screen Shot 2023-03-27 at 5 29 46 PM" src="https://user-images.githubusercontent.com/11454665/228096893-a59d3d24-6375-419b-be4e-55a656122c10.png">

Full App.js code:
```js
import './App.css';
import { Amplify, Auth, PubSub } from 'aws-amplify';
import { AWSIoTProvider } from '@aws-amplify/pubsub';
import awsExports from './aws-exports';
import { withAuthenticator } from '@aws-amplify/ui-react';
Amplify.configure(awsExports);

// Apply plugin with configuration
Amplify.addPluggable(
  new AWSIoTProvider({
    aws_pubsub_region: 'us-west-1',
    aws_pubsub_endpoint:
      'wss://<myurlfrompage-ats.iot.us-west-1.amazonaws.com/mqtt'
  })
);

var PUB_TOPIC = "testTopic/pub";

async function SendMessage() {
  let payload=document.getElementById('msg').value;
  document.getElementById('msg').value='';
  console.log("Publishing to ", PUB_TOPIC, payload);
  await PubSub.publish(PUB_TOPIC, { msg: payload });
  let sentMsgBox = document.getElementById('sentMsg');
  sentMsgBox.innerHTML += payload + "<br></br>";
  sentMsgBox.scrollTop = sentMsgBox.scrollHeight;
}

function App() {
  Auth.currentCredentials().then((info) => {
    console.log("Current identity", info.identityId);
    });
  return (
    <div className="App">
      <div id="publisher" className="col ml-5 mt-5 mb-5 mr-3" style={{"borderStyle": "solid", "borderWidth": "2px"}}>
          <h2>Publisher</h2>
          <h5>Message to <b>{PUB_TOPIC}</b>: </h5>
          <input type="text" className="form-control" id="msg" name="msg" placeholder="Enter Message Here"></input>
          <br></br>
          <button className="btn btn-primary" onClick={SendMessage}>Send Message</button>
          <div id='returnMsg'></div>
          <br></br>
          <h3>Sent Messages:</h3>
          <p>Your sent messages will appear here</p>
          <div id='sentMsg' className="overflow-auto mb-5 border" syle={{"maxHeight": "220px"}}></div>
        </div>
    </div>
  );
}

export default withAuthenticator(App, true);

Any other ideas? I've confirmed that I am able to both publish and subscribe to these topics successfully from my embedded IoT device and see that reflected in the IoT Core console's MQTT Test Client so it would seem that this issue is something specific to Amplify.

torablien commented 1 year ago

FWIW, I sent a friend that uses AWS IoT in production that getting started guide and asked him to try it to see if it might be something specific to just my setup. He set it up fresh on a personal account and ran into the exact same issue.

nadetastic commented 1 year ago

@torablien following up here, I took the exact steps that you outlined above and i was able to successfully get this working using the same front end code as you have.

Could do the following?

torablien commented 1 year ago

Thanks @nadetastic for following up! Very weird that it works for you both but fails for me and my friend.

Yes, I did the following and I only receive the same nondescript error: [DEBUG] 15:51.543 MqttOverWSProvider - Publishing to topic(s) failed (2) ['testTopic/pub', '{"msg":"Hi Dan!"}'

image

The auth role itself only has the two AWSIoT policies. I don't see any Deny's (unless I am not looking in the right place):

image

The IoT policy created indeed has only one target and it seems to be the valid Cognito identity

image

and it matches what I see under the Federated Identities in the Identity Pool that was created for the app when I created the user

image

I'm happy to check anything else. If it helps, you're also more than welcome to check any internal logging you may have associated with my account as I don't have anything particularly sensitive associated with this AWS account.

nadetastic commented 1 year ago

Hi @torablien following up here - I think I may have reproduced this issue.

Question for you, do you have your pubsub config and Pubsub calls in seperate components/files?

For example, I have defined the following in my index.js:

// index.js
...
Amplify.configure(awsExports);

Amplify.addPluggable(
  new AWSIoTProvider({
    aws_pubsub_region: 'us-west-2',
    aws_pubsub_endpoint:
       'wss://<myurlfrompage-ats.iot.us-west-2.amazonaws.com/mqtt'
  })
);
...

Then my sub and publish in App.js:

// App.js
...
async function SendMessage() {
  await PubSub.publish(PUB_TOPIC, { message: payload });
}

PubSub.subscribe(PUB_TOPIC).subscribe({
    next: data => {
      console.log('Message received', data);
    },
    error: error => console.error(error),
    close: () => console.log('Done'),
  });
...

With this setup, I get the same error [DEBUG] 15:51.543 MqttOverWSProvider - Publishing to topic(s) failed (2) ['testTopic/dev', '{"msg":"Testing"}' and messages fail to subscribe or publish.

If this is the case, can you try by having both within the same file, ex App.js and confirm if it works?

This was brought to my attention based on the comment on https://github.com/aws-amplify/docs/issues/7554 however the proposed fixes do not work for me.

Thanks!

torablien commented 1 year ago

Unfortunately not, I had created a fresh project and tried to keep it as basic as possible so they were all in the same file. I haven't tried it using Amplify since then but I can retry it again if it helps. It'd be great if verbose logging gave more context on these failures.

free-lunch commented 1 year ago

I went through the same thing, The reason is unknown, but I went through a phenomenon of adding and operating a subscription. It didn't matter what the subject matter.

  PubSub.subscribe('unuseless/topic', { provider: 'AWSIoTProvider' }).subscribe({
    next: (data) => console.log('Message received', data),
    error: (error) => console.error(error),
    complete: () => console.log('Done'),
  });

I also recommend adding the code below.

Amplify.configure(awsExports);
// https://github.com/aws-amplify/docs/issues/7554
Amplify.register(PubSub);
Julz-afk commented 1 year ago

Thanks @nadetastic for following up! Very weird that it works for you both but fails for me and my friend.

Yes, I did the following and I only receive the same nondescript error: [DEBUG] 15:51.543 MqttOverWSProvider - Publishing to topic(s) failed (2) ['testTopic/pub', '{"msg":"Hi Dan!"}'

image

The auth role itself only has the two AWSIoT policies. I don't see any Deny's (unless I am not looking in the right place): image

The IoT policy created indeed has only one target and it seems to be the valid Cognito identity image and it matches what I see under the Federated Identities in the Identity Pool that was created for the app when I created the user image

I'm happy to check anything else. If it helps, you're also more than welcome to check any internal logging you may have associated with my account as I don't have anything particularly sensitive associated with this AWS account.

Did you get this issue resolved at your side? I am experiencing the same problem.

torablien commented 1 year ago

Thanks @nadetastic for following up! Very weird that it works for you both but fails for me and my friend. Yes, I did the following and I only receive the same nondescript error: [DEBUG] 15:51.543 MqttOverWSProvider - Publishing to topic(s) failed (2) ['testTopic/pub', '{"msg":"Hi Dan!"}'

image

The auth role itself only has the two AWSIoT policies. I don't see any Deny's (unless I am not looking in the right place): image The IoT policy created indeed has only one target and it seems to be the valid Cognito identity image and it matches what I see under the Federated Identities in the Identity Pool that was created for the app when I created the user image I'm happy to check anything else. If it helps, you're also more than welcome to check any internal logging you may have associated with my account as I don't have anything particularly sensitive associated with this AWS account.

Did you get this issue resolved at your side? I am experiencing the same problem.

No sorry, I ended up moving off of Amplify so I have not tried it since.

Julz-afk commented 1 year ago

@torablien Got it up and running on my side, think I missed a step somewhere with the policy and identity stuff.

nadetastic commented 1 year ago

@Julz-afk glad to hear you were able to get it working! Curios to know what difference in steps you may have taken before vs now when you got it working?

Flgado commented 1 year ago

Hi there! I have the same problem of @torablien. I dont see any problem with the configuration but i get the "[DEBUG] 15:51.543 MqttOverWSProvider - Publishing to topic(s) failed (2) ['testTopic/pub'" Can someone give any help? @Julz-afk can you tell me what was wrong ? thanks :)