EC Server not connecting to Gateway running on predix with 2737 v1.1beta version

simran3695 commented 4 years ago

We have recently deployed the EC Gateway on Predix with 2737 v1.1beta version and on CF3 Service. The gateway is up and running but while attempting to run server with same 2737 version and connecting to predix gateway, the connectivity is not working. Below are the server logs:

Also not sure why the gateway url protocol is https instead of wss

2-dev.ice.predix.io/v1beta [EC Agent] loading application parameters.. [ ▄▄▄▄▄▄▄▄▄▄▄ ▄▄▄▄▄▄▄▄▄▄▄ ▐░░░░░░░░░░░▌▐░░░░░░░░░░░▌ ▐░█▀▀▀▀▀▀▀▀▀ ▐░█▀▀▀▀▀▀▀▀▀ ▐░▌ ▐░▌ ▐░█▄▄▄▄▄▄▄▄▄ ▐░▌ ▐░░░░░░░░░░░▌▐░▌ ▐░█▀▀▀▀▀▀▀▀▀ ▐░▌ ▐░▌ ▐░▌ ▐░█▄▄▄▄▄▄▄▄▄ ▐░█▄▄▄▄▄▄▄▄▄ ▐░░░░░░░░░░░▌▐░░░░░░░░░░░▌ ▀▀▀▀▀▀▀▀▀▀▀ ▀▀▀▀▀▀▀▀▀▀▀ @Digital Connect ] [EC Server x0] 2020/07/14 09:54:48 [cleaned up logging file(s). connectivity.log plugin.log] [EC Server x0] 2020/07/14 09:54:48 [Rev: v1.1beta.fukuoka.2737] [EC Server x0] 2020/07/14 09:54:48 [Agent websocket compression is being negotiated at level 0] [EC Server x0] 2020/07/14 09:54:48 [refresh duration is specified. Token auto-refresh is set.] [EC Server x0] 2020/07/14 09:54:48 [Fetching new token..] [EC Server x0] 2020/07/14 09:54:48 [Token refreshed. The token will be expired in 719 minutes. Approx. 20 minutes to the next auto-refresh.] [EC Server x0] 2020/07/14 09:54:48 [health api will be available on the local port#1199] [EC Server x0] 2020/07/14 09:54:48 [creating EC_Network ..] [EC Server x0] 2020/07/14 09:54:49 [unidentified gateway hostname https://eks-dev-ec-gateway.run.aws-usw02-pr.ice.predix.io bypassed from the gateway list.] [EC Server x0] 2020/07/14 09:54:49 [unidentified gateway hostname https://eks-dev-ec-gateway.run.aws-usw02-pr.ice.predix.io bypassed from the gateway list.] [EC Server x0] 2020/07/14 09:54:49 [unidentified gateway hostname https://eks-dev-ec-gateway.run.aws-usw02-pr.ice.predix.io bypassed from the gateway list.] [EC Server x0] 2020/07/14 09:54:49 [unidentified gateway hostname https://eks-dev-ec-gateway.run.aws-usw02-pr.ice.predix.io bypassed from the gateway list.] [EC Server x0] 2020/07/14 09:54:59 [unidentified gateway hostname https://eks-dev-ec-gateway.run.aws-usw02-pr.ice.predix.io bypassed from the gateway list.] [EC Server x0] 2020/07/14 09:54:59 [unidentified gateway hostname https://eks-dev-ec-gateway.run.aws-usw02-pr.ice.predix.io bypassed from the gateway list.] [EC Server x0] 2020/07/14 09:54:59 [unidentified gateway hostname https://eks-dev-ec-gateway.run.aws-usw02-pr.ice.predix.io bypassed from the gateway list.] [EC Server x0] 2020/07/14 09:54:59 [unidentified gateway

ayasuda2OO3 commented 4 years ago

what does the -hst or conf.hst=<gateway-url> look like in the gateway chart spec? @simran3695 @sheshankgujjari

Gayatri212 commented 4 years ago

@ayasda2003 The -hst in gateway is exactly same as -hst in server including wss:// and /agent but as per our testing we arrive to following conclusions

For gateway in ASW or running locally if we configure -hst flag as wss://gateway-url/agent it returns the same in gatewayList call but for gateway in predix(i.e. cf) even if we configure -hst as wss://gateway-url/agent the gatewayList call is returning the gateway as https://gateway-url.
As the gateway url returned by gatewayList call is not exact match for -hst in server we are getting unidentified gateway error even if the requested gateway is there in the gatewayList.

ayasuda2OO3 commented 4 years ago

Acknowledged. Release update scheduled.

Gayatri212 commented 4 years ago

@ayasuda2OO3 Here are the details of issue replicated

Gateway Script

./agent_linux_sys -mod gateway \
-gpt :8080 \
-zon <zone-id> \
-sst https://<zone-id>.run.aws-usw02-dev.ice.predix.io/v1beta \
-hst wss://test-gateway-2737.run.aws-usw02-dev.ice.predix.io/agent \
-tkn XXXXXXXXXXXXXX

Server Script

./agent_windows_sys -mod server \
-aid XXXXX -grp XXXXX \ 
-cid XXXXX -csc XXXXX \ 
-dur 3000 \
-oa2 https://<uaa-url>.predix-uaa.run.aws-usw02-dev.ice.predix.io/oauth/token \
-hst wss://test-gateway-2737.run.aws-usw02-dev.ice.predix.io/agent \
-zon <zone-id> \
-sst https://<zone-id>.run.aws-usw02-dev.ice.predix.io/v1beta \
-rht localhost -rpt :5432 \
-hca :8081 -dbg

Gateway Logs

2020-08-14T14:41:36.10+0530 [APP/PROC/WEB/0] OUT  [EC Gateway 0] 2020/08/14 09:11:36 [cleaned up logging file(s). connectivity.log plugin.log]
   2020-08-14T14:41:36.10+0530 [APP/PROC/WEB/0] OUT  [EC Gateway 0] 2020/08/14 09:11:36 [Rev: v1.1beta.fukuoka.2737]
   2020-08-14T14:41:36.10+0530 [APP/PROC/WEB/0] OUT  [EC Gateway 0] 2020/08/14 09:11:36 [Agent websocket compression is being negotiated at level 0]
   2020-08-14T14:41:36.10+0530 [APP/PROC/WEB/0] OUT  [EC Gateway 0] 2020/08/14 09:11:36 [Id: ae37a459-549e-4000-a402-1b3b042b058a]
   2020-08-14T14:41:36.10+0530 [APP/PROC/WEB/0] OUT  [EC Gateway 0] 2020/08/14 09:11:36 [instantiate usage..]
   2020-08-14T14:41:36.15+0530 [APP/PROC/WEB/0] OUT  [EC Gateway 0] 2020/08/14 09:11:36 [gateway is listening on the port# :8080]
   2020-08-14T14:41:36.15+0530 [APP/PROC/WEB/0] OUT  [EC Gateway 0] 2020/08/14 09:11:36 [creating EC_Network ..]
   2020-08-14T14:41:38.10+0530 [CELL/0] OUT Container became healthy
   2020-08-14T14:42:36.15+0530 [APP/PROC/WEB/0] OUT  [EC Gateway 0] 2020/08/14 09:12:36 [reporting usage..]
   2020-08-14T14:42:36.17+0530 [APP/PROC/WEB/0] OUT  [EC Gateway 0] 2020/08/14 09:12:36 [usage reported.]
   2020-08-14T14:43:36.15+0530 [APP/PROC/WEB/0] OUT  [EC Gateway 0] 2020/08/14 09:13:36 [reporting usage..]
   2020-08-14T14:43:36.16+0530 [APP/PROC/WEB/0] OUT  [EC Gateway 0] 2020/08/14 09:13:36 [usage reported.]
   2020-08-14T14:44:36.15+0530 [APP/PROC/WEB/0] OUT  [EC Gateway 0] 2020/08/14 09:14:36 [reporting usage..]
   2020-08-14T14:44:36.19+0530 [APP/PROC/WEB/0] OUT  [EC Gateway 0] 2020/08/14 09:14:36 [usage reported.]
   2020-08-14T14:45:36.15+0530 [APP/PROC/WEB/0] OUT  [EC Gateway 0] 2020/08/14 09:15:36 [reporting usage..]

Server Logs

 [EC Server x0] 2020/08/14 15:02:34 [[Rev: v1.1beta.fukuoka.2737]]
 [EC Server x0] 2020/08/14 15:02:34 [[Agent websocket compression is being negotiated at level 0]]
 [EC Server x0] 2020/08/14 15:02:34 [[refresh duration is specified. Token auto-refresh is set.]]
 [EC Server x0] 2020/08/14 15:02:34 [[Fetching new token..]]
 [EC Server x0 Dbg] 2020/08/14 15:02:34 [[the http scheme in the proxy is either empty or invalid.]]
 [EC Server x0 Dbg] 2020/08/14 15:02:35 [[eyJhbGciOiJSUzI1NiIsImprdSI6Imh0dHBzOi8vOTMxY2U4NWQtYjdmZS00YjFhLTlhZTYtZjdhZWUzOWE2NTE0LnByZWRpeC11YWEucnVuLmF3cy11c3cwMi1kZXYuaWNlLnByZWRpeC5pby90b2tlbl9rZXlzIiwia2lkIjoiUjVqakwiLCJ0eXAiOiJKV1QifQ.eyJqdGkiOiJkMDljODAzZDk3Yjg0MzA5YmYwZjhjM2UxOTgxZmEzOCIsInN1YiI6InVhYUNsaWVudCIsInNjb3BlIjpbInVhYS5ub25lIiwiZW50ZXJwcmlzZS1jb25uZWN0LnpvbmVzLjU2ODFjMmM1LWYxMjEtNDU1Yy1hMzhlLWZlNjQ4Mzg2NGQxNS51c2VyIl0sImNsaWVudF9pZCI6InVhYUNsaWVudCIsImNpZCI6InVhYUNsaWVudCIsImF6cCI6InVhYUNsaWVudCIsImdyYW50X3R5cGUiOiJjbGllbnRfY3JlZGVudGlhbHMiLCJyZXZfc2lnIjoiNGJhNTNmZTEiLCJpYXQiOjE1OTczOTc1NTUsImV4cCI6MTU5NzQ0MDc1NSwiaXNzIjoiaHR0cHM6Ly85MzFjZTg1ZC1iN2ZlLTRiMWEtOWFlNi1mN2FlZTM5YTY1MTQucHJlZGl4LXVhYS5ydW4uYXdzLXVzdzAyLWRldi5pY2UucHJlZGl4LmlvL29hdXRoL3Rva2VuIiwiemlkIjoiOTMxY2U4NWQtYjdmZS00YjFhLTlhZTYtZjdhZWUzOWE2NTE0IiwiYXVkIjpbInVhYUNsaWVudCIsImVudGVycHJpc2UtY29ubmVjdC56b25lcy41NjgxYzJjNS1mMTIxLTQ1NWMtYTM4ZS1mZTY0ODM4NjRkMTUiXX0.aML8UPcITfJtVlYtG21LocASdiQBRpvf4_KrTDxR7p1ArIf1RfK-y0xHyg6_S-uhmRDPunVh80XHEhQkjXmBV5BTXAfKfgvL3VBacI4ZBYWocBRbJ8_bxvBIifUg_rjUsvSq8honfSDOs4o3QnxVP06mDFUF8SIPyCrBHHbRWS7PvHA0tNz_FFBuEa2VFPphBI4Aleek84HaApIuRpRLFHVWFd9Tr89t-SmWzL2zXPAyOrJhkFoBk24F1tlF266EyZqKYmdSY793szCUiEm4fhZ6kpn8a760pGylJkV8szXRGqU4obMlqn056pnjH1PBNpmFhpjQq2gObauTyzei5Q]]
 [EC Server x0] 2020/08/14 15:02:35 [[Token refreshed. The token will be expired in 719 minutes. Approx. 50 minutes to the next auto-refresh.]]
 [EC Server x0] 2020/08/14 15:02:35 [[health api will be available on the local port#8081]]
 [EC Server x0 Dbg] 2020/08/14 15:02:35 [[0xc0004b8000]]
 [EC Server x0] 2020/08/14 15:02:35 [[creating EC_Network ..]]
 [EC Server x0 Dbg] 2020/08/14 15:02:35 [[failed create network error during connect: This error may indicate that the docker daemon is not running.: Post http://%2F%2F.%2Fpipe%2Fdocker_engine/v1.39/networks/create: open //./pipe/docker_engine: The system cannot find the file specified. continue daemon subprocess]]
 [EC Server x0 Dbg] 2020/08/14 15:02:35 [[lp C:\Users\212712078/.ec/agent_linux_sys.tar.gz]]
 [EC Server x0 Dbg] 2020/08/14 15:02:35 [[sp C:\Users\212712078/.ec/agent]]
 [EC Server x0 Dbg] 2020/08/14 15:02:35 [[downloadURL https://raw.githubusercontent.com/Enterprise-connect/sdk/v1.1beta/dist/agent//agent_linux_sys.tar.gz]]
 [EC Server x0 Dbg] 2020/08/14 15:02:35 [[the http scheme in the proxy is either empty or invalid.]]
 [EC Server x0 Dbg] 2020/08/14 15:02:35 [[the http scheme in the proxy is either empty or invalid.]]
 [EC Server x0 Dbg] 2020/08/14 15:02:36 [[{"glist":{},"data":"CLRZDCLK+Oj6vWDXmUt7Zt9lJ2N+aBmV2AJmLu1eOF578poM3hjNOQLv9EtvXyo20hIzC5OlUO5/sIKH2Ts0vb/UXRUg39eY4lhptMvZV0s8/LN9UOYBwlpOQc0HMEZT2W/OgF8Jnl62W+/4I8JvrOoOkkHTPDy9OKOpE1VWSqqlCFK/d8YufnFF9KvrntrgAN9vJzMBijkTCgrCdaACycM6BgWhQkj8zW+PuxzLal+A7+XP2QhEtXEaBTrsQv/U6AGCAGBJjR84AUFoEIzcwefj5EyMKrVtfQSkozIb9ODYTIzVG6fjnRZtT+Ni19zD/d0Yln3mQV5nd10abmDcDQ=="}]]
 [EC Server x0 Dbg] 2020/08/14 15:02:36 [[the http scheme in the proxy is either empty or invalid.]]
 [EC Server x0 Dbg] 2020/08/14 15:02:37 [[{"glist":{","ae37a459-549e-4000-a402-1b3b042b058a:0":{"refId":"0","gtwId":"ae37a459-549e-4000-a402-1b3b042b058a","zone":"5681c2c5-f121-455c-a38e-fe6483864d15","cfURL":"https://test-gateway-2737.run.aws-usw02-dev.ice.predix.io","timeCreated":1597397556167,"active":true}]]
 [EC Server x0 Dbg] 2020/08/14 15:02:37 [[gateway list exchange called.]]
 [EC Server x0 Dbg] 2020/08/14 15:02:37 [[CFRoutingURL: https://test-gateway-2737.run.aws-usw02-dev.ice.predix.io Server: wss://test-gateway-2737.run.aws-usw02-dev.ice.predix.io/agent]]
 [EC Server x0] 2020/08/14 15:02:37 [[unidentified gateway hostname https://test-gateway-2737.run.aws-usw02-dev.ice.predix.io bypassed from the gateway list.]]

Here we can see in the script that in both gateway and server script -hst is exactly same.
But if we see the response of gatewayList call from service it is giving CFUrl of gateway as https:// instead of wss:// and because of this mismatch the server is throwing error unidentified gateway although the gateway is present in the list

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

EC-Release / sdk