loopbackio / loopback-next

LoopBack makes it easy to build modern API applications that require complex integrations.
https://loopback.io
Other
4.94k stars 1.06k forks source link

Unable to deploy successfully to Cloud Foundry #2033

Closed cloudwheels closed 5 years ago

cloudwheels commented 5 years ago

Description / Steps to reproduce / Feature proposal

Please help - this has been driving me mental for 24hours, I've had plenty of stabs at different approaches but it just doesn't seem to work. I must be missing something really obvious but I have the fear that this version is all a bit bleeding edge right now and things seems to be in a state of flux with the IBM CLI tools!

...
2018-11-16T00:20:11.38+0000 [APP/PROC/WEB/0] OUT Server is running at http://127.0.0.1:8080
   2018-11-16T00:20:11.38+0000 [APP/PROC/WEB/0] OUT Try http://127.0.0.1:8080/ping
   2018-11-16T00:21:05.96+0000 [CELL/0] ERR Failed to make TCP connection to port 8080: connection refused
   2018-11-16T00:21:05.96+0000 [CELL/0] ERR Timed out after 1m0s: health check never passed.
   2018-11-16T00:21:05.97+0000 [CELL/SSHD/0] OUT Exit status 0
   2018-11-16T00:21:22.42+0000 [CELL/0] OUT Cell 457412ee-4634-4ef1-8a30-09bb659d4528 stopping instance bf9085db-41c2-4900-5105-ab7d
   2018-11-16T00:21:22.42+0000 [CELL/0] OUT Cell 457412ee-4634-4ef1-8a30-09bb659d4528 destroying container for instance bf9085db-41c2-4900-5105-ab7d
   2018-11-16T00:21:22.42+0000 [API/17] OUT Process has crashed with type: "web"
   2018-11-16T00:21:22.45+0000 [API/17] OUT App instance exited with guid 49593ebe-4e94-4887-b9bc-22df27e7cd6e payload: {"instance"=>"bf9085db-41c2-4900-5105-ab7d", "index"=>0, "reason"=>"CRASHED", "exit_description"=>"Instance never hea
lthy after 1m0s: Failed to make TCP connection to port 8080: connection refused; process did not exit", "crash_count"=>2, "crash_timestamp"=>1542327682377182159, "version"=>"7ccff2db-027d-4178-8fbf-dc10c7d08caa"}
   2018-11-16T00:21:22.70+0000 [CELL/0] OUT Cell fe12f71d-7f0c-4253-aad2-75fb6d829ddb creating container for instance de474677-0293-4847-4824-75cd
   2018-11-16T00:21:23.11+0000 [PROXY/0] OUT Exit status 137
   2018-11-16T00:21:30.97+0000 [CELL/0] OUT Cell 457412ee-4634-4ef1-8a30-09bb659d4528 successfully destroyed container for instance bf9085db-41c2-4900-5105-ab7d
   2018-11-16T00:21:32.70+0000 [CELL/0] OUT Cell fe12f71d-7f0c-4253-aad2-75fb6d829ddb successfully created container for instance de474677-0293-4847-4824-75cd
   2018-11-16T00:21:48.84+0000 [CELL/0] OUT Starting health monitoring of container
   2018-11-16T00:21:52.74+0000 [APP/PROC/WEB/0] OUT > getting-started@1.0.0 start /home/vcap/app
   2018-11-16T00:21:52.74+0000 [APP/PROC/WEB/0] OUT > node index.js
   2018-11-16T00:21:56.64+0000 [APP/PROC/WEB/0] OUT Server is running at http://127.0.0.1:8080
   2018-11-16T00:21:56.64+0000 [APP/PROC/WEB/0] OUT Try http://127.0.0.1:8080/ping
   2018-11-16T00:22:50.43+0000 [CELL/0] ERR Failed to make TCP connection to port 8080: connection refused
   2018-11-16T00:22:50.43+0000 [CELL/0] ERR Timed out after 1m0s: health check never passed.
   2018-11-16T00:22:50.50+0000 [CELL/SSHD/0] OUT Exit status 0
   2018-11-16T00:23:06.68+0000 [CELL/0] OUT Cell fe12f71d-7f0c-4253-aad2-75fb6d829ddb stopping instance de474677-0293-4847-4824-75cd
   2018-11-16T00:23:06.68+0000 [CELL/0] OUT Cell fe12f71d-7f0c-4253-aad2-75fb6d829ddb destroying container for instance de474677-0293-4847-4824-75cd
   2018-11-16T00:23:06.70+0000 [API/4] OUT Process has crashed with type: "web"
   2018-11-16T00:23:06.72+0000 [API/4] OUT App instance exited with guid 49593ebe-4e94-4887-b9bc-22df27e7cd6e payload: {"instance"=>"de474677-0293-4847-4824-75cd", "index"=>0, "reason"=>"CRASHED", "exit_description"=>"Instance never heal
thy after 1m0s: Failed to make TCP connection to port 8080: connection refused; process did not exit", "crash_count"=>3, "crash_timestamp"=>1542327786672180733, "version"=>"7ccff2db-027d-4178-8fbf-dc10c7d08caa"}
   2018-11-16T00:23:08.73+0000 [PROXY/0] OUT Exit status 137
   2018-11-16T00:23:21.91+0000 [CELL/0] OUT Cell fe12f71d-7f0c-4253-aad2-75fb6d829ddb successfully destroyed container for instance de474677-0293-4847-4824-75cd
   2018-11-16T00:23:39.43+0000 [CELL/0] OUT Cell 19f96f82-bf0b-406a-83d4-75e6f9603d22 creating container for instance 47011f03-85be-4932-79d8-c3cc
   2018-11-16T00:23:40.57+0000 [CELL/0] OUT Cell 19f96f82-bf0b-406a-83d4-75e6f9603d22 successfully created container for instance 47011f03-85be-4932-79d8-c3cc
   2018-11-16T00:23:48.34+0000 [CELL/0] OUT Starting health monitoring of container
   2018-11-16T00:23:49.78+0000 [APP/PROC/WEB/0] OUT > getting-started@1.0.0 start /home/vcap/app
   2018-11-16T00:23:49.78+0000 [APP/PROC/WEB/0] OUT > node index.js
   2018-11-16T00:23:51.45+0000 [APP/PROC/WEB/0] OUT Server is running at http://127.0.0.1:8080
   2018-11-16T00:23:51.46+0000 [APP/PROC/WEB/0] OUT Try http://127.0.0.1:8080/ping
   2018-11-16T00:24:49.02+0000 [CELL/0] ERR Failed to make TCP connection to port 8080: connection refused
   2018-11-16T00:24:49.02+0000 [CELL/0] ERR Timed out after 1m0s: health check never passed.
   2018-11-16T00:24:49.04+0000 [CELL/SSHD/0] OUT Exit status 0
   2018-11-16T00:25:05.24+0000 [CELL/0] OUT Cell 19f96f82-bf0b-406a-83d4-75e6f9603d22 stopping instance 47011f03-85be-4932-79d8-c3cc
   2018-11-16T00:25:05.24+0000 [CELL/0] OUT Cell 19f96f82-bf0b-406a-83d4-75e6f9603d22 destroying container for instance 47011f03-85be-4932-79d8-c3cc
   2018-11-16T00:25:05.27+0000 [API/9] OUT Process has crashed with type: "web"
   2018-11-16T00:25:05.28+0000 [API/9] OUT App instance exited with guid 49593ebe-4e94-4887-b9bc-22df27e7cd6e payload: {"instance"=>"47011f03-85be-4932-79d8-c3cc", "index"=>0, "reason"=>"CRASHED", "exit_description"=>"Instance never heal
thy after 1m0s: Failed to make TCP connection to port 8080: connection refused; process did not exit", "crash_count"=>4, "crash_timestamp"=>1542327905234935974, "version"=>"7ccff2db-027d-4178-8fbf-dc10c7d08caa"}
   2018-11-16T00:25:05.57+0000 [PROXY/0] OUT Exit status 137
   2018-11-16T00:25:08.39+0000 [CELL/0] OUT Cell 19f96f82-bf0b-406a-83d4-75e6f9603d22 successfully destroyed container for instance 47011f03-85be-4932-79d8-c3cc
{
  "name": "getting-started",
  "version": "1.0.0",
  "description": "getting-started",
  "keywords": [
    "loopback-application",
    "loopback"
  ],
  "main": "index.js",
  "engines": {
    "node": ">=8.9"
  },
  "scripts": {
    "build:apidocs": "lb-apidocs",
    "build": "lb-tsc es2017 --outDir dist",
    "build:watch": "lb-tsc --watch",
    "clean": "lb-clean dist",
    "lint": "npm run prettier:check && npm run tslint",
    "lint:fix": "npm run tslint:fix && npm run prettier:fix",
    "prettier:cli": "lb-prettier \"**/*.ts\" \"**/*.js\"",
    "prettier:check": "npm run prettier:cli -- -l",
    "prettier:fix": "npm run prettier:cli -- --write",
    "tslint": "lb-tslint",
    "tslint:fix": "npm run tslint -- --fix",
    "pretest": "npm run clean && npm run build",
    "test": "lb-mocha --allow-console-logs \"dist/test\"",
    "posttest": "npm run lint",
    "test:dev": "lb-mocha --allow-console-logs dist/test/**/*.js && npm run posttest",
    "start": "node index.js",
    "prepublishOnly": "npm run test"
  },
  "repository": {
    "type": "git"
  },
  "author": "",
  "license": "",
  "files": [
    "README.md",
    "index.js",
    "index.d.ts",
    "dist/src",
    "dist/index*",
    "src"
  ],
  "dependencies": {
    "@loopback/boot": "^1.0.4",
    "@loopback/context": "^1.1.0",
    "@loopback/core": "^1.1.0",
    "@loopback/openapi-v3": "^1.1.1",
    "@loopback/repository": "^1.0.4",
    "@loopback/rest": "^1.3.0",
    "@loopback/service-proxy": "^1.0.2"
  },
  "devDependencies": {
    "@loopback/build": "^1.0.1",
    "@loopback/testlab": "^1.0.1",
    "@types/node": "^10.11.2"
  }
}
$ node -v
v8.9.4
$ npm --v
5.6.0
dhmlau commented 5 years ago

@cloudwheels, please ignore my previous comment. I think the better way to get this working is to add this line in src/index.ts:

  options.rest.host = appEnv.isLocal ? options.rest.host : appEnv.host;

i.e.

export async function main(options: ApplicationConfig = {}) {
  // Set the port assined for the app
  if (!options) options = {};
  if (!options.rest) options.rest = {};
  options.rest.port = appEnv.isLocal ? options.rest.port : appEnv.port;
  // ------- ADD THIS LINE ----------
  options.rest.host = appEnv.isLocal ? options.rest.host : appEnv.host;
  // ---------------------------------
  const app = new TodoListApplication(options);

I'll submit a PR to update the docs. Thanks for reporting this issue.

~@cloudwheels, I'm just trying out as well. It might be related to the recent change: https://github.com/strongloop/loopback-next/commit/2e7d4bb6e7b30aedfcc373a0d2bed3c0f1ad07c8. My workaround is not to use the config value in index.js, i.e.~

cloudwheels commented 5 years ago

Thank you much @dhmlau -confirm this works - please see next comment for more succinct summary than this one...

I have / had taken time to check the repo & issues & web generally before posting this issue, so I can see how hard you are all working and pressing issues such as reviewing & extending the deployment docs for various platforms are being dealt with via the monthly milestones.

I'm sure you are aware there is little more frustrating to a developer than the cycle: 2 minute working quick start all looks amazing > follow instructions carefully on learning more > doesn't work, so multiple hours head-bashing > finally get correct info on 10 second fix and find problem is due to undocumented change in the code base

My suggestion would be to start deployment documentation with the simplest (ping) starter with the assumption that developer has no working knowledge of IBM Cloud/cloud foundry/bluemix whatsoever (I'm coming from Google coud so I don't struggle with the concepts of PaaS, just IBMs silly names for the same things), so that the basics of setting up and deploying to a free tier can be understood and verified before moving to more complex examples. (Adding a cloudant database both remotely & locally for the todo example within the 'deployment' how to introduces other potential problems in the code / environment and has nothing to do with learning about deployment.)

The deployment how to should also follow the same login / CLI tool installations and commands as the node.js SDK starter app on CF (ie using the bx commands) .

I have seen that the ability to use a local explorer is now live and partially documented: with the redirected version I get a probable CORS error from the cloud foundry app (and in my 'local' google cloud shell environment where I have no control over CORS). I would be helpful to know whether I need to get involved with the 'api' section of the cloud foundry console: 'expose manged api' etc. to manage CORS for the cf app and/or how this is relevant to loopback app running on cf and how to deal with this issue.

I will attempt to turn these comments into more granular / useful contributions to existing specific issues when I have some time [update - see comments below], so apologies this is a bit of an unstructured record of my experience / rant, and many thanks for all the excellent work on this exciting project and the help getting over this little hurdle.

cloudwheels commented 5 years ago

So, tldr;

// Modified src/index.ts (Generated Starter Application) for deployment to IBM Cloud Foundry
// Use cfenv.getAppEnv() to get port & host environment variables

import {StarterApplication} from './application';
import {ApplicationConfig} from '@loopback/core';
const cfenv = require('cfenv');
const appEnv = cfenv.getAppEnv();

export {StarterApplication};

export async function main(options: ApplicationConfig = {}) {
    // Set the port and host assigned for the app
  if (!options) options = {};
  if (!options.rest) options.rest = {};
  options.rest.port = appEnv.isLocal ? options.rest.port : appEnv.port;
  options.rest.host = appEnv.isLocal ? options.rest.host : appEnv.host;

  const app = new StarterApplication(options);
  await app.boot();
  await app.start();

  const url = app.restServer.url;
  console.log(`Server is running at ${url}`);
  console.log(`Try ${url}/ping`);

  return app;
}

I have just written this as an aide-memoir so will work through and check that it works :) & comment back...

Could a 'Deploy to IBM cloud?' option be added to the generator to automatically install the 2 necessary dependencies and generate a modified src/index.ts to include the cfenv variables? (and going forward run the bx commands to login & deploy to the correct endpoint)

cloudwheels commented 5 years ago

Related open issues:

1908

1709

As above, I think it is confusing to introduce data persistence within the deployment tutorial - why do I want to start messing with docker when I'm trying to see if I can get a simple ping from the cloud? (more below)...

In any event adding hosted cloudant URL to the cloudant datasource connector and using the same database for local and cloud versions for the time being is a lot simpler to describe and understand as a temporary development step (no need to even add the cloudant service to the app in the console - the benefits of doing so can be explained later), and avoids having to use the docker environment which has other complications: e.g. using Google Cloud Shell I can get a cloudant in docker set up running on port 9000 at the same time as my app is running on 3000, but cannot connect to it from the app because the GCShell preview context it's running in is tied to my google account, so the request for the db url just gets back the Google accounts auth page (and I'm unaware if this can be overcome by adding a bearer token to the header in the controller code for example). OK, I worked out my own workaround using the hosted database only (no separate cloudant-in-docker) but having to deal with additional issues like this when following 'quickstart' tutorials at a tender stage of learning is somewhat frustrating and off-putting.

dhmlau commented 5 years ago

@cloudwheels, thanks for your feedback and the feature proposal! I agree that the how-to guide should contain more detailed information (perhaps with screenshots especially for the IBM Cloud side). We've investigated there are additional steps required if the LoopBack application is using other IBM Cloud service for persistence, so think that it would be useful to include it as well.

Based on your feedback, I'd like to propose the following to move things forward:

  1. Add the options.rest.host line as mentioned in my previous comment so that users can deploy by following the instruction
  2. Include more details for steps that involve IBM Cloud, e.g. how to sign up, login (cf login), etc.
  3. Have a progression in complexity of the app in terms of the deployment. i.e. first to show how to get ping works on IBM Cloud, then next to show additional steps when using provisioned services.
  4. Eventually (longer term), we'll work on a better user experience in deployment, like your proposal https://github.com/strongloop/loopback-next/issues/2038

Thoughts?

cloudwheels commented 5 years ago

@dhmlau - thank for the swift reply :)

I confirm I now have todo running on the IBM cloud by making these changes (but no docker database or connected services, just a connection string from the cloudant connector shared by both environments - simples! - I was guided in this by I think the docs for the github sample (which didn't build for me btw and is probs a bit outdated also)).

With some minor level of familiarity now I was able to just update the manifest to upload the todo example over the ping starter: ibmcloud cf create-app-manifest lb4-starter-app -p manifest.yml

Your plan sounds good.

Absolutely - step by step breakdowns where poss - get a ping (starter + deployment), add a simple controller (hello), add more complex (todo)/related (todoList) models & controllers, get external api stuff (github stargazer), persist data in memory, persist to single hosted cloudant, setup cloudant dev in docker, add authentication etc. etc. User can always jump to the most complex example and run that first if they want or are returning / progressing their experience: each should work as a unit.

Then, yes, see it all working all ultimately as one pagers / clickers for the best ux. (ref. google cloud where for many examples a button click in the product overview will pull code into the console from git, build & deployed for you, apis enabled etc with a bit of entering your email address.)

I would say trying to align / integrate it to the current process on the IBM (Bluemix / whatever) console for creating a node.js sdk app, with the prefilled copy-pasta commands, with 'setup as a loopback app' being an option from here to provide relevant instructions from a single point that the user has to go to in order to set up the node app anyway (no duplication of documentation) [does that make sense].

Please also go with one CLI tool for this across all example documentation if that is workable (seems to be) i.e. the one currently referred to in these node.js docs, and clarify which commands of bluemix,bx,cf,ibmcloud etc. are relevant and how used.

Hope that helps! Look forward to following progress :)

cloudwheels commented 5 years ago

ie.

@dhmlau - I have added this suggestion to my #2038

dhmlau commented 5 years ago

see PR https://github.com/strongloop/loopback-next/pull/3254.

dhmlau commented 5 years ago

@cloudwheels, since it's similar to the other issue you created https://github.com/strongloop/loopback-next/issues/2072, I'd like to close this issue and continue the discussion over there. Thanks.