Improve project usability

[dgaubert]

This is the kind of PR that makes me happier, a.k.a developer happiness:

• Stop using two different tools for package management, testing, and any other developer workflow.
  • Removes Makefile and related bash scripts
  • Use npm scripts as the only tool for testing, CI and linting.
• Simplified CI configuration.
• Improved documentation:
  • Centralized several documents into README.md
  • Remove outdated sections
  • Update old sections
  • Added missing sections.
• Remove deprecated coverage tool istanbul, using nyc instead.
• Removed unused dockerfiles [...]

[Algunenano]

I get the following error when running npm test:

{ Error: Command failed: psql -c "CREATE EXTENSION IF NOT EXISTS cartodb CASCADE;" test_windshaft_cartodb_user_1_db
could not find a "psql" to execute
could not find a "psql" to execute
psql: fatal: could not find own program executable

    at ChildProcess.exithandler (child_process.js:294:12)
    at ChildProcess.emit (events.js:198:13)
    at maybeClose (internal/child_process.js:982:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:259:5)
  killed: false,
  code: 1,
  signal: null,
  cmd:
   'psql -c "CREATE EXTENSION IF NOT EXISTS cartodb CASCADE;" test_windshaft_cartodb_user_1_db',
  stdout: '',
  stderr:
   'could not find a "psql" to execute\ncould not find a "psql" to execute\npsql: fatal: could not find own program executable\n' }

psql is accesible and in the search path:

$ which psql
/usr/bin/psql

$ psql
psql (12.1)
Type "help" for help.

raul=# 

[Algunenano]

Is test/support/prepare_db.sh still needed after these changes?

[dgaubert]

Is test/support/prepare_db.sh still needed after these changes?

Nope

[Algunenano]

About the psql thing. It works if I remove the ENV options, so it seems node might be overwritting all environment variables when that's passed. I'd suggest either extending the current env or dropping that and ask PGUSER to be setup properly with superadmin rights.

[dgaubert]

Yes, totally. Going to find a better way to do that.

[dgaubert]

Can you try this, please? https://github.com/CartoDB/Windshaft-cartodb/pull/1146

[Algunenano]

Can you try this, please? #1146

That works for me. Now it runs the tests but I get a different issue, which seems unrelated to this.

[dgaubert]

But I get a different issue, which seems unrelated to this.

Please, let me know if it's related.

[Algunenano]

Looks like the setup used to add cartodb to the search_path of the user and it no longer does it, so I have several failures:

• camshaft: When calling CDB_QueryTables_Updated_At. Using cartodb.CDB_QueryTables_Updated_At is enough but I guess this should be replaced with node-cartodb-query-tables.
• windshaft-cartodb / camshaft: Reading cdb_analysis_catalog. Changing it to cartodb.cdb_analysis_catalog both in Winsdshaft-cartodb and camshaft is the best option to avoid conflicts.

I can create PRs if you want to fix those issues there instead of forcing cartodb to be in the search_path if you want.

[dgaubert]

Whoa!

I think I should keep the previous behavior and set the search_path up. Wondering why you have so many issues with this PR.

[dgaubert]

I thought we were setting search_path here: https://github.com/CartoDB/Windshaft-cartodb/blob/master/test/support/sql/windshaft.test.sql#L14

[Algunenano]

I think I should keep the previous behavior and set the search_path up. Wondering why you have so many issues with this PR.

Probably because things are tested with 3 different flavours of Ubuntu with the same setup.

I thought we were setting search_path here: https://github.com/CartoDB/Windshaft-cartodb/blob/master/test/support/sql/windshaft.test.sql#L14

That doesn't include other users. The failing tests are using test_windshaft_publicuser and test_windshaft_regular1.

[dgaubert]

I see, sorry if I didn't get what you meant: what does this PR different for having so many issues with your setup?

[Algunenano]

I see, sorry if I didn't get what you meant: what does this PR different for having so many issues with your setup?

I don't think this PR has many issues with my setup, instead I'd say that testing with a different setup has uncovered 2 bugs that were either introduced or uncovered by this PR. Ideally those should have been detected by the CI, not me, but we can't test everything.

[dgaubert]

You man, the wild CI.

[dgaubert]

In fact, by solving the issue you had I've realized I can simplify npm-scripts to be able to run coverage.