s3 backup script does not work

vrrao commented 10 months ago

Please read these guidelines before opening an issue.

Bug Description

https://github.com/IBM/couchbackup/blob/main/examples/s3-backup-stream.js does not work

1. Steps to reproduce and the simplest code sample possible to demonstrate the issue

This is what I did:

Clone the https://github.com/IBM/couchbackup repo

vrrao@vidyas-mbp Cloudant % git clone https://github.com/IBM/couchbackup.git
Cloning into 'couchbackup'...
remote: Enumerating objects: 4250, done.
remote: Counting objects: 100% (1134/1134), done.
remote: Compressing objects: 100% (262/262), done.
remote: Total 4250 (delta 994), reused 914 (delta 872), pack-reused 3116
Receiving objects: 100% (4250/4250), 1.82 MiB | 2.70 MiB/s, done.
Resolving deltas: 100% (3074/3074), done.

tried running `node ./s3-backup-stream.js -b -s

2. What you expected to happen

I expected the script to work and the DB to be backed up to my cos bucket

3. What actually happened

First it complained

vrrao@vidyas-mbp couchbackup % node ./examples/s3-backup-stream.js -b vidya-bucket -s https://3fc06ccb-c18a-46cf-892c-861a7cb06269-bluemix.cloudantnosqldb.appdomain.cloud/todo 
node:internal/modules/cjs/loader:1152
  throw err;
  ^

Error: Cannot find module 'aws-sdk'
Require stack:
- /Users/vrrao/Git-Projects/Cloudant/couchbackup/examples/s3-backup-stream.js
    at Module._resolveFilename (node:internal/modules/cjs/loader:1149:15)

I was told to install aws-sdk

vrrao@vidyas-mbp couchbackup % npm install --save aws-sdk
npm WARN EBADENGINE Unsupported engine {
npm WARN EBADENGINE   package: '@cloudant/couchbackup@2.9.17-SNAPSHOT',
npm WARN EBADENGINE   required: { node: '^18 || ^20' },
npm WARN EBADENGINE   current: { node: 'v21.6.0', npm: '10.3.0' }
npm WARN EBADENGINE }
npm WARN EBADENGINE Unsupported engine {
npm WARN EBADENGINE   package: '@ibm-cloud/cloudant@0.8.2',
npm WARN EBADENGINE   required: { node: '^18 || ^20' },
npm WARN EBADENGINE   current: { node: 'v21.6.0', npm: '10.3.0' }
npm WARN EBADENGINE }
npm WARN deprecated querystring@0.2.0: The querystring API is considered Legacy. new code should use the URLSearchParams API instead.

added 17 packages, and audited 358 packages in 3s

after which I got the error

vrrao@vidyas-mbp couchbackup % node ./examples/s3-backup-stream.js -b vidya-bucket -s https://3fc06ccb-c18a-46cf-892c-861a7cb06269-bluemix.cloudantnosqldb.appdomain.cloud/todo 
node:internal/modules/cjs/loader:1152
  throw err;
  ^

Error: Cannot find module '@cloudant/couchbackup'
Require stack:
- /Users/vrrao/Git-Projects/Cloudant/couchbackup/examples/s3-backup-stream.js
    at Module._resolveFilename (node:internal/modules/cjs/loader:1149:15)

So I did

vrrao@vidyas-mbp examples % npm install --save @cloudant/couchbackup
npm WARN EBADENGINE Unsupported engine {
npm WARN EBADENGINE   package: '@cloudant/couchbackup@2.9.16',
npm WARN EBADENGINE   required: { node: '^18 || ^20' },
npm WARN EBADENGINE   current: { node: 'v21.6.0', npm: '10.3.0' }
npm WARN EBADENGINE }
npm WARN EBADENGINE Unsupported engine {
npm WARN EBADENGINE   package: '@ibm-cloud/cloudant@0.8.2',
npm WARN EBADENGINE   required: { node: '^18 || ^20' },
npm WARN EBADENGINE   current: { node: 'v21.6.0', npm: '10.3.0' }
npm WARN EBADENGINE }
npm WARN deprecated querystring@0.2.0: The querystring API is considered Legacy. new code should use the URLSearchParams API instead.
npm WARN deprecated har-validator@5.1.5: this library is no longer supported
npm WARN deprecated json3@3.3.2: Please use the native JSON object instead of JSON 3
npm WARN deprecated cloudant-follow@0.18.2: This package is no longer maintained.
npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
npm WARN deprecated mkdirp@0.5.1: Legacy versions of mkdirp are no longer supported. Please update to mkdirp 1.x. (Note that the API surface has changed to use Promises in 1.x.)
npm WARN deprecated cloudant@1.10.0: This package is now scoped. The new name is @cloudant/cloudant

But after trying to run again I get

vrrao@vidyas-mbp examples % pwd
/Users/vrrao/Git-Projects/Cloudant/couchbackup/examples
vrrao@vidyas-mbp examples % node ./s3-backup-stream.js -b vidya-bucket -s https://3fc06ccb-c18a-46cf-892c-861a7cb06269-bluemix.cloudantnosqldb.appdomain.cloud/todo  
(node:10097) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
vrrao@vidyas-mbp examples % node ./s3-backup-stream.js -b vidya-bucket -s https://3fc06ccb-c18a-46cf-892c-861a7cb06269-bluemix.cloudantnosqldb.appdomain.cloud/todo --trace-deprecation
(node:10274) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
vrrao@vidyas-mbp examples % node --trace-deprecation ./s3-backup-stream.js -b vidya-bucket -s https://3fc06ccb-c18a-46cf-892c-861a7cb06269-bluemix.cloudantnosqldb.appdomain.cloud/todo 
(node:10314) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
    at node:punycode:3:9
    at BuiltinModule.compileForInternalLoader (node:internal/bootstrap/realm:397:7)
    at BuiltinModule.compileForPublicLoader (node:internal/bootstrap/realm:333:10)
    at loadBuiltinModule (node:internal/modules/helpers:101:7)
    at Module._load (node:internal/modules/cjs/loader:1006:17)
    at Module.require (node:internal/modules/cjs/loader:1237:19)
    at require (node:internal/modules/helpers:176:18)
    at Object.<anonymous> (/Users/vrrao/Git-Projects/Cloudant/couchbackup/examples/node_modules/psl/index.js:5:16)
    at Module._compile (node:internal/modules/cjs/loader:1378:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1437:10)

Environment details

vrrao@vidyas-mbp examples % node --version v21.6.0

ricellis commented 10 months ago

Thanks for reporting this we'll look to get the example dependencies updated.

Just to manage expectations though I will say that examples/s3-backup-stream.js is an example designed to help understand usage of couchbackup APIs for that kind of use case. It should not be considered production ready code.

Regarding the other warnings you observe:

npm WARN EBADENGINE Unsupported engine { npm WARN EBADENGINE package: '@cloudant/couchbackup@2.9.16', npm WARN EBADENGINE required: { node: '^18 || ^20' },

The README Requirements for this repository state Node.js LTS version 18 or 20. (i.e. the LTS releases of Node that are considered production stable). The EBADENGINE warnings are npm reflecting that, it is only a warning and will not prevent the install/use of the package if you want to try to use it in Node 21 (unless you are using npm with engine-strict: true).

(node:10097) [DEP0040] DeprecationWarning: The punycode module is deprecated. Please use a userland alternative instead.

This is a transitive dependency in the stack we use that will be resolved when updates become available. couchbackup continues to operate normally despite this warning.

ricellis commented 9 months ago

https://github.com/IBM/couchbackup/tree/main/examples has been updated and some additional documentation added.

IBM / couchbackup