Remote CouchDB is not encrypted

vardas-bill commented 7 years ago

When using crypto-pouch in an ionic 2 App and replicating to a remote CouchDB database the CouchDB documents are not encypted.

The code I am using looks like this:

` init(localDBName, remoteDBName) { this._DB = new PouchDB(localDBName);

PouchDB.plugin(CryptoPouch);
this._DB.crypto('a password');

this.remote = remoteDBName;    

console.log('Data: init(): PouchDB database opened for localDBName = ' + localDBName + ' : ' + this._DB + ', remoteDBName = ' + this.remote);

// Insert the url for the CouchDB server into the remoteDBName 
var realRemoteDB = this.remote.replace("localhost:5984", COUCHDB_SERVER_URL);

console.log('Data: init(): remoteDB path being used is: ' + realRemoteDB);

this._remoteDB = new PouchDB(realRemoteDB);
this._remoteDB.crypto('a password');

let options = {
  live: true,
  retry: true,
  continuous: true
};

return this._DB.sync(realRemoteDB) //No options here -> One time sync
  .on('complete', (info) => {
    console.log('Data: init(): first one time sync has completed about to do live syncing now');
    this.events.publish('SYNC_FINISHED', true); // Let login know we have synced the data
    return this._DB.sync(realRemoteDB, options) //Continous sync with options
      .on('complete', (info) => {
        console.log('***** DATA: init() Complete: Handling syncing complete');
        console.dir(info);
      })
      .on('change', (info) => {
        console.log('***** DATA: init() Change: Handling syncing change');
        console.dir(info);
      })
      .on('paused', (info) => {
        console.log('***** DATA: init() Paused: Handling syncing pause');
        console.dir(info);
      })
      .on('active', (info) => {
        console.log('***** DATA: init() Active: Handling syncing resumption');
        console.dir(info);
      })
      .on('error', (err) => {
        console.log('***** DATA: init() Error: Handling syncing error');
        console.dir(err);
      })
      .on('denied', (err) => {
        console.log('***** DATA: init() Denied: Handling syncing denied');
        console.dir(err);
      });
  });

}`

calvinmetcalf commented 7 years ago

likely an issue with transform pouch which this is based on

vardas-bill commented 7 years ago

Hi Calvin,

Is there anything you can suggest? Should I hunt for the bug in the transform pouch code and if so can you point me in the right direction?

Very Best

Bill

From: Calvin Metcalf notifications@github.com Reply-To: calvinmetcalf/crypto-pouch reply@reply.github.com Date: Wednesday, 5 April 2017 at 23:23 To: calvinmetcalf/crypto-pouch crypto-pouch@noreply.github.com Cc: Bill Noble bill.noble@vardas.com, Author author@noreply.github.com Subject: Re: [calvinmetcalf/crypto-pouch] Remote CouchDB is not encrypted (#54)

likely an issue with transform pouch which this is based on

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

calvinmetcalf commented 7 years ago

I can take a deeper look today, what versions of everything are you using?

On Thu, Apr 6, 2017, 5:04 AM Bill Noble notifications@github.com wrote:

Hi Calvin,

Is there anything you can suggest? Should I hunt for the bug in the transform pouch code and if so can you point me in the right direction?

Very Best

Bill

From: Calvin Metcalf notifications@github.com Reply-To: calvinmetcalf/crypto-pouch reply@reply.github.com Date: Wednesday, 5 April 2017 at 23:23 To: calvinmetcalf/crypto-pouch crypto-pouch@noreply.github.com Cc: Bill Noble bill.noble@vardas.com, Author author@noreply.github.com Subject: Re: [calvinmetcalf/crypto-pouch] Remote CouchDB is not encrypted (#54)

likely an issue with transform pouch which this is based on

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/calvinmetcalf/crypto-pouch/issues/54#issuecomment-292113173, or mute the thread https://github.com/notifications/unsubscribe-auth/ABE4ny5gSpewFzKceUjoMn9hFPeS85Eyks5rtKp_gaJpZM4M0w0i .

vardas-bill commented 7 years ago

Thanks Calvin that is greatly appreciated.

The versions I am using are:

"pouchdb": "^6.1.2",

"crypto-pouch": "^3.1.1",

From: Calvin Metcalf notifications@github.com Reply-To: calvinmetcalf/crypto-pouch reply@reply.github.com Date: Thursday, 6 April 2017 at 12:34 To: calvinmetcalf/crypto-pouch crypto-pouch@noreply.github.com Cc: Bill Noble bill.noble@vardas.com, Author author@noreply.github.com Subject: Re: [calvinmetcalf/crypto-pouch] Remote CouchDB is not encrypted (#54)

I can take a deeper look today, what versions of everything are you using?

On Thu, Apr 6, 2017, 5:04 AM Bill Noble notifications@github.com wrote:

Hi Calvin,

Is there anything you can suggest? Should I hunt for the bug in the transform pouch code and if so can you point me in the right direction?

Very Best

Bill

From: Calvin Metcalf notifications@github.com Reply-To: calvinmetcalf/crypto-pouch reply@reply.github.com Date: Wednesday, 5 April 2017 at 23:23 To: calvinmetcalf/crypto-pouch crypto-pouch@noreply.github.com Cc: Bill Noble bill.noble@vardas.com, Author author@noreply.github.com Subject: Re: [calvinmetcalf/crypto-pouch] Remote CouchDB is not encrypted (#54)

likely an issue with transform pouch which this is based on

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/calvinmetcalf/crypto-pouch/issues/54#issuecomment-292113173, or mute the thread https://github.com/notifications/unsubscribe-auth/ABE4ny5gSpewFzKceUjoMn9hFPeS85Eyks5rtKp_gaJpZM4M0w0i .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

calvinmetcalf commented 7 years ago

in both calls to sync you need to use the remote pouchdb instance, not just the name of the remote database,

this._DB.sync(this._remote)

you may need to save this._remote in a variable for use in the second call to sync but the gist of it is that since encryption is done locally, you can't refer to the database by name, because then pouchdb will internally create a new instance that won't have encryption associated with it.

vardas-bill commented 7 years ago

Thanks Calvin!

I will try that out now and let you know how I get on.

From: Calvin Metcalf notifications@github.com Reply-To: calvinmetcalf/crypto-pouch reply@reply.github.com Date: Friday, 7 April 2017 at 15:15 To: calvinmetcalf/crypto-pouch crypto-pouch@noreply.github.com Cc: Bill Noble bill.noble@vardas.com, Author author@noreply.github.com Subject: Re: [calvinmetcalf/crypto-pouch] Remote CouchDB is not encrypted (#54)

in both calls to sync you need to use the remote pouchdb instance, not just the name of the remote database,

this._DB.sync(this._remote) you may need to save this._remote in a variable for use in the second call to sync but the gist of it is that since encryption is done locally, you can't refer to the database by name, because then pouchdb will internally create a new instance that won't have encryption associated with it.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

vardas-bill commented 7 years ago

Yaay! That seems to have worked.

I assume that all of the encrypted data is in the ‘data’ field of the document?

Massive thanks for helping me sort this it is greatly appreciated.

Very Best,

Bill

From: Calvin Metcalf notifications@github.com Reply-To: calvinmetcalf/crypto-pouch reply@reply.github.com Date: Friday, 7 April 2017 at 15:15 To: calvinmetcalf/crypto-pouch crypto-pouch@noreply.github.com Cc: Bill Noble bill.noble@vardas.com, Author author@noreply.github.com Subject: Re: [calvinmetcalf/crypto-pouch] Remote CouchDB is not encrypted (#54)

in both calls to sync you need to use the remote pouchdb instance, not just the name of the remote database,

this._DB.sync(this._remote) you may need to save this._remote in a variable for use in the second call to sync but the gist of it is that since encryption is done locally, you can't refer to the database by name, because then pouchdb will internally create a new instance that won't have encryption associated with it.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

calvinmetcalf commented 7 years ago

yup thats the encrypted data

vardas-bill commented 7 years ago

Hi Calvin,

I have tested the revised sync calls running my ionic 2 app in the browser and it seems to work great.

However, when I run my app on my iPhone the sync times out with an ETIMEDOUT error:

{"line":3,"column":4869,"sourceURL":"file:///var/containers/Bundle/Application/CD1C2BBE-1425-4E54-AA21-BA1518B7BFBD/vanilla%20app.app/www/build/polyfills.js","originalStack":"v@file:///var/containers/Bundle/Application/CD1C2BBE-1425-4E54-AA21-BA1518B7BFBD/vanilla%20app.app/www/build/polyfills.js:3:4869\nonreadystatechange@file:///var/containers/Bundle/Application/CD1C2BBE-1425-4E54-AA21-BA1518B7BFBD/vanilla%20app.app/www/build/main.js:173087:24\nn@file:///var/containers/Bundle/Application/CD1C2BBE-1425-4E54-AA21-BA1518B7BFBD/vanilla%20app.app/www/build/polyfills.js:2:27345\ninvokeTask@file:///var/containers/Bundle/Application/CD1C2BBE-1425-4E54-AA21-BA1518B7BFBD/vanilla%20app.app/www/build/polyfills.js:3:10289\nrunTask@file:///var/containers/Bundle/Application/CD1C2BBE-1425-4E54-AA21-BA1518B7BFBD/vanilla%20app.app/www/build/polyfills.js:3:7647\ninvoke@file:///var/containers/Bundle/Application/CD1C2BBE-1425-4E54-AA21-BA1518B7BFBD/vanilla%20app.app/www/build/polyfills.js:3:11404\nabort@[native code]\ncancelTask@file:///var/containers/Bundle/Application/CD1C2BBE-1425-4E54-AA21-BA1518B7BFBD/vanilla%20app.app/www/build/polyfills.js:3:10646\ncancelTask@file:///var/containers/Bundle/Application/CD1C2BBE-1425-4E54-AA21-BA1518B7BFBD/vanilla%20app.app/www/build/polyfills.js:3:8206\nfile:///var/containers/Bundle/Application/CD1C2BBE-1425-4E54-AA21-BA1518B7BFBD/vanilla%20app.app/www/build/polyfills.js:3:3203\nabort\ntimeoutReq@file:///var/containers/Bundle/Application/CD1C2BBE-1425-4E54-AA21-BA1518B7BFBD/vanilla%20app.app/www/build/main.js:172991:14\ninvokeTask@file:///var/containers/Bundle/Application/CD1C2BBE-1425-4E54-AA21-BA1518B7BFBD/vanilla%20app.app/www/build/polyfills.js:3:10289\nrunTask@file:///var/containers/Bundle/Application/CD1C2BBE-1425-4E54-AA21-BA1518B7BFBD/vanilla%20app.app/www/build/polyfills.js:3:7647\ninvoke@file:///var/containers/Bundle/Application/CD1C2BBE-1425-4E54-AA21-BA1518B7BFBD/vanilla%20app.app/www/build/polyfills.js:3:11404\nfile:///var/containers/Bundle/Application/CD1C2BBE-1425-4E54-AA21-BA1518B7BFBD/vanilla%20app.app/www/build/polyfills.js:2:30198","zoneAwareStack":"v@file:///var/containers/Bundle/Application/CD1C2BBE-1425-4E54-AA21-BA1518B7BFBD/vanilla%20app.app/www/build/polyfills.js:3:4869\nonreadystatechange@file:///var/containers/Bundle/Application/CD1C2BBE-1425-4E54-AA21-BA1518B7BFBD/vanilla%20app.app/www/build/main.js:173087:24\nn@file:///var/containers/Bundle/Application/CD1C2BBE-1425-4E54-AA21-BA1518B7BFBD/vanilla%20app.app/www/build/polyfills.js:2:27345\ninvokeTask@file:///var/containers/Bundle/Application/CD1C2BBE-1425-4E54-AA21-BA1518B7BFBD/vanilla%20app.app/www/build/polyfills.js:3:10289\nrunTask@file:///var/containers/Bundle/Application/CD1C2BBE-1425-4E54-AA21-BA1518B7BFBD/vanilla%20app.app/www/build/polyfills.js:3:7647\ninvoke@file:///var/containers/Bundle/Application/CD1C2BBE-1425-4E54-AA21-BA1518B7BFBD/vanilla%20app.app/www/build/polyfills.js:3:11404\nabort@[native code]\ncancelTask@file:///var/containers/Bundle/Application/CD1C2BBE-1425-4E54-AA21-BA1518B7BFBD/vanilla%20app.app/www/build/polyfills.js:3:10646\ncancelTask@file:///var/containers/Bundle/Application/CD1C2BBE-1425-4E54-AA21-BA1518B7BFBD/vanilla%20app.app/www/build/polyfills.js:3:8206\nfile:///var/containers/Bundle/Application/CD1C2BBE-1425-4E54-AA21-BA1518B7BFBD/vanilla%20app.app/www/build/polyfills.js:3:3203\nabort\ntimeoutReq@file:///var/containers/Bundle/Application/CD1C2BBE-1425-4E54-AA21-BA1518B7BFBD/vanilla%20app.app/www/build/main.js:172991:14\ninvokeTask@file:///var/containers/Bundle/Application/CD1C2BBE-1425-4E54-AA21-BA1518B7BFBD/vanilla%20app.app/www/build/polyfills.js:3:10289\nrunTask@file:///var/containers/Bundle/Application/CD1C2BBE-1425-4E54-AA21-BA1518B7BFBD/vanilla%20app.app/www/build/polyfills.js:3:7647\ninvoke@file:///var/containers/Bundle/Application/CD1C2BBE-1425-4E54-AA21-BA1518B7BFBD/vanilla%20app.app/www/build/polyfills.js:3:11404\nfile:///var/containers/Bundle/Application/CD1C2BBE-1425-4E54-AA21-BA1518B7BFBD/vanilla%20app.app/www/build/polyfills.js:2:30198","code":"ETIMEDOUT","status":0,"result":{"ok":false,"start_time":"2017-04-10T08:09:58.265Z","docs_read":0,"docs_written":0,"doc_write_failures":0,"errors":[],"status":"aborting","end_time":"2017-04-10T08:12:49.175Z","last_seq":0}}

Before I started using the pouchdb instance in the sync call (when I was using the name of the remote database) the sync did not timeout on my iPhone.

Do you have any thoughts on why this might be happening? The iPhone is connecting to the server ok and successfully logging in with SuperLogin it is just the PouchDB sync that is causing problems (but not when running on the browser when it runs ok).

One thing that may or may not be making a difference is my setup (but again this worked ok when using the remote database name)

· SuperLogin and CouchDB are running on my laptop

· I use ngrok to tunnel back to my laptop from the iPhone

Any help you can give would be hugely appreciated.

Many thanks,

Bill

From: Calvin Metcalf notifications@github.com Reply-To: calvinmetcalf/crypto-pouch reply@reply.github.com Date: Friday, 7 April 2017 at 15:15 To: calvinmetcalf/crypto-pouch crypto-pouch@noreply.github.com Cc: Bill Noble bill.noble@vardas.com, Author author@noreply.github.com Subject: Re: [calvinmetcalf/crypto-pouch] Remote CouchDB is not encrypted (#54)

in both calls to sync you need to use the remote pouchdb instance, not just the name of the remote database,

this._DB.sync(this._remote) you may need to save this._remote in a variable for use in the second call to sync but the gist of it is that since encryption is done locally, you can't refer to the database by name, because then pouchdb will internally create a new instance that won't have encryption associated with it.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

vardas-bill commented 7 years ago

Hi Calvin,

I forgot to add the CouchDB log to my previous email:

[info] 2017-04-10T08:55:34.993290Z couchdb@localhost <0.18233.0> -------- Starting index update for db: shards/80000000-ffffffff/sl-users.1491582997 idx: _design/auth

[info] 2017-04-10T08:55:35.009309Z couchdb@localhost <0.18233.0> -------- Index update finished for db: shards/80000000-ffffffff/sl-users.1491582997 idx: _design/auth

[notice] 2017-04-10T08:55:35.010439Z couchdb@localhost <0.27087.27> e3747ff4ce localhost:5984 127.0.0.1 undefined GET /sl-users/_design/auth/_view/emailUsername?include_docs=true&key=%22a%40a.com%22 200 ok 19

[notice] 2017-04-10T08:55:35.014278Z couchdb@localhost <0.27086.27> b30df4c41a localhost:5984 127.0.0.1 undefined GET /sl-users/a%40a.com? 200 ok 1

[notice] 2017-04-10T08:55:35.047596Z couchdb@localhost <0.27074.27> 4f0fa3bcdb localhost:5984 127.0.0.1 undefined POST /_users/_bulk_docs 201 ok 30

[notice] 2017-04-10T08:55:35.054389Z couchdb@localhost <0.27067.27> 4665acf960 localhost:5984 127.0.0.1 undefined GET /vanilla%24a(40)a(2e)com/ 200 ok 1

[notice] 2017-04-10T08:55:35.054410Z couchdb@localhost <0.27085.27> 9543e4ba02 localhost:5984 127.0.0.1 undefined GET /product/ 200 ok 1

[notice] 2017-04-10T08:55:35.060044Z couchdb@localhost <0.27084.27> c4175a0c37 localhost:5984 127.0.0.1 undefined GET /vanilla%24a(40)a(2e)com/_security? 200 ok 1

[notice] 2017-04-10T08:55:35.060851Z couchdb@localhost <0.27097.27> 90e12ea4ae localhost:5984 127.0.0.1 undefined GET /product/_security? 200 ok 1

[notice] 2017-04-10T08:55:35.141181Z couchdb@localhost <0.27083.27> f3e2b2d819 localhost:5984 127.0.0.1 undefined PUT /vanilla%24a(40)a(2e)com/_security 200 ok 77

[notice] 2017-04-10T08:55:35.150234Z couchdb@localhost <0.27096.27> 508acbb022 localhost:5984 127.0.0.1 undefined PUT /product/_security 200 ok 85

[notice] 2017-04-10T08:55:35.185608Z couchdb@localhost <0.27082.27> 5451a658ae localhost:5984 127.0.0.1 undefined POST /sl-users/_bulk_docs 201 ok 32

[notice] 2017-04-10T08:55:36.272372Z couchdb@localhost <0.27066.27> e464eac4a8 087deddc.ngrok.io 109.144.219.199 aQD-rrhATXC3kbeTcQ6iXA GET /vanilla%24a(40)a(2e)com/ 200 ok 2

Thanks,

Bill

From: Calvin Metcalf notifications@github.com Reply-To: calvinmetcalf/crypto-pouch reply@reply.github.com Date: Friday, 7 April 2017 at 15:15 To: calvinmetcalf/crypto-pouch crypto-pouch@noreply.github.com Cc: Bill Noble bill.noble@vardas.com, Author author@noreply.github.com Subject: Re: [calvinmetcalf/crypto-pouch] Remote CouchDB is not encrypted (#54)

in both calls to sync you need to use the remote pouchdb instance, not just the name of the remote database,

this._DB.sync(this._remote) you may need to save this._remote in a variable for use in the second call to sync but the gist of it is that since encryption is done locally, you can't refer to the database by name, because then pouchdb will internally create a new instance that won't have encryption associated with it.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

calvinmetcalf commented 7 years ago

Safari runs all the encryption in JavaScript in the main thread which is probably what's so slow, if you can out all the database stuff in a web worker that would probably be your best bet

On Mon, Apr 10, 2017, 5:57 AM Bill Noble notifications@github.com wrote:

Hi Calvin,

I forgot to add the CouchDB log to my previous email:

[info] 2017-04-10T08:55:34.993290Z couchdb@localhost <0.18233.0> -------- Starting index update for db: shards/80000000-ffffffff/sl-users.1491582997 idx: _design/auth

[info] 2017-04-10T08:55:35.009309Z couchdb@localhost <0.18233.0> -------- Index update finished for db: shards/80000000-ffffffff/sl-users.1491582997 idx: _design/auth

[notice] 2017-04-10T08:55:35.010439Z couchdb@localhost <0.27087.27> e3747ff4ce localhost:5984 127.0.0.1 undefined GET /sl-users/_design/auth/_view/emailUsername?include_docs=true&key=%22a% 40a.com%22 200 ok 19

[notice] 2017-04-10T08:55:35.014278Z couchdb@localhost <0.27086.27> b30df4c41a localhost:5984 127.0.0.1 undefined GET /sl-users/a%40a.com? 200 ok 1

[notice] 2017-04-10T08:55:35.047596Z couchdb@localhost <0.27074.27> 4f0fa3bcdb localhost:5984 127.0.0.1 undefined POST /_users/_bulk_docs 201 ok 30

[notice] 2017-04-10T08:55:35.054389Z couchdb@localhost <0.27067.27> 4665acf960 localhost:5984 127.0.0.1 undefined GET /vanilla%24a(40)a(2e)com/ 200 ok 1

[notice] 2017-04-10T08:55:35.054410Z couchdb@localhost <0.27085.27> 9543e4ba02 localhost:5984 127.0.0.1 undefined GET /product/ 200 ok 1

[notice] 2017-04-10T08:55:35.060044Z couchdb@localhost <0.27084.27> c4175a0c37 localhost:5984 127.0.0.1 undefined GET /vanilla%24a(40)a(2e)com/_security? 200 ok 1

[notice] 2017-04-10T08:55:35.060851Z couchdb@localhost <0.27097.27> 90e12ea4ae localhost:5984 127.0.0.1 undefined GET /product/_security? 200 ok 1

[notice] 2017-04-10T08:55:35.141181Z couchdb@localhost <0.27083.27> f3e2b2d819 localhost:5984 127.0.0.1 undefined PUT /vanilla%24a(40)a(2e)com/_security 200 ok 77

[notice] 2017-04-10T08:55:35.150234Z couchdb@localhost <0.27096.27> 508acbb022 localhost:5984 127.0.0.1 undefined PUT /product/_security 200 ok 85

[notice] 2017-04-10T08:55:35.185608Z couchdb@localhost <0.27082.27> 5451a658ae localhost:5984 127.0.0.1 undefined POST /sl-users/_bulk_docs 201 ok 32

[notice] 2017-04-10T08:55:36.272372Z couchdb@localhost <0.27066.27> e464eac4a8 087deddc.ngrok.io 109.144.219.199 aQD-rrhATXC3kbeTcQ6iXA GET /vanilla%24a(40)a(2e)com/ 200 ok 2

Thanks,

Bill

From: Calvin Metcalf notifications@github.com Reply-To: calvinmetcalf/crypto-pouch reply@reply.github.com Date: Friday, 7 April 2017 at 15:15 To: calvinmetcalf/crypto-pouch crypto-pouch@noreply.github.com Cc: Bill Noble bill.noble@vardas.com, Author author@noreply.github.com Subject: Re: [calvinmetcalf/crypto-pouch] Remote CouchDB is not encrypted (#54)

in both calls to sync you need to use the remote pouchdb instance, not just the name of the remote database,

this._DB.sync(this._remote) you may need to save this._remote in a variable for use in the second call to sync but the gist of it is that since encryption is done locally, you can't refer to the database by name, because then pouchdb will internally create a new instance that won't have encryption associated with it.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you modified the open/close state.

Reply to this email directly, view it on GitHub https://github.com/calvinmetcalf/crypto-pouch/issues/54#issuecomment-292903962, or mute the thread https://github.com/notifications/unsubscribe-auth/ABE4n6VAII95DcyBJKArkXE_Wkh-7Jx7ks5ruf0cgaJpZM4M0w0i .

vardas-bill commented 7 years ago

Many thanks for the speedy response Calvin!

For my test case I only have two small documents that I need to sync and the current timeout is @10mins which seems rather long for what is being encrypted/decrypted?

My App needs to wait for the PouchDB to finish syncing before it can proceed as it needs to show the data. If I use a web worker will this significantly speed things up? I am wondering why this takes seconds when running the App in a web browser and over 10 mins, and timing out, when running on an iPhone?

Bill

From: Calvin Metcalf notifications@github.com Reply-To: calvinmetcalf/crypto-pouch reply@reply.github.com Date: Monday, 10 April 2017 at 11:40 To: calvinmetcalf/crypto-pouch crypto-pouch@noreply.github.com Cc: Bill Noble bill.noble@vardas.com, Author author@noreply.github.com Subject: Re: [calvinmetcalf/crypto-pouch] Remote CouchDB is not encrypted (#54)

Safari runs all the encryption in JavaScript in the main thread which is probably what's so slow, if you can out all the database stuff in a web worker that would probably be your best bet

On Mon, Apr 10, 2017, 5:57 AM Bill Noble notifications@github.com wrote:

Hi Calvin,

I forgot to add the CouchDB log to my previous email:

[info] 2017-04-10T08:55:34.993290Z couchdb@localhost <0.18233.0> -------- Starting index update for db: shards/80000000-ffffffff/sl-users.1491582997 idx: _design/auth

[info] 2017-04-10T08:55:35.009309Z couchdb@localhost <0.18233.0> -------- Index update finished for db: shards/80000000-ffffffff/sl-users.1491582997 idx: _design/auth

[notice] 2017-04-10T08:55:35.010439Z couchdb@localhost <0.27087.27> e3747ff4ce localhost:5984 127.0.0.1 undefined GET /sl-users/_design/auth/_view/emailUsername?include_docs=true&key=%22a% 40a.com%22 200 ok 19

[notice] 2017-04-10T08:55:35.014278Z couchdb@localhost <0.27086.27> b30df4c41a localhost:5984 127.0.0.1 undefined GET /sl-users/a%40a.com? 200 ok 1

[notice] 2017-04-10T08:55:35.047596Z couchdb@localhost <0.27074.27> 4f0fa3bcdb localhost:5984 127.0.0.1 undefined POST /_users/_bulk_docs 201 ok 30

[notice] 2017-04-10T08:55:35.054389Z couchdb@localhost <0.27067.27> 4665acf960 localhost:5984 127.0.0.1 undefined GET /vanilla%24a(40)a(2e)com/ 200 ok 1

[notice] 2017-04-10T08:55:35.054410Z couchdb@localhost <0.27085.27> 9543e4ba02 localhost:5984 127.0.0.1 undefined GET /product/ 200 ok 1

[notice] 2017-04-10T08:55:35.060044Z couchdb@localhost <0.27084.27> c4175a0c37 localhost:5984 127.0.0.1 undefined GET /vanilla%24a(40)a(2e)com/_security? 200 ok 1

[notice] 2017-04-10T08:55:35.060851Z couchdb@localhost <0.27097.27> 90e12ea4ae localhost:5984 127.0.0.1 undefined GET /product/_security? 200 ok 1

[notice] 2017-04-10T08:55:35.141181Z couchdb@localhost <0.27083.27> f3e2b2d819 localhost:5984 127.0.0.1 undefined PUT /vanilla%24a(40)a(2e)com/_security 200 ok 77

[notice] 2017-04-10T08:55:35.150234Z couchdb@localhost <0.27096.27> 508acbb022 localhost:5984 127.0.0.1 undefined PUT /product/_security 200 ok 85

[notice] 2017-04-10T08:55:35.185608Z couchdb@localhost <0.27082.27> 5451a658ae localhost:5984 127.0.0.1 undefined POST /sl-users/_bulk_docs 201 ok 32

[notice] 2017-04-10T08:55:36.272372Z couchdb@localhost <0.27066.27> e464eac4a8 087deddc.ngrok.io 109.144.219.199 aQD-rrhATXC3kbeTcQ6iXA GET /vanilla%24a(40)a(2e)com/ 200 ok 2

Thanks,

Bill

From: Calvin Metcalf notifications@github.com Reply-To: calvinmetcalf/crypto-pouch reply@reply.github.com Date: Friday, 7 April 2017 at 15:15 To: calvinmetcalf/crypto-pouch crypto-pouch@noreply.github.com Cc: Bill Noble bill.noble@vardas.com, Author author@noreply.github.com Subject: Re: [calvinmetcalf/crypto-pouch] Remote CouchDB is not encrypted (#54)

in both calls to sync you need to use the remote pouchdb instance, not just the name of the remote database,

this._DB.sync(this._remote) you may need to save this._remote in a variable for use in the second call to sync but the gist of it is that since encryption is done locally, you can't refer to the database by name, because then pouchdb will internally create a new instance that won't have encryption associated with it.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you modified the open/close state.

Reply to this email directly, view it on GitHub https://github.com/calvinmetcalf/crypto-pouch/issues/54#issuecomment-292903962, or mute the thread https://github.com/notifications/unsubscribe-auth/ABE4n6VAII95DcyBJKArkXE_Wkh-7Jx7ks5ruf0cgaJpZM4M0w0i .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

calvinmetcalf commented 7 years ago

you should check for errors on the outer sync, or try to read a document, it could be something is wrong there, otherwise it could be similar to #53 where it takes a long time to run the key derivation (this is where it turns the human readable password into the machine readable encryption key, this intentionally takes some time in order to prevent brute force hacking the key)

vardas-bill commented 7 years ago

Thanks Calvin.

I am still not sure why it is just seconds when running the App on a browser and timesout at 10 minutes when running on an iPhone. It is on the outer sync that the timeout occurs.

From: Calvin Metcalf notifications@github.com Reply-To: calvinmetcalf/crypto-pouch reply@reply.github.com Date: Monday, 10 April 2017 at 13:36 To: calvinmetcalf/crypto-pouch crypto-pouch@noreply.github.com Cc: Bill Noble bill.noble@vardas.com, Author author@noreply.github.com Subject: Re: [calvinmetcalf/crypto-pouch] Remote CouchDB is not encrypted (#54)

you should check for errors on the outer sync, or try to read a document, it could be something is wrong there, otherwise it could be similar to #53 where it takes a long time to run the key derivation (this is where it turns the human readable password into the machine readable encryption key, this intentionally takes some time in order to prevent brute force hacking the key)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

calvinmetcalf commented 7 years ago

in chrome and firefox there are built in encryption functions that are very efficient, in safari, both desktop and mobile there are not so any encryption is done in javascript which is much less efficient, in addition, your desktop computer is probably more powerful then your phone. Lastly there is a chance that it isn't actually taking 10 minutes but that there is an error and it's not taking ten minutes, it's just errored out and will never finish, you just don't notice until it times out

On Mon, Apr 10, 2017 at 8:42 AM Bill Noble notifications@github.com wrote:

Thanks Calvin.

I am still not sure why it is just seconds when running the App on a browser and timesout at 10 minutes when running on an iPhone. It is on the outer sync that the timeout occurs.

From: Calvin Metcalf notifications@github.com Reply-To: calvinmetcalf/crypto-pouch reply@reply.github.com Date: Monday, 10 April 2017 at 13:36 To: calvinmetcalf/crypto-pouch crypto-pouch@noreply.github.com Cc: Bill Noble bill.noble@vardas.com, Author author@noreply.github.com Subject: Re: [calvinmetcalf/crypto-pouch] Remote CouchDB is not encrypted (#54)

you should check for errors on the outer sync, or try to read a document, it could be something is wrong there, otherwise it could be similar to #53 where it takes a long time to run the key derivation (this is where it turns the human readable password into the machine readable encryption key, this intentionally takes some time in order to prevent brute force hacking the key)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you modified the open/close state.

Reply to this email directly, view it on GitHub https://github.com/calvinmetcalf/crypto-pouch/issues/54#issuecomment-292937834, or mute the thread https://github.com/notifications/unsubscribe-auth/ABE4nxTNxSNFXm_63aVgeGXKCOYHaXCoks5ruiOkgaJpZM4M0w0i .

vardas-bill commented 7 years ago

That makes perfect sense Calvin thanks. I think for my needs I need to use a faster encryption option. I really appreciate you taking the time to help with this though thanks.

Cheers,

Bill

From: Calvin Metcalf notifications@github.com Reply-To: calvinmetcalf/crypto-pouch reply@reply.github.com Date: Monday, 10 April 2017 at 13:53 To: calvinmetcalf/crypto-pouch crypto-pouch@noreply.github.com Cc: Bill Noble bill.noble@vardas.com, Author author@noreply.github.com Subject: Re: [calvinmetcalf/crypto-pouch] Remote CouchDB is not encrypted (#54)

in chrome and firefox there are built in encryption functions that are very efficient, in safari, both desktop and mobile there are not so any encryption is done in javascript which is much less efficient, in addition, your desktop computer is probably more powerful then your phone. Lastly there is a chance that it isn't actually taking 10 minutes but that there is an error and it's not taking ten minutes, it's just errored out and will never finish, you just don't notice until it times out

On Mon, Apr 10, 2017 at 8:42 AM Bill Noble notifications@github.com wrote:

Thanks Calvin.

I am still not sure why it is just seconds when running the App on a browser and timesout at 10 minutes when running on an iPhone. It is on the outer sync that the timeout occurs.

From: Calvin Metcalf notifications@github.com Reply-To: calvinmetcalf/crypto-pouch reply@reply.github.com Date: Monday, 10 April 2017 at 13:36 To: calvinmetcalf/crypto-pouch crypto-pouch@noreply.github.com Cc: Bill Noble bill.noble@vardas.com, Author author@noreply.github.com Subject: Re: [calvinmetcalf/crypto-pouch] Remote CouchDB is not encrypted (#54)

you should check for errors on the outer sync, or try to read a document, it could be something is wrong there, otherwise it could be similar to #53 where it takes a long time to run the key derivation (this is where it turns the human readable password into the machine readable encryption key, this intentionally takes some time in order to prevent brute force hacking the key)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you modified the open/close state.

Reply to this email directly, view it on GitHub https://github.com/calvinmetcalf/crypto-pouch/issues/54#issuecomment-292937834, or mute the thread https://github.com/notifications/unsubscribe-auth/ABE4nxTNxSNFXm_63aVgeGXKCOYHaXCoks5ruiOkgaJpZM4M0w0i .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

calvinmetcalf commented 7 years ago

so a couple things you can try:

deriving the keys ahead of time like mentioned #53
pass the iterations option to reduce how long it takes to derive the keys, the default is 100000 but you can change that to something smaller, can also be used to test to make sure that's actually whats slow, if you change it to 5 and it's still slow then that's probably not the issue.
you can pass the algorithm option set to chacha20, this will use a different encryption scheme that should be faster then the default when they both run in JavaScript (though it's slower then the default when the default runs natively).

vardas-bill commented 7 years ago

Thanks Calvin I will give those a try.

From: Calvin Metcalf notifications@github.com Reply-To: calvinmetcalf/crypto-pouch reply@reply.github.com Date: Monday, 10 April 2017 at 14:06 To: calvinmetcalf/crypto-pouch crypto-pouch@noreply.github.com Cc: Bill Noble bill.noble@vardas.com, Author author@noreply.github.com Subject: Re: [calvinmetcalf/crypto-pouch] Remote CouchDB is not encrypted (#54)

so a couple things you can try: deriving the keys ahead of time like mentioned #53 pass the iterations option to reduce how long it takes to derive the keys, the default is 100000 but you can change that to something smaller, can also be used to test to make sure that's actually whats slow, if you change it to 5 and it's still slow then that's probably not the issue. you can pass the algorithm option set to chacha20, this will use a different encryption scheme that should be faster then the default when they both run in JavaScript (though it's slower then the default when the default runs natively). —You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

calvinmetcalf / crypto-pouch

Remote CouchDB is not encrypted #54