Saving the encrypted DB inside CouchDB?

[rwjack]

Hey @vrtmrz, first off, what an awesome project!

I feel like there's a single step from achieving total functionality that would make this plugin amazing.

Is there a way of saving the DB inside the docker container, and encrypting it? I've tried adding the following lines in my compose file, but couchdb doesn't seem to like it and won't start.

volumes:
  db:
... SNIP ...
  volumes:
    - db:/var/lib/couchdb

That way we could have live sync on all enrolled devices, while the DB (vault) would be safe and sound, encrypted on a remote server.

From my understanding the docker container currently doesn't save anything to disk, and is basically just used for it's processing power to help obsidian clients talk to each other.

[vactomas]

Hi, I am not the dev and do not know of any way to encrypt the Docker volume, but I wanted to ask – Wouldn't End 2 End Encryption (E2EE) fix the problem for you? That way, your notes should be only decrypted on your devices with Obsidian.

[rwjack]

Thanks @vactomas, Perhaps I didn't explain what I wanted properly. I did see that E2EE is available, but I'm not sure how to encrypt the vault I'm using and keep it at a centralized location.

From my understanding if I simply enable E2EE, I have absolutely no idea what happens, does the vault that's already on my SMB server get encrypted, or does only the local copy get encrypted or is there a third chain of events?

[vrtmrz]

@rwjack Thank you for asking me. (and, thanks @vactomas)

The mounting volume is like this: https://github.com/vrtmrz/self-hosted-livesync-server/blob/dbf28eaffc07592d742be2a1fc00891ee9cdf65c/docker-compose.yml#L33

As @vactomas says, If you enable E2EE, the plugin sends all notes with encrypting and receives with decrypting automatically. All your note's bodies in the remote server should be encrypted with AES-GCM.

That way we could have live sync on all enrolled devices, while the DB (vault) would be safe and sound, encrypted on a remote server.

This is not enough. If we have to keep it secure, we should not send any data without encryption. So, we have to encrypt within our device or while transferring.

From my understanding the docker container currently doesn't save anything to disk, and is basically just used for it's processing power to help obsidian clients talk to each other.

If you like, you can use this.

Does this answer the question?

[rwjack]

As @vactomas says, If you enable E2EE, the plugin sends all notes with encrypting and receives with decrypting automatically. All your note's bodies in the remote server should be encrypted with AES-GCM.

So I'm supposed to encrypt my notes manually? Did I get this right?

For now I just activated E2EE on my desktop, added a password and hit apply & send. On my phone I did the same, but instead of apply & send, I did just apply.

I still see the notes in plaintext on my SMB server, which is also hosting the couchDB instance.

If you like, you can use this.

I already have a reverse proxy, SSL and everything set up. I'm just trying to understand the functionality of the plugin.

Have you heard of joplin? I used it before and it has the same functionality I'm trying to describe. All notes stay on a centralized location, other devices can point at the URL of the server and use a decryption passphrase to decrypt the notes locally.

[vactomas]

Joplin indeed uses E2EE. Basically, E2EE encrypts the notes on your devices, and they stay encrypted until another client (Obsidian instance with plugin) with a password decrypts them. They are encrypted during transport and when they are stored.

This is at least how E2EE works. I am not sure how it is implemented here, as I currently am unable to test it due to issues with my CouchDB instance.

[rwjack]

I don't know why, but live sync seems to be broken.

I have tried doing full resets, without luck. I have tried resetting my phone and desktop vault, and deleting the database docker volume for a fresh start.

Then setup on desktop and note files into the db and sync. Setup on phone, some files don't sync. Live sync works on neither device.

[vactomas]

I think I read somewhere, that due to new API, it has problems with mobile and Live sync. People have suggested using sync on save instead.

[rwjack]

Thanks @vactomas, that fixed some issues.

I completely disabled saving the DB in docker, but now I'm having problems on my laptop. Inspector says it's an invalid CA. Obviously because it's my internal self-signed CA, which is already in my debian based system ca-certificate list. Quite strange that this works on both my phone and desktop, but not the laptop. It would really be easier if we had an option to ignore self signed CA errors.

[vactomas]

Do you have it hosted on a domain? If so, use Let's encrypt to get a free certificate. I believe, it is mentioned in docs, that self-signed certificates might pose issues.

[rwjack]

Managed to get that solved by re-installing mkcert.

It's an internal domain, I don't need access from the outside for this.

Still no luck on the encryption + live sync + persistent docker DB

[vactomas]

What is your exact issue with Docker? I do have an issue as well, but it is probably not related.

[rwjack]

As I said, I want to have the notes persist inside couchDB, so any device can simply login and pull the notes. For now, if the container is restated, nothing is saved, and the notes stay only on the devices that have already pulled them.

Syncing between 3 devices still presents a lot of issues. It simply does not work the way it should.

If I make a change on my phone and press replicate. It should send the latest version of the file to the server. If I press replicate on one of my other devices, they should pull the latest version. Simply nothing happens

[vactomas]

That is weird. Could you post here the whole docker-compose file, with sensitive information, such as DB password, censored?

[rwjack]

version: "3"

#volumes:
  #  db:

services:

  couchdb:
    image: couchdb:latest
    restart: unless-stopped

    container_name: obsidian-couchdb
    hostname: obsidian-couchdb

    env_file:
      - .env

    ports:
      - "x:5984"

    volumes:
      - ./local.ini:/opt/couchdb/etc/local.ini
      #- db:/opt/couchdb/

.env:

COUCHDB_USER=x
COUCHDB_PASSWORD=x
COUCHDB_SERVER=internal.doma.in

[vactomas]

This is how I have it defined.

db:/opt/couchdb/data

It is based on this file

[vrtmrz]

@rwjack

So I'm supposed to encrypt my notes manually?

No, all process will be done automatically. Your files will be stored in plain text on your device storage as same as it has been so. But, it is encrypted in the server.

You can see how are they stored by CouchDB’s admin tool. Open the URI which inserted _utils/#database/ before the DB name (i.e. example.net/obsidiandb -> example.net/_utils/#database/obsidiandb/)

Have you heard of joplin?

Yes! I had been one of the users. I guessed that you might wonder if your files are still plain text. It is right. But everything gets done till sent to the remote server.

A self-signed certificate will be ignored even though the device trusted it. especially in mobile, we can’t solve that.

[vrtmrz]

@rwjack Could you check permissions and owners of /opt/couchdb/ and its children? CouchDB uses a user that has id:5986 even if it is inside the container. Perhaps we have to fix this mismatch.

[vrtmrz]

Close this issue once. Feel free to open it again if you have new something.