search

vrtmrz / obsidian-livesync

MIT License
5.16k stars 168 forks source link

Inconsistent encryption #432

Closed NekoiNemo closed 4 months ago

NekoiNemo commented 6 months ago

Abstract

With End-to-End Encryption set up, some chunks of data in the database are still unencrypted.

Expected behaviour

All "leafs" should be fully encrypted

Actually happened

It seems that only some of the leaf nodes got encrypted, and the rest have plain text in the data field:

Report from the LiveSync

Report from hatch ``` https://gist.github.com/NekoiNemo/f9f8794d9cb06b821d74a8a56bfaf4bf ```

Plug-in log

We can see the log by tapping the Document box icon. If you noticed something suspicious, please let me know. Note: Please enable Verbose Log. For detail, refer to Logging, please.

Plug-in log ``` Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:39->Replication closed Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:39->Cache initialized 300 / 50000000 Nemo-978cbb4a3ad7488e:23/05/2024,` 00:49:39->Scanning customizing files. Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:39->STORAGE --> DB:ix:izumi/CONFIG/app.json.md: (config) Skipped (Same content) Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:39->STORAGE --> DB:ix:izumi/CONFIG/appearance.json.md: (config) Skipped (Same content) Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:39->STORAGE --> DB:ix:izumi/CONFIG/community-plugins.json.md: (config) Skipped (Same content) Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:39->STORAGE --> DB:ix:izumi/CONFIG/core-plugins-migration.json.md: (config) Skipped (Same content) Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:39->STORAGE --> DB:ix:izumi/CONFIG/core-plugins.json.md: (config) Skipped (Same content) Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:39->STORAGE --> DB:ix:izumi/CONFIG/types.json.md: (config) Skipped (Same content) Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:39->STORAGE --> DB:ix:izumi/PLUGIN_DATA/calendar.md: (config) Skipped (Same content) Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:40->STORAGE --> DB:ix:izumi/PLUGIN_DATA/obsidian-excalidraw-plugin.md: (config) Skipped (Same content) Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:40->Chunks saved: doc: izumi/PLUGIN_DATA/obsidian-livesync.md ,chunks: 5 (new:3, recycled:0, cached:2) Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:40->STORAGE --> DB:ix:izumi/PLUGIN_DATA/obsidian-livesync.md: (config) Done Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:40->STORAGE --> DB:ix:izumi/PLUGIN_MAIN/obsidian-livesync.md: (config) Skipped (Same content) Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:40->STORAGE --> DB:ix:izumi/PLUGIN_DATA/obsidian-tasks-plugin.md: (config) Skipped (Same content) Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:40->STORAGE --> DB:ix:izumi/PLUGIN_DATA/periodic-notes.md: (config) Skipped (Same content) Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:40->STORAGE --> DB:ix:izumi/PLUGIN_DATA/table-editor-obsidian.md: (config) Skipped (Same content) Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:40->Scanning customizing files. Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:40->Scanning hidden files. Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:41->STORAGE --> DB:ix:izumi/CONFIG/app.json.md: (config) Skipped (Same content) Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:41->STORAGE --> DB:ix:izumi/CONFIG/appearance.json.md: (config) Skipped (Same content) Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:41->STORAGE --> DB:ix:izumi/CONFIG/community-plugins.json.md: (config) Skipped (Same content) Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:41->STORAGE --> DB:ix:izumi/CONFIG/core-plugins-migration.json.md: (config) Skipped (Same content) Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:41->STORAGE --> DB:ix:izumi/CONFIG/core-plugins.json.md: (config) Skipped (Same content) Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:41->STORAGE --> DB:ix:izumi/CONFIG/types.json.md: (config) Skipped (Same content) Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:41->Chunks saved: doc: izumi/CONFIG/workspace.json.md ,chunks: 5 (new:3, recycled:0, cached:2) Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:41->STORAGE --> DB:ix:izumi/CONFIG/workspace.json.md: (config) Done Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:41->STORAGE --> DB:ix:izumi/PLUGIN_DATA/calendar.md: (config) Skipped (Same content) Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:41->STORAGE --> DB:ix:izumi/PLUGIN_DATA/obsidian-excalidraw-plugin.md: (config) Skipped (Same content) Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:42->STORAGE --> DB:ix:izumi/PLUGIN_MAIN/obsidian-livesync.md: (config) Skipped (Same content) Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:42->STORAGE --> DB:ix:izumi/PLUGIN_DATA/obsidian-tasks-plugin.md: (config) Skipped (Same content) Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:42->STORAGE --> DB:ix:izumi/PLUGIN_DATA/periodic-notes.md: (config) Skipped (Same content) Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:42->STORAGE --> DB:ix:izumi/PLUGIN_DATA/table-editor-obsidian.md: (config) Skipped (Same content) Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:44->Hidden files scanned: 0 files had been modified Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:44->Before LiveSync, start OneShot once... Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:44->OneShot Sync begin... (pullOnly) Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:45->Replication paused Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:45->Replication completed Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:45->LiveSync begin... Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:45->Replication activated Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:46->Replication paused Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:57->Chunks saved: doc: Obsidian.md ,chunks: 76 (new:2, recycled:0, cached:74) Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:57->Replication activated Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:57->STORAGE -> DB (plain) Obsidian.md Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:57->Sync on Editor Save. Nemo-978cbb4a3ad7488e:23/05/2024, 00:49:58->Replication paused Nemo-978cbb4a3ad7488e:23/05/2024, 00:50:44->Scanning hidden files. Nemo-978cbb4a3ad7488e:23/05/2024, 00:50:47->Hidden files scanned: 0 files had been modified ```

Screenshots

screenshot_1716413869

This is me randomly querying a note in the database, then looking up it's "children".

The corresponding part of the note in question:

# Obsidian

## Syntax

### Comments

Comments can be made by wrapping a block of text in `%%` on each side

```markdown
%%
This is a comment
%%

A comment can be %%this is also a comment%% inline too

%% This is a comment %%

A comment can be %%this is also a comment%% inline too



### Other information

This vault has been created in the desktop app, then livesync plugin was installed and configured, then a mobile client (with empty vault) got added through sharing the URI
vrtmrz commented 5 months ago

Sorry for being late! The leading h:+ and h: in the chunk ID mean encrypted and unencrypted respectively. Therefore, some chunks seem to be made plain from the beginning.

However, following the note (Thank you so much for filling in), all your devices have enabled encryption.

One possibility is the chunk has been reused from the cache. If the content-matched cache is found, LiveSync will reuse it. To reduce the amount of traffic for differences. I thought it was not wrong behaviour, however, it is not ideal in this situation.

However, we cannot delete the chunks that have been created once. Would you please try Rebuild everything on your most trusted device? Obsidian will be restarted and begin rebuilding the database with all caches discarded.

NekoiNemo commented 5 months ago

Thank you, this seems to have fixed the issue and now i can't find any chunks that aren't encrypted!

vrtmrz commented 4 months ago

Thank you for your confirmation! I am so relieved! May a safer environment keep us free!