[dexie-encrypted-addon]: Collection.filter is using encrypted data

[knikolov-nuvolo]

Hello, and thanks for the great dexie addons. While using the encryption addon I found that when I try to filter by encrypted column, the data is not derypted. It gets decrypted after I call toArray or sortBy.

Here is an example: https://codesandbox.io/s/smoosh-frost-x6fhzf?file=/src/dbActions.ts:314-483 Click INSERT, and then SELECT - it should render 2 records, but it does not. Check the console, the name and shoeSize are encoded.

db.friends
    .filter((f) => {
      console.log(f); // name and shoeSize are encoded
      return !!f.shoeSize && f.shoeSize > 41;
    })
    .sortBy("name");

I can filter the data after sortBy, but performance wise it is better to first apply the filters and then sort the data. Is there a workaround? Am I missing something?

I tried dexie-encrypted, and it decodes the data earlier, so inside filter() the data is already decoded. Unfortunately, dexie-encrypted has other issues and it is no longer maintained.

Our use case requires filtering and sorting large amounts of encrypted data, so such a feature would be great. If code changes are required, I am willing to help, but will need assistance.

[PVermeer]

Hi there,

It is correct that Table.filter() does not decrypt the data. Methods called on Table that return a collection do not call the installed decryption hook since this can be very expensive. These methods are mostly optimized by Dexie for querying IndexedDB.

As [Table.filter()](https://dexie.org/docs/Table/Table.filter()) docs states, this method returns the whole store and calls this method on every entry before calling a read method (e.g. toArray()) and thus reduces work for the reading hooks. dexie-encrypted Uses a newer method of installing hooks so maybe that's why it's working there. Maybe Dexie has optimized this? I might look into this when I have the time.

Your query could be rewritten as:

(await db.friends.toArray())
    .filter((f) => {
      console.log(f);
      return !!f.shoeSize && f.shoeSize > 41;
    })
    .sort((a,b) => (a.name > b.name) ? 1 : ((b.name > a.name) ? -1 : 0)); // Or use other sorting algorithm (e.g. lodash);

Calling a read method (e.g. toArray()) decrypts the data first. This would just as efficient as your query (minus the decrypting part).

You could also use each(). This is probably a lot slower but more memory efficient for large stores.

Sidenote: When using an encryption in databases you have to model your data around this and accept some drawbacks. In this case you always have to query the whole store. IndexedDB queries work by ordering data and query for ranges and Dexie does a great job of simplifying this!