upgrade function copying values from different DB (idb-keyval)

[pantchox]

Hello, After using idb-keyval package I wanted to have multiple stores so I started using this library.

Now in the upgrade function when opening the DB I would like to open the store created by the idb-keyval which is keyval and copy all the keys and values to my new database and store created by idb package.

here is my code

async function openSettingsDB() {
        return await openDB('settings', 1, {
            async upgrade(db, oldVersion, newVersion, transaction) {
                db.createObjectStore('channels');
                if (oldVersion === 0) {
                    // need to migrate channel data from old keyval
                    const oldKeyValDB = await openDB('keyval-store');
                    const oldKeys = await oldKeyValDB.getAllKeys('keyval');
                    const oldValues = [];
                    oldKeys.forEach(async (key) => {
                        // of course here will be done some filtering since I need specific keys and not all
                        // in order not to get each key value loaded into memory, this is just an example
                        const keyVal = await oldKeyValDB.get('keyval', key);
                        oldValues.push(keyVal);
                    });

                     // since we use transaction and we follow this guideline:
                    // https://github.com/jakearchibald/idb/blob/master/README.md#transaction-lifetime
                    oldKeys.forEach((key, idx) => {
                        transaction.objectStore('channels').add(oldValues[idx], key);
                    });
                } else {
                    console.log('no upgrade');
                }
            },
        });
    }

Running this causes an error

wrap-idb-value.js:150 Uncaught (in promise) DOMException: Failed to execute 'objectStore' on 'IDBTransaction': The transaction has finished.

When calling transacation.objectStore('channels').add

What am I missing? thanks!

[jakearchibald]

You're running into this issue https://github.com/jakearchibald/idb#transaction-lifetime.

Copying from one database to another cannot be done in a single transaction as transactions must involve a single database.

Here's how I'd do what you're aiming for:

async function openSettingsDB() {
  let needsKeyvalTransfer = false;

  const db = await openDB('settings', 1, {
    async upgrade(db, oldVersion, newVersion, transaction) {
      if (oldVersion < 1) {
        db.createObjectStore('channels');
        needsKeyvalTransfer = true;
      }
    },
  });

  if (needsKeyvalTransfer) {
    const keysToTransfer = ['foo', 'bar', 123];
    const oldDb = await openDB('keyval-store');
    const oldStore = db.transaction('keyval').store;
    // Get values from the old DB
    const toTransfer = new Map(
      await Promise.all(
        keysToTransfer.map(async (key) => {
          return [key, await oldStore.get(key)];
        })
      )
    );
    // Add the values to the new DB in a single transaction
    const tx = db.transaction('channels', 'readwrite');
    for (const [key, val] of toTransfer) tx.store.put(value, key);
    await tx.done;
  }

  return db;
}

[pantchox]

Thanks for the quick reply! But in this part

  if (needsKeyvalTransfer) {
    const keysToTransfer = ['foo', 'bar', 123];
...

I don't know the keys from advanced and I need to query it. so if I apply keysToTransfer this:

const oldDb = await openDB('keyval-store');
const oldStore = db.transaction('keyval').store;
const keysToTransfer = await oldStore.getAllKeys();

keysToTransfer returns empty array.

to test it is not empty of course I looked the the "application" tab on the dev tools and also I was able to query like on my original code to see it does have values (even though the error happens after as in my original code).

any idea why this happen?

[jakearchibald]

I can't reproduce the issue. https://static-misc-3.glitch.me/idb-keyval-test/copy-keys.html

I suspect it's because you're written db.transaction('keyval') where db refers to the new database, rather than oldDb.

If you run into more issues, please create a reduced reproduction of the issue for me to debug, so I don't need to create it myself.

[pantchox]

That was a typo (copied originally from your first solution :) ) I did refer to `oldDb' in my code. eventually what I did this:

          const oldDb = await openDB('keyval-store');
          const keysToTransfer = await oldDb.getAllKeys('keyval');

          // start a transaction part
          const oldStore = oldDb.transaction('keyval').store;
          const toTransfer = new Map(
                await Promise.all(
                    keysToTransfer.map(async (key) => {
                        return [key, await oldStore.get(key)];
                    })
                )
            );

it has some kind of "duplication" in getting the oldDb old store, but it works and the DB is few lines so it works fine.

thanks for your support - much appreciated!