Executing transactions in a for loop only executes one transaction and finishes

[rostgoat]

I am trying to overload indexDB by inserting lots of 64 base images in a loop. However, the transaction in my loop executes only once. How can this be done?

const initIDB = async () => {
  const db = await openDB('db', 1, {
    // eslint-disable-next-line no-shadow
    upgrade(db) {
      db.createObjectStore('tempStore', { keyPath: 'id', autoIncrement: true });
    },
  });
  const tx = db.transaction('tempStore', 'readwrite');
  await overloadIDB(tx.store);
  await tx.done;

  return true;
};

const getRandomArbitrary = (min, max) => Math.random() * (max - min) + min;

const overloadIDB = async (store) => {
  const imgurl = "bla.png";
  const img64 = await toDataURL(imgurl);
  for (let i = 0; i < 10; i++) {
    if (i > 0 && i % 100 === 0) console.log('A set done');
    try {
      const num = Math.round(getRandomArbitrary(1, 1000000));
      const data = {
        id: num,
        img: img64,
      };
      await new Promise(resolve => resolve(store.add(data)));
    } catch (e) {
      console.log(e.toString());
      console.dir(e);
      break;
    }
  }
};

const toDataURL = url => new Promise((resolve, reject) => {
  const xhr = new XMLHttpRequest();
  xhr.open('GET', url);
  xhr.responseType = 'blob';
  xhr.onload = () => {
    const reader = new FileReader();
    reader.onloadend = () => {
      resolve((reader.result));
    };
    reader.readAsDataURL(xhr.response);
  };
  xhr.send();
});

Keep getting this error

Failed to execute 'add' on 'IDBObjectStore': The transaction has finished

[jakearchibald]

https://github.com/jakearchibald/idb/blob/master/README.md#transaction-lifetime

On Mon, 28 Oct 2019, 20:50 Rost Myshkin, notifications@github.com wrote:

I am trying to overload indexDB by inserting lots of 64 base images in a loop. However, the transaction in my loop executes only once. How can this be done?

const initIDB = async () => { const db = await openDB('db', 1, { // eslint-disable-next-line no-shadow upgrade(db) { db.createObjectStore('tempStore', { keyPath: 'id', autoIncrement: true }); }, }); const tx = db.transaction('tempStore', 'readwrite'); await overloadIDB(tx.store); await tx.done;

return true; };

const getRandomArbitrary = (min, max) => Math.random() * (max - min) + min;

const overloadIDB = async (store) => { const imgurl = "bla.png"; const img64 = await toDataURL(imgurl); for (let i = 0; i < 10; i++) { if (i > 0 && i % 100 === 0) console.log('A set done'); try { const num = Math.round(getRandomArbitrary(1, 1000000)); const data = { id: num, img: img64, }; await new Promise(resolve => resolve(store.add(data))); } catch (e) { console.log(e.toString()); console.dir(e); break; } } };

const toDataURL = url => new Promise((resolve, reject) => { const xhr = new XMLHttpRequest(); xhr.open('GET', url); xhr.responseType = 'blob'; xhr.onload = () => { const reader = new FileReader(); reader.onloadend = () => { resolve((reader.result)); }; reader.readAsDataURL(xhr.response); }; xhr.send(); });

Keep getting this error

Failed to execute 'add' on 'IDBObjectStore': The transaction has finished

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/jakearchibald/idb/issues/135?email_source=notifications&email_token=AAAW3GSLBWHGVC2VCPKDUCTQQ5GADA5CNFSM4JF7ZNPKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HU4E7TA, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAW3GVEC2AU4XCIPKNGKJTQQ5GADANCNFSM4JF7ZNPA .

[rostgoat]

I see. Even if I remove the Promise call around the transaction it still fails..why? Sorry I don't quite understand

[jakearchibald]

I'm not sure what you mean. TL;DR: transactions will close on the next microtask unless you've given it something to do.

On Mon, 28 Oct 2019, 20:59 Rost Myshkin, notifications@github.com wrote:

I see. Even if I remove the Promise call around the transaction it still fails..why? Sorry I don't quite understand

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jakearchibald/idb/issues/135?email_source=notifications&email_token=AAAW3GVIHXHO5WCGIGTYQEDQQ5HCJA5CNFSM4JF7ZNPKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECOL5GQ#issuecomment-547143322, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAW3GUZ3LBAF6RSCNGEJ53QQ5HCJANCNFSM4JF7ZNPA .

[rostgoat]

for (let i = 0; i < 10; ++i) {
  const num = Math.round(getRandomArbitrary(1, 1000000));
      const data = {
        id: num,
        img: img64,
      };
  store.put(data)
}

for example, calling store.put or store.add 10 times in a loop. Isn't that giving it something to do?

[jakearchibald]

That should be fine. I imagine your code is waiting a microtask somewhere else in the transaction.

On Mon, 28 Oct 2019, 21:08 Rost Myshkin, notifications@github.com wrote:

for (let i = 0; i < 10; ++i) { const num = Math.round(getRandomArbitrary(1, 1000000)); const data = { id: num, img: img64, }; store.put(data) }

for example, calling store.put or store.add 10 times in a loop. Isn't that giving it something to do?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jakearchibald/idb/issues/135?email_source=notifications&email_token=AAAW3GTGQZTBPDNXZ2J4KA3QQ5IGDA5CNFSM4JF7ZNPKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECOMYFY#issuecomment-547146775, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAW3GT47AWKEA5DIUPVTMDQQ5IGDANCNFSM4JF7ZNPA .

[rostgoat]

so in the docs example is there no possible way to wait for newVal before calling a transaction?

const tx = db.transaction('keyval', 'readwrite');
const store = tx.objectStore('keyval');
const val = (await store.get('counter')) || 0;
// This is where things go wrong:
const newVal = await fetch('/increment?val=' + val);
// And this throws an error:
store.put(newVal, 'counter');
await tx.done;

[jakearchibald]

You await stuff before and after a transaction, but not during.

On Mon, 28 Oct 2019, 21:33 Rost Myshkin, notifications@github.com wrote:

so in the docs example is there no possible way to wait for newVal before calling a transaction?

const tx = db.transaction('keyval', 'readwrite'); const store = tx.objectStore('keyval'); const val = (await store.get('counter')) || 0; // This is where things go wrong: const newVal = await fetch('/increment?val=' + val); // And this throws an error: store.put(newVal, 'counter'); await tx.done;

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jakearchibald/idb/issues/135?email_source=notifications&email_token=AAAW3GVIU5ZVTYCVKNFPARDQQ5LAZA5CNFSM4JF7ZNPKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECOO5QA#issuecomment-547155648, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAW3GSLM2O6DGB4LS4AXFLQQ5LAZANCNFSM4JF7ZNPA .

[rostgoat]

but let's say you need newValue to fetch something before storing it in a transaction. is that possible? if so, how?

[jakearchibald]

You would need to create two transactions

On Mon, 28 Oct 2019, 21:59 Rost Myshkin, notifications@github.com wrote:

but let's say you need newValue to fetch something before storing it in a transaction. is that possible? if so, how?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jakearchibald/idb/issues/135?email_source=notifications&email_token=AAAW3GUMTBEW5ZLGOBELUMLQQ5OEJA5CNFSM4JF7ZNPKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECOQ5EY#issuecomment-547163795, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAW3GWPQDQWTHZG26PURSDQQ5OEJANCNFSM4JF7ZNPA .

[rostgoat]

Can you please elaborate?

[jakearchibald]

Right now you're creating a single transaction to do the read and the write, but it doesn't work because you've got a fetch in the middle of it.

Instead, do the read in a transaction, then do the fetch, then do the write in a second transaction.

On Mon, 28 Oct 2019, 22:23 Rost Myshkin, notifications@github.com wrote:

Can you please elaborate?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jakearchibald/idb/issues/135?email_source=notifications&email_token=AAAW3GVLOD3MRFBHOA4M7KLQQ5Q5TA5CNFSM4JF7ZNPKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECOSXBY#issuecomment-547171207, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAW3GWN7AIUXPX5PFHXKUDQQ5Q5TANCNFSM4JF7ZNPA .

[rostgoat]

ah! that makes perfect sense now and the code runs as it should. Cheers!

[jakearchibald]

You'll need to handle cases where something else alters the counter while you're fetching. Here's how I'd do it:

const originalVal = await db.get('keyval', 'counter');
const newVal = await getNewValSomeAsyncWay();
const tx = db.transaction('keyval', 'readwrite');
const val = await tx.store.get('counter');
if (val !== originalVal) {
  // The value has been changed while we were fetching
  tx.abort();
} else {
  tx.store.put(newVal, 'counter');
}
await tx.done; // This will throw if the transaction was aborted

[rostgoat]

Thanks Jake that really helps a lot!