Callidon / bloom-filters

JS implementation of probabilistic data structures: Bloom Filter (and its derived), HyperLogLog, Count-Min Sketch, Top-K and MinHash
https://callidon.github.io/bloom-filters/
MIT License
378 stars 43 forks source link

saveToJSON doesn't export default seed #64

Closed TimDaub closed 1 year ago

TimDaub commented 1 year ago

This code imports the cuckoo filter to the localStorage in v3.0.0, but upon importing it again from localStorage, it'll find that the default seed wasn't serialized, and so it'll throw an error https://github.com/Callidon/bloom-filters/blob/b4ca772e810027d8777bc097340599c0b7884567/src/cuckoo/cuckoo-filter.ts#L74

import { CuckooFilter } from "bloom-filters";

// Check if there's a saved filter in localStorage.
let filter;
const savedFilter = JSON.parse(localStorage.getItem("--kiwi-news-read-links"));
console.log(savedFilter);
try {
  // Try to load the filter from localStorage.
  filter = CuckooFilter.fromJSON(savedFilter);
} catch (error) {
  console.log(error);
  // If an error is thrown, create a new filter.
  filter = CuckooFilter.create(1000, 0.01);
}

// Keep a queue of the most recent N links.
let queue = [];

// Check the current path.
const path = window.location.pathname;

if (path === "/" || path === "/new") {
  // Extract all the links on page load.
  const links = Array.from(document.querySelectorAll("a.story-link")).map(
    (a) => a.href,
  );

  links.forEach((link) => {
    if (!filter.has(link)) {
      const parentElement = document
        .querySelector(`a[href="${link}"]`)
        .closest("div");
      const domainElement = parentElement.querySelector(".story-domain");
      domainElement.textContent += " •";
      addLink(link);
    }
  });

  console.log("safe", filter.saveAsJSON());
  // Save the filter to localStorage.
  localStorage.setItem(
    "--kiwi-news-read-links",
    JSON.stringify(filter.saveAsJSON()),
  );
}

function addLink(link) {
  // If the filter is full, remove the oldest link.
  if (queue.length >= 1000) {
    const oldestLink = queue.shift();
    filter.remove(oldestLink);
  }

  // Add the new link to the filter and the queue.
  filter.add(link);
  queue.push(link);
}

however, after adding a custom seed manually, the seed gets exported as JSON and the problem is resolved

try {
  8   // Try to load the filter from localStorage.
  9   filter = CuckooFilter.fromJSON(savedFilter);
 10 } catch (error) {
 11   console.log(error);
 12   // If an error is thrown, create a new filter.
 13   filter = CuckooFilter.create(1000, 0.01);
 14   filter.seed = 0xabcd;
 15 }

still, I think this should also work with the default seed, shouldn't it?

folkvir commented 1 year ago

Hi! Thank you for reporting this. In fact the seed is correctly exported but badly imported... https://github.com/Callidon/bloom-filters/blob/b4ca772e810027d8777bc097340599c0b7884567/src/cuckoo/cuckoo-filter.ts#L99 Note the json.seed instead of json._seed.

folkvir commented 1 year ago

This has been merged and deployed. Here is the new version: