Suggestion: use perceptual hashing to compare images

[simon987]

Using perceptual hashing would make the database much smaller (~16 bytes per image) and more robust. Example python implementation: https://github.com/JohannesBuchner/imagehash, explanation: https://www.hackerfactor.com/blog/index.php?/archives/432-Looks-Like-It.html. dhash would be the easiest to implement in JavaScript, I can post a code snippet it you wish, let me know!

[bethylamine]

This looks promising - go ahead if you'd like to do a proof of concept. I'd include it alongside the current algorithm and do some profiling

[bethylamine]

At some point it will probably be prudent to set up a proper test harness to evaluate different algorithms with actual data, which I can get set up if you're willing to have a go with it.

[simon987]

go ahead if you'd like to do a proof of concept

Cool! I've created this repository/package you can use: https://github.com/simon987/imagehash-web You should be able to copy the dist/imagehash-web.min.js file in your project or manually pick and choose the code you want to add (it's using MIT license you can do whatever) and do something like:

    imgEl.onload = () => {
+     var anotherImgHashToTest = phash(imgEl, 8); // 8 is the hash size in bytes

      ctx.drawImage(imgEl, 0, 0, 16, 16);

      // get the image data
      var imgData = ctx.getImageData(0, 0, canvas.width, canvas.height);
      var d = imgData.data;
      var imgHash = "";
      // loop through all pixels

    ...
+        // You can also store the hashes with base64 instead of hex string with .fromBase64() .toBase64()
+        const hammingDist = anotherImgHashToTest.hammingDistance(ImageHash.fromHexString(tImg["phash"]));
+        if (hammingDist <= 10) {
+               // This is a close match with the database
+               // ...
+        }

        const imgHashResult = compareImageHash(imgHash, tImg["hash"]);
        const diff = imgHashResult["diffs"];
        const entropy = imgHashResult["entropy"]
        if (diff < 200 + entropy * 100) {

You will have to adjust the hash size and algorithm based on the precision/recall compromise that works best for the extension.

set up a proper test harness

If you already have a decently sized set of pictures, we can create a list of transformations (ex. resizing, jpeg compression, slight watermark, cropping, color changes etc.) to see which parameter are the most robust. I'd be happy to help!

[bethylamine]

Awesome! I'll check this out as soon as I can. I think for Twitter, compressing degradation, resizing, cropping and screenshotting, image-within-image, etc., would be good transforms to test. I have image sets we can use for testing.

[bethylamine]

I've sent an invite to a private repository where I'm putting sample images.

[bethylamine]

Did you really put all this together in two days?? What the heck 👀

[bethylamine]

I forked imagehash-web and am making a rudimentary ui to play with things. It's working great so far

[simon987]

Did you really put all this together in two days?? What the heck eyes

hyperfocus goes brrr 😜

I forked imagehash-web and am making a rudimentary ui to play with things. It's working great so far

That's awesome :open_mouth: !

I added the whash algorithm in 1.1.0, I'm curious to see how it performs vs. phash.