search

greentfrapp / compute-your-own-neuralhash

Simple demo to compute your own NeuralHash on-device.
Apache License 2.0
13 stars 2 forks source link

How were the seed.js and model.json files gotten #2

Open csandman opened 2 years ago

csandman commented 2 years ago

Sorry if this is a stupid question, but I'm really curious how you got the seed.js and model.json files from the original NeuralHash files. I'd assume the seed file comes from the neuralhash_128x96_seed1.dat file, but I'm really not sure how you converted it to get to that point. Same for the model, I'd assume that's from the .net, .shape, and .weights file but I have no idea how.

I'm really new to ML space so I'm not super familiar with the tools, but I'm very curious! I was never able to get the ONNX model that the original repo shows how to make to work on my machine, I believe because of a hardware incompatibility, but I was able to get the tensorflow model you have to work, so I was thinking it would be ideal to be able to convert that in the future in order to keep up to date if Apple changes their model or seed.

greentfrapp commented 2 years ago

Hey @csandman, thanks for your interest in this repo!

It's been awhile so I might be wrong but here's what I remember:

For seed.js, like you mention you'd first need neuralhash_128x96_seed1.dat.

Then you should run something like

import numpy as np

seed1 = open('neuralhash_128x96_seed1.dat', 'rb').read()[128:]
seed1 = np.frombuffer(seed1, dtype=np.float32)
seed1 = seed1.reshape([96, 128])
# seed1 will be the array that you see in seed.js

As for the model - you first need the ONNX model, then I believe I opened it in Torch and re-exported it again into tensorflow-js compatible formats.

csandman commented 2 years ago

Thanks a bunch for responding! So for the seed, I see that you basically just saved the output of that part of the original script as a JS/JSON object and used that in your own script, that makes a lot of sense haha. I'm curious though, I see the reshape part of the script is run both in the original script and in your script, is that correct? Or did you save the output of loading the seed before you ran reshape in the original script?

This might also be due to a lack of understanding on my part, I'm not entirely sure what the reshape part of the script is doing.