search

ipfs / kubo

An IPFS implementation in Go
https://docs.ipfs.tech/how-to/command-line-quick-start/
Other
16k stars 3k forks source link

Extending the api for hashing file(s) from multihash without data transfers. #4420

Open Amxx opened 6 years ago

Amxx commented 6 years ago

Version information:

go-ipfs version: 0.4.13- Repo version: 6 System version: amd64/linux Golang version: go1.9.2

Type: Enhancement

Description:

For my project I need to get a hash (using sha256) of files stored on ipfs. I also need a salted hash (for signature purpose). So far, what I do, using the python api is looks like this (simplified for readability, my real code gets the files and reads it by blocks of 64k to be more memory efficient). api = ipfsapi.connect(host=host, port=port) data = api.cat(multihash) vote = hashlib.sha256( data).hexdigest() sign = hashlib.sha256(salt+data).hexdigest()

my issue is that I have to get the file, whether it is by using api.get or api.cat, which cost bandwidth. How complicated would it be to implement something like ipfsapi.sha256(multihash, salt=b'') that computes the hash (with potential salt) remotely and avoid huge data exchanges ?

Stebalien commented 6 years ago

I'd be very weary of adding special purpose functions like this. Providing a simple way to hash files may make sense but almost no programs (except for yours) will need the salt parameter.

Assuming this your node and this program are running on the same computer, this shouldn't really cost any "bandwidth". Unfortunately, there is a fair amount of HTTP + TCP overhead. I think the best solution here would be to make the API faster by using a better RPC system with a persistent connection (planned), for local connections, use unix domain sockets (#4218).

Eventually like to be able to send a (sandboxed) program to a remote node and have that node execute it (which would solve this) but that's a ways off.