alanshaw / ipfs-only-hash

#️⃣ Just enough code to calculate the IPFS hash for some data
MIT License
137 stars 28 forks source link

Hashes differ from ones on pinata.cloud #21

Open aradica opened 3 years ago

aradica commented 3 years ago

They don't start with Qm...?

saeta-eth commented 3 years ago

There are two versions of CIDs. Have you seen them? I think it's related to your issue.

Related links:

fandrupaw commented 3 years ago

hey, Im facing same issue, Pinata by default uses CID v0, both hashes starts with Qm, the rest is different

alanshaw commented 2 years ago

@aradica see the comment here https://github.com/alanshaw/ipfs-only-hash/issues/20#issuecomment-983493673 TL;DR you can pass --cid-version=0 to get Qm... hashes on the command line.

@fandrupaw any chance you can share the data, and the CID you're getting from Pinata so we can verify? 🙏 The unixfs importer code is interop tested with go-ipfs to ensure the same CID is produced for the same data but there could be a bug!

titusz commented 2 years ago

ipfs-only-hash does also seem to create different CIDv1s when compared to ipfs-go version 0.11.0: Testing for example the bitcoin whitepaper

ipfs add --only-hash --cid-version=1 bitcoin.pdf
added bafkreifrm5azdkeoyxg5om7eeqfidabraxoecllmm4enkovzj7ber5hvkm bitcoin.pdf

versus

ipfs-only-hash --cid-version=1 bitcoin.pdf
bafybeibj3nf4iyxt2guxihs77sylpuwu4l4yn4cfqumpc2xplxgxt4ssoa
MAkcanca commented 2 years ago

From here and checking the two CIDs you provided, it's obvious that one of them is wrapped with UnixFS, and other one is raw block. That's why the CIDs are different

chichke commented 2 years ago

Same here works great with CIDv0 but doesn't with CIDv1

naviangie commented 2 years ago

The same chunking and encoding algorithm is required.

importer([{content}], block, { onlyHash: true, cidVersion: 1, rawLeaves: true })
larskarbo commented 11 months ago

I had some issues getting the same hashes as Pinata for files above 262kb (when chunking kicks in).

This is my solution using Helia:

import { unixfs } from "@helia/unixfs"
import { BlackHoleBlockstore } from "blockstore-core/black-hole"
import { fixedSize } from "ipfs-unixfs-importer/chunker"
import { balanced } from "ipfs-unixfs-importer/layout"

export const calculateCid = async (bytes: Uint8Array) => {
    const unixFs = unixfs({
      blockstore: new BlackHoleBlockstore(),
    })

    const cid = await unixFs.addBytes(bytes, {
      cidVersion: 0,
      rawLeaves: false,
      leafType: "file",
      layout: balanced({
        maxChildrenPerNode: 174,
      }),
      chunker: fixedSize({
        chunkSize: 262144,
      }),
    })

    const cidv0 = cid.toV0().toString() // QmPK1s...
    const cidv1 = cid.toV1().toString() // b45165...
}

I wrote some more about it here.

lancelot-c commented 9 months ago

Finally something that works ! Thanks @larskarbo !

endomorphosis commented 2 months ago

I also noticed that there are differences in hashes between ipfs command and this package.

['bafybeif7ztnhq65lumvvtr4ekcwd2ifwgm3awq4zfr3srh462rwyinlb4y'] -- this package ['bafkreihdwdcefgh4dqkjv67uzcmw7ojee6xedzdetojuzjevtenxquvyku'] multiformats python library ['bafkreihdwdcefgh4dqkjv67uzcmw7ojee6xedzdetojuzjevtenxquvyku'] ipfs kubo