cybercongress / cyb-ts

Immortal robot for the Great Web
https://cyb.ai
160 stars 37 forks source link

incorrect computation of cid on upload #1180

Open mastercyb opened 1 week ago

mastercyb commented 1 week ago

cid of files uploaded through cyb.ai are differs from cid of the same files uploaded to go-ipfs

added from cyb.ai https://cyb.ai/oracle/ask/QmZrLA5Dkdw7fred7QLyx8v7i345QUk8XjJtHfbxxWaFoP added from go-ipfs https://cyb.ai/oracle/ask/Qmc7ANo78MJ9yDXUxmErnWuPZPL5y2baYRZpN4CV9axeym

mastercyb commented 1 week ago

find related that correct ipfs hashes does not resolves

Screen Shot 2024-06-22 at 08 40 38

case https://cyb.ai/oracle/ask/QmeVMzMbknpwBU3pB7NhHjwvzVVeuUxAJ1wQnbkpQXPVAu vs https://emerald-raw-leopon-384.mypinata.cloud/ipfs/QmeVMzMbknpwBU3pB7NhHjwvzVVeuUxAJ1wQnbkpQXPVAu https://ipfs.io/ipfs/QmeVMzMbknpwBU3pB7NhHjwvzVVeuUxAJ1wQnbkpQXPVAu

mastercyb commented 1 week ago
Screen Shot 2024-06-22 at 08 41 57
mastercyb commented 1 week ago

looks like we will loose a chunk of content

dasein108 commented 1 week ago

echo "хуй" > example.txt ipfs add example.txt added QmdPxoRJkBoiMA1bbA6K6gXSxtq29VgTuChaAso9bBXXmc example.txt 7 B / 7 B ipfs cat QmdPxoRJkBoiMA1bbA6K6gXSxtq29VgTuChaAso9bBXXmc хуй ipfs cat QmWQixYeubM54RFHunos6K6aYYb1EE4XY6pBVDJtTNGr9D хуй%

ipfs object stat QmdPxoRJkBoiMA1bbA6K6gXSxtq29VgTuChaAso9bBXXmc NumLinks: 0 BlockSize: 15 LinksSize: 2 DataSize: 13 CumulativeSize: 15

ipfs object stat QmWQixYeubM54RFHunos6K6aYYb1EE4XY6pBVDJtTNGr9D NumLinks: 0 BlockSize: 14 LinksSize: 2 DataSize: 12 CumulativeSize: 14

ipfs cat QmdPxoRJkBoiMA1bbA6K6gXSxtq29VgTuChaAso9bBXXmc | xxd
00000000: d185 d183 d0b9 0a

ipfs cat QmWQixYeubM54RFHunos6K6aYYb1EE4XY6pBVDJtTNGr9D | xxd
00000000: d185 d183 d0b9

0a means - new line, echo "хуй" > example.txt - adds new line to the end of file content O_o, cut new line manually now I got

ipfs add example.txt

added QmWQixYeubM54RFHunos6K6aYYb1EE4XY6pBVDJtTNGr9D example.txt

so no problem with cyb, everything fits. this workflow can be used for investigation of mentioned files.

starcat4me commented 2 days ago

Is this error related?

Screenshot 2024-06-28 at 6 50 18 AM
dasein108 commented 2 days ago

Is this error related? Screenshot 2024-06-28 at 6 50 18 AM

this prob not, related to ipfs resolving, fill be improved with next release

dasein108 commented 2 days ago

@cyborgshead @mastercyb Problem is related to different block size of source node(chunker), so different block size/block count produce different hash. CID = hash of the root of merkle three CID != hash of the content (where blocks_count > 1)

dasein108 commented 2 days ago

need to investigate of block size of helia & kubo(top ipfs implementation) and config chunker for same block size if that differs.