[WIP] Initial attempt to validate >256kb ipfs files

[njgheorghita]

What was wrong?

Validating ipfs contents retrieved against their hash is broken for files > 256kb.

useful links:

• https://stackoverflow.com/questions/51655352/how-recreate-a-hash-digest-of-a-multihash-in-ipfs/51658540
• https://github.com/noffle/ipfs-merkle-dag-node/blob/master/index.js

Cute Animal Picture

[njgheorghita]

@pipermerriam I've been running down some rabbitholes to get >256kb ipfs file content-hash validation working. If you can take a quick :eyes: and see if anything jumps out to you? As far as I can tell, everything's in the right place, but the output hashes just won't match the expected value. I've manually decoded the individual root and leaf ipfs objects, and everything in my code corresponds to the output. i.e.

• the ipfs object for txt.txt

  ipfs object get QmegXsU7EopJ9EVj9ELwgHGE75Xke6FYBP84PaygiAsqB5
  {"Links":[{"Name":"","Hash":"QmbYfXF7A7hgXUzEZSifji3D6Nf856kdK8Edu8n7knQSHr","Size":262158},{"Name":"","Hash":"QmbYfXF7A7hgXUzEZSifji3D6Nf856kdK8Edu8n7knQSHr","Size":262158},{"Name":"","Hash":"QmbYfXF7A7hgXUzEZSifji3D6Nf856kdK8Edu8n7knQSHr","Size":262158},{"Name":"","Hash":"QmbYfXF7A7hgXUzEZSifji3D6Nf856kdK8Edu8n7knQSHr","Size":262158},{"Name":"","Hash":"QmbYfXF7A7hgXUzEZSifji3D6Nf856kdK8Edu8n7knQSHr","Size":262158},{"Name":"","Hash":"QmbYfXF7A7hgXUzEZSifji3D6Nf856kdK8Edu8n7knQSHr","Size":262158},{"Name":"","Hash":"QmbYfXF7A7hgXUzEZSifji3D6Nf856kdK8Edu8n7knQSHr","Size":262158},{"Name":"","Hash":"QmbYfXF7A7hgXUzEZSifji3D6Nf856kdK8Edu8n7knQSHr","Size":262158},{"Name":"","Hash":"QmbYfXF7A7hgXUzEZSifji3D6Nf856kdK8Edu8n7knQSHr","Size":262158},{"Name":"","Hash":"QmbYfXF7A7hgXUzEZSifji3D6Nf856kdK8Edu8n7knQSHr","Size":262158},{"Name":"","Hash":"QmbYfXF7A7hgXUzEZSifji3D6Nf856kdK8Edu8n7knQSHr","Size":262158},{"Name":"","Hash":"QmbYfXF7A7hgXUzEZSifji3D6Nf856kdK8Edu8n7knQSHr","Size":262158},{"Name":"","Hash":"QmbYfXF7A7hgXUzEZSifji3D6Nf856kdK8Edu8n7knQSHr","Size":262158},{"Name":"","Hash":"QmbYfXF7A7hgXUzEZSifji3D6Nf856kdK8Edu8n7knQSHr","Size":262158},{"Name":"","Hash":"QmbYfXF7A7hgXUzEZSifji3D6Nf856kdK8Edu8n7knQSHr","Size":262158},{"Name":"","Hash":"QmbYfXF7A7hgXUzEZSifji3D6Nf856kdK8Edu8n7knQSHr","Size":262158},{"Name":"","Hash":"QmbYfXF7A7hgXUzEZSifji3D6Nf856kdK8Edu8n7knQSHr","Size":262158},{"Name":"","Hash":"QmbYfXF7A7hgXUzEZSifji3D6Nf856kdK8Edu8n7knQSHr","Size":262158},{"Name":"","Hash":"QmbYfXF7A7hgXUzEZSifji3D6Nf856kdK8Edu8n7knQSHr","Size":262158},{"Name":"","Hash":"QmZw2JEKQu8n5zisuU7JDAeWSUm2XvqjqV72bizCdDQUDM","Size":19278}],"Data":"\u0008\u0002\u0018���\u0002 ��\u0010 ��\u0010 ��\u0010 ��\u0010 ��\u0010 ��\u0010 ��\u0010 ��\u0010 ��\u0010 ��\u0010 ��\u0010 ��\u0010 ��\u0010 ��\u0010 ��\u0010 ��\u0010 ��\u0010 ��\u0010 ��\u0010 ��\u0001"}

• the "data" field for the txt.txt ipfs object decoded

  ipfs object data QmegXsU7EopJ9EVj9ELwgHGE75Xke6FYBP84PaygiAsqB5 | unixfs
  {"type":"file","blockSizes":[262144,262144,262144,262144,262144,262144,262144,262144,262144,262144,262144,262144,262144,262144,262144,262144,262144,262144,262144,19264]}