search

ipfs / js-ipfs

IPFS implementation in JavaScript
https://js.ipfs.tech
Other
7.44k stars 1.25k forks source link

Adding files preserving mode or mtime results in data corruption #4093

Closed kstuart closed 1 year ago

kstuart commented 2 years ago

Linux ce937568e283 5.17.5-arch1-1 #1 SMP PREEMPT Wed, 27 Apr 2022 20:56:11 +0000 x86_64 Linux

js-ipfs version: 0.15.2-0078aad85f92d7bc8e3a22eb9ef8efd257b439b5 interface-ipfs-core version: ^0.155.0 ipfs-http-client version: 1.0.0 Repo version: 12 System version: x64/linux Node.js version: v16.4.2 Commit: 0078aad85f92d7bc8e3a22eb9ef8efd257b439b5

jsipfs run in daemon offline mode

Steps to reproduce with --preserve-mtime (empty file)

touch -d@1652299617 /tmp/empty_file jsipfs add --preserve-mtime /tmp/empty_file added QmcrCRhLSqsroGBX2PyBtdCrKWBDxjgPpATLg3KxMqywna empty_file

jsipfs files stat /ipfs/QmcrCRhLSqsroGBX2PyBtdCrKWBDxjgPpATLg3KxMqywna QmcrCRhLSqsroGBX2PyBtdCrKWBDxjgPpATLg3KxMqywna Size: 6 CumulativeSize: 22 ChildBlocks: 0 Type: file Mode: -rw-r--r-- Mtime: May 11, 2022, 08:06:57 PM UTC

jsipfs get QmcrCRhLSqsroGBX2PyBtdCrKWBDxjgPpATLg3KxMqywna Saving file(s) QmcrCRhLSqsroGBX2PyBtdCrKWBDxjgPpATLg3KxMqywna

xxd QmcrCRhLSqsroGBX2PyBtdCrKWBDxjgPpATLg3KxMqywna 00000000: 0a04 0802 1800 ......

Expected:

The file 'QmcrCRhLSqsroGBX2PyBtdCrKWBDxjgPpATLg3KxMqywna' to have size 0 and contain no data.

Steps to reproduce with --preserve-mode (file with content)

echo -n "A" > /tmp/test_file touch -d@1652299617 /tmp/test_file jsipfs add --preserve-mode /tmp/test_file added QmW198BawsGUvwKQN8hxUHKdCNSS3fYf7pMmPQujUAJnBH test_file

jsipfs files stat /ipfs/QmW198BawsGUvwKQN8hxUHKdCNSS3fYf7pMmPQujUAJnBH QmW198BawsGUvwKQN8hxUHKdCNSS3fYf7pMmPQujUAJnBH Size: 9 CumulativeSize: 17 ChildBlocks: 0 Type: file Mode: -rw-r--r-- Mtime:

jsipfs get QmW198BawsGUvwKQN8hxUHKdCNSS3fYf7pMmPQujUAJnBH Saving file(s) QmW198BawsGUvwKQN8hxUHKdCNSS3fYf7pMmPQujUAJnBH

xxd QmW198BawsGUvwKQN8hxUHKdCNSS3fYf7pMmPQujUAJnBH 00000000: 0a07 0802 1201 4118 01 ......A..

xxd /tmp/test_file 00000000: 41 A

Expected:

The file 'QmW198BawsGUvwKQN8hxUHKdCNSS3fYf7pMmPQujUAJnBH' to have size 1 and its content to exactly match /tmp/test_file.

BigLep commented 2 years ago

2022-06-10 conversation: there are two things to look into:

  1. Why js-ipfs is returning a non-zero size
  2. Why the CIDs are different

Maintainers agree this needs to get looked into. There isn't an estimate yet on when this will get picked up though given other priorities. Is this blocking for you @kstuart on your work?

github-actions[bot] commented 2 years ago

Oops, seems like we needed more information for this issue, please comment with more details or this issue will be closed in 7 days.

kstuart commented 2 years ago

@BigLep no, this doesn't block me. After further investigation I've updated the issue, note that preserving mode or mtime will add erroneous file data to the node, essentially a corruption of the original file.

whizzzkid commented 1 year ago

js-ipfs is being deprecated in favor of Helia. You can follow the migration plan here https://github.com/ipfs/js-ipfs/issues/4336 and read the migration guide.

Please feel to reopen with any comments before 2023-06-05. We will do a final pass on reopened issues afterward (see https://github.com/ipfs/js-ipfs/issues/4336).

This issue might be resolved in Helia, if not, please let us know!