dasec / fishy

Toolkit for Filesystem based Data Hiding Techniques.
MIT License
36 stars 1 forks source link

Metadata: Fix reading metadata #21

Closed importantchoice closed 6 years ago

importantchoice commented 6 years ago

Reading metadata files via the cli tool failes since the introduction of the metadata encryption feature.

Without encryption of the metadata file:

$ echo nonsense | fishy -d testfs-fat12.dd fileslack -d onedirectory -m /tmp/foo -w
$ fishy metadata -m /tmp/foo                                                       
Traceback (most recent call last):
  File "/usr/bin/fishy", line 11, in <module>
    load_entry_point('fishy', 'console_scripts', 'fishy')()
  File "/home/matti/Seafile/Dokumente/Studium/5.Semester/PSE_-_Projekt_Systementwicklung/fishy/fishy/cli.py", line 204, in main
    do_metadata(args)
  File "/home/matti/Seafile/Dokumente/Studium/5.Semester/PSE_-_Projekt_Systementwicklung/fishy/fishy/cli.py", line 24, in do_metadata
    meta.read(args.metadata)
  File "/home/matti/Seafile/Dokumente/Studium/5.Semester/PSE_-_Projekt_Systementwicklung/fishy/fishy/metadata.py", line 211, in read
    self.metadata = json.loads(instream.read().decode("utf8"))
AttributeError: 'str' object has no attribute 'decode'

With encryption of the metadata file:

$ echo nonsense | fishy -d testfs-fat12.dd -p lol fileslack -d onedirectory -m /tmp/foo -w
$ fishy metadata -m /tmp/foo                                                              
Traceback (most recent call last):
  File "/usr/bin/fishy", line 11, in <module>
    load_entry_point('fishy', 'console_scripts', 'fishy')()
  File "/home/matti/Seafile/Dokumente/Studium/5.Semester/PSE_-_Projekt_Systementwicklung/fishy/fishy/cli.py", line 204, in main
    do_metadata(args)
  File "/home/matti/Seafile/Dokumente/Studium/5.Semester/PSE_-_Projekt_Systementwicklung/fishy/fishy/cli.py", line 24, in do_metadata
    meta.read(args.metadata)
  File "/home/matti/Seafile/Dokumente/Studium/5.Semester/PSE_-_Projekt_Systementwicklung/fishy/fishy/metadata.py", line 211, in read
    self.metadata = json.loads(instream.read().decode("utf8"))
  File "/usr/lib/python3.6/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa6 in position 9: invalid start byte

As the unit tests for reading and writing metadata pass, there might some inconsistencies between the tests and the usage via the cli.

D-Kern commented 6 years ago

fixed in 10e1bfb883eb5ce36dc5546624371c70a9bfde8f

importantchoice commented 6 years ago

This bug is still there when reading encrypted metadata!

D-Kern commented 6 years ago

@importantchoice works for me:

$ echo "test" | sudo python3 -m fishy.cli -d /dev/sdb1 -p pass fileslack -m /tmp/meta.json -d / -w
File hidden
$ sudo python3 -m fishy.cli -p pass metadata -m /tmp/meta.json
Version: 2
Module Identifier: ntfs-slack
Stored Files:
  File_ID: 0
  Filename: 0
  Associated File Metadata:
    {'addrs': [[89888, 5]]}

looks like you forgot to set the password when reading and it failed while trying to utf-8 decode the encrypted data.

$ echo "test" | sudo python3 -m fishy.cli -d /dev/sdb1 -p pass fileslack -m /tmp/meta.json -d / -w
File hidden
$ sudo python3 -m fishy.cli metadata -m /tmp/meta.json
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/local/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/fs/pse/fishy/fishy/cli.py", line 232, in <module>
    main()
  File "/home/fs/pse/fishy/fishy/cli.py", line 215, in main
    do_metadata(args)
  File "/home/fs/pse/fishy/fishy/cli.py", line 24, in do_metadata
    meta.read(args.metadata)
  File "/home/fs/pse/fishy/fishy/metadata.py", line 244, in read
    self.metadata = json.loads(instream.read().decode("utf8"))
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd2 in position 4: invalid continuation byte

I could probably add a better error message for this case.

D-Kern commented 6 years ago

Added error message in b458b362b287c2e43fe0bc4540baf1003395f271

importantchoice commented 6 years ago

Yes, you are right! That was just me, who misused the cli. Thanks for adding the error message!