search

TalAloni / MD5Stream

Store MD5 hash in NTFS Alternate Data Stream
GNU Lesser General Public License v3.0
12 stars 4 forks source link

[Feature Request] Command Line support #1

Open swarup459 opened 3 years ago

swarup459 commented 3 years ago

I use this program a lot nearly everyday while archiving Internet files. It would have been great if there was Command Line Support so as it could be used with batch script. If it is hard to implement, then at least a Drag-n-Drop support for Folders on the program window is requested. Thank you in advance...

TalAloni commented 3 years ago

Thanks for the feedback. Command line support means creating a textual output that's easy to understand and return codes that makes sense for batch execution, this is possible, but I'm not sure it would be as useful as the GUI. Drag-n-Drop support, on the other hand, is a neat idea and I'll try to find time to implement it in the coming weeks.

swarup459 commented 3 years ago

Command line support is really useful for someone who wants to save time and labour by avoiding repetitive tasks. My use case scenario:

The process of archiving the Internet files has the following steps:

  1. Collecting of Internet files from various public resources. (Manual Mode)

  2. Add Timestamp(YY-mm-DD) and CRC32 to their filenames like '210913-filename [CRC32].ext'. (Batch script automation- using 'FileMenu Tools Advanced Renamer'). RapidCRC Unicode can verify file integrity if it has CRC32 in its filename.

  3. Add MD5 streams to every file in order to verify the integrity of file in a later time. (using MD5Stream- needs user interaction: Open the program -> Browse for the folder -> Start)

  4. Create Torrent(with minimum piece size) per file basis, so as it can be used to reconstruct the file in case either/both of the copies is/are partiallly damaged due to developing Bad Sectors in HDDs over time. (can be automated- using py3createtorrent/dottorrent-cli)

  5. Add MD5 streams to every newly created Torrent file. (using MD5Stream- needs user interaction)

  6. Make every Torrent file 'Read-only' in order to prevent accidental deletion of MD5 streams. (can be automated)

  7. Create 10% PAR2 recovery per file basis as an additional safety from data corruption(for whatever reason). (can be automated- MultiPar)

  8. Add MD5 streams to every newly created PAR2 file. (using MD5Stream- needs user interaction)

  9. Make every PAR2 file 'Read-only' in order to prevent accidental deletion of MD5 streams. (can be automated)

By the way, PAR2 files and Torrent files are created maintaining the exact directory structure of Data Folder but inside a Base Folder named 'PAR2' and 'Torrents' respectively in the root of the drive. See Screenshots:

capture_par2

capture_torrent

  1. Create a copy of every Data file, corresponding Torrent file and PAR2 file in another NTFS formatted Disk 2 for backup purpose. (can be automated)
  2. Verify the integrity of copied Data files, Torrent files and PAR2 files in Disk 2 using MD5Stream. (needs user interaction)
  3. Verify Torrents and PAR2s against duplicates of Data Files in Disk 2. (can be automated- using torrentcheck and MultiPar respectively)

The whole process is repeated in every 2-3 days interval.

You see, all the steps from 2-12 except 3, 5, 8 and 11(where use of MD5Stream is needed) can be performed per file basis by iterating them with a 'for' loop. If MD5Stream supported command line, then steps 3, 5, 8 and 11 could also be automated and all the steps could be implemented in one batch file.

Right now, I am doing the steps 3, 5, 8 and 11 manually. It is doable for a fresh backup job where Torrents and PAR2s are created for all the files in a directory. I can add MD5 streams to all the Data files, created Torrent files and PAR2 files simply by browsing the Base Data folder, Base Torrent folder and Base PAR2 folder in the 'MD5Stream' program.

But the problem arises, when I modify some previously backed-up files inside a deep folder structure or I update a folder by adding new files to it. The batch scripts update the corresponding Torrent and PAR2 files. But, now it is difficult to add MD5 streams to the new Data files, their corresponding Torrent and PAR2 files. Either I have to process the whole folder(which is time consuming) or I have to selectively Hardlink those to another folder and then only I can add MD5 streams to them.

TalAloni commented 3 years ago

it seems that I wasn't clear enough: You'll have no argument for me that command line support is really useful, my point was that command line support will require a lot of effort to create a textual output that is as readable as the GUI.