martymac / fpart

Sort files and pack them into partitions
https://www.fpart.org/
BSD 2-Clause "Simplified" License
230 stars 39 forks source link

Format change for '-S' output #38

Closed hjmangalam closed 2 years ago

hjmangalam commented 2 years ago

Hi Ganael, The new '-S' (skip files larger than partition size) output format is the current output of '-S' is a bit noisy: ie: S (1902116864): /home/hjm/Downloads/isos/lmde-4-cinnamon-32bit.iso

Could this be simplifired to: 1902116864<tab>/home/hjm/Downloads/isos/lmde-4-cinnamon-32bit.iso

the '-v' output goes to STDERR so it can be filtered and that leaves a nicely segregated file

$ ~/bin/fpart-1.4.1 -vvv  -L -S  -s 200m \
-o ~/fpart/Fs/f  ~/Downloads > chunk.exceptions

(lots of verbose STDERR output)

$ head -5 chunk.exceptions
S (327155712): /home/hjm/Downloads/RT-contents.ibd.most-of-rt-database.data
S (492830720): /home/hjm/Downloads/isos/debian-11.1.0-i386-netinst.iso
S (2009333760): /home/hjm/Downloads/isos/Fedora-Workstation-Live-x86_64-35-1.2.iso
S (3204448256): /home/hjm/Downloads/isos/kubuntu-20.04.3-desktop-amd64.iso

so rather than perform a couple of complex regex splits (OK, not very complex), all you need is simple (and usually default in many languages) split on whitespace.

The current format of the partition files is simply a list of fully qualified filename paths with no prefixes. so the the simpler format suggested above is similar (prefixed with the byte size of the file.)

Thanks Harry

martymac commented 2 years ago

Hello Harry,

I've just pushed a change that modifies the global output format of files listed to stdout. The new format is as below :

partition_index file_size file_path

and will be much easier to parse.

Cheers,

Ganael.

hjmangalam commented 2 years ago

I'll give it a try on the weekend. THANK YOU!! harry

On Thu, Mar 3, 2022 at 8:27 AM Ganael Laplanche @.***> wrote:

Hello Harry,

I've just pushed a change that modifies the global output format of files listed to stdout. The new format is as below :

partition_index file_size file_path

and will be much easier to parse.

Cheers,

Ganael.

— Reply to this email directly, view it on GitHub https://github.com/martymac/fpart/issues/38#issuecomment-1058231002, or unsubscribe https://github.com/notifications/unsubscribe-auth/AASF3Y5RUINLJC7RZGNDLY3U6DR6JANCNFSM5MOR5TRQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

--

Harry Mangalam

martymac commented 2 years ago

Hello Harry,

Have you been able to test the change ?

hjmangalam commented 2 years ago

Hi Ganael,

It looks good:

$ ~/Downloads/fpart/src/fpart -L -S -s 200m -o ~/chunks/f ~/Downloads S 327155712 /home/hjm/Downloads/RT-contents.ibd.most-of-rt-database.data S 492830720 /home/hjm/Downloads/isos/debian-11.1.0-i386-netinst.iso S 2009333760 /home/hjm/Downloads/isos/Fedora-Workstation-Live-x86_64-35-1.2.iso S 3204448256 /home/hjm/Downloads/isos/kubuntu-20.04.3-desktop-amd64.iso S 1902116864 /home/hjm/Downloads/isos/lmde-4-cinnamon-32bit.iso ...

I would prefer a less common delimiter than a space between the fields since many filenames have spaces - a tab character would be more convenient (for me) and easier to handle in general I think..? ie: instead of [file_entry.c:423] (I think)

fprintf(stdout, "S %ju %s\n", size, path); use fprintf(stdout, "S\t%ju\t%s\n", size, path);

And what is the purpose of the leading 'S'? Is that to differentiate these lines from any other fpart STDOUT output? I can't tell from the src code and surrounding comments. It's a hard-coded 'S', not a var. I can certainly work with the 'S'; just interested in the rationale.

thanks! Harry

.

On Tue, Mar 8, 2022 at 3:31 AM Ganael Laplanche @.***> wrote:

Hello Harry,

Have you been able to test the patch ?

— Reply to this email directly, view it on GitHub https://github.com/martymac/fpart/issues/38#issuecomment-1061682511, or unsubscribe https://github.com/notifications/unsubscribe-auth/AASF3Y25WQCTETYUQEOJUSLU643CLANCNFSM5MOR5TRQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

--

Harry Mangalam

martymac commented 2 years ago

Hi Harry,

Right, it will also be cleaner. I have just pushed a change in that way.

The 'S' letter means 'S'kipped, see issue #36.

Cheers,

Ganael.

hjmangalam commented 1 year ago

Sorry Ganael, I blanked on trying it. I'll do it later today. Sorry. Harry

On Tue, Mar 8, 2022 at 3:31 AM Ganael Laplanche @.***> wrote:

Hello Harry,

Have you been able to test the patch ?

— Reply to this email directly, view it on GitHub https://github.com/martymac/fpart/issues/38#issuecomment-1061682511, or unsubscribe https://github.com/notifications/unsubscribe-auth/AASF3Y25WQCTETYUQEOJUSLU643CLANCNFSM5MOR5TRQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

--

Harry Mangalam

martymac commented 1 year ago

Hello Harry!

No pb, take your time... the change is already available in the latest release.

Cheers,

Ganael.