search

leahneukirchen / mblaze

Unix utilities to deal with Maildir
Other
450 stars 50 forks source link

mpick messages which have been flagged with mflag #21

Closed larryhynes closed 7 years ago

larryhynes commented 7 years ago

Having applied the flags S and T with mflag, I would expect mpick to pick those messages, using the test for trashed (or seen, or either/both)...

$ mlist .mail/inbox
.mail/inbox/cur/1490462878.95690_0.sutro.larryhynes.com:2,ST
.mail/inbox/cur/1490613356.59772_0.sutro.larryhynes.com:2,ST
.mail/inbox/cur/1490656715.32897_0.sutro.larryhynes.com:2,ST
.mail/inbox/cur/1490691433.86410_0.sutro.larryhynes.com:2,ST

$ mlist .mail/inbox | mpick -t 'trashed'
4 mails tested, 0 picked.

$ mlist .mail/inbox | mpick -t '!trashed'
.mail/inbox/cur/1490462878.95690_0.sutro.larryhynes.com:2,ST
.mail/inbox/cur/1490613356.59772_0.sutro.larryhynes.com:2,ST
.mail/inbox/cur/1490656715.32897_0.sutro.larryhynes.com:2,ST
.mail/inbox/cur/1490691433.86410_0.sutro.larryhynes.com:2,ST
4 mails tested, 4 picked.

 $ mlist .mail/inbox/ | mpick :T
4 mails tested, 0 picked.

And then things get a little weird, when :S matches, then doesn't:

$ mlist .mail/inbox/ | mpick :S
.mail/inbox//cur/1490462878.95690_0.sutro.larryhynes.com:2,ST
.mail/inbox//cur/1490613356.59772_0.sutro.larryhynes.com:2,ST
.mail/inbox//cur/1490656715.32897_0.sutro.larryhynes.com:2,ST
.mail/inbox//cur/1490691433.86410_0.sutro.larryhynes.com:2,ST
4 mails tested, 4 picked.

 $ mlist .mail/inbox/ | mpick :T
4 mails tested, 0 picked.

$ mlist .mail/inbox/ | mpick :S
4 mails tested, 0 picked.

What am I missing here?

Duncaen commented 7 years ago 
~$ mlist mail/personal/ | mpick :T
mail/personal//cur/1490738402.14763_17.tux,U=37489:2,ST
3067 mails tested, 1 picked.
~$ mlist mail/personal/ | mpick -t 'trashed'
mail/personal//cur/1490738402.14763_17.tux,U=37489:2,ST
3067 mails tested, 1 picked.
~$ mlist mail/personal/ | mpick -t '!trashed' | grep U=37489
3067 mails tested, 3066 picked.
~$ mlist mail/personal/ | mpick :S >/dev/null
3067 mails tested, 2957 picked.
~$ mlist mail/personal/ | mpick -t 'trashed || (!trashed && seen && replied)'
mail/personal//cur/1487948612.2431_1.tux,U=32127:2,RS
mail/personal//cur/1487966486.1300_2.tux,U=32190:2,RS
mail/personal//cur/1463663678.2979_97.tux,U=1080:2,FRS
mail/personal//cur/1487957442.28059_15.tux,U=32169:2,RS
mail/personal//cur/1490738402.14763_17.tux,U=37489:2,ST
3067 mails tested, 5 picked.

I can't reproduce the problems, I have no idea why it would work once and then not 😕.

Tested it on openbsd and it worked too.

mblaze@obsdtst$ mlist ../../testinbox/| ./mpick :T
../../testinbox//cur/51:2,ST
../../testinbox//cur/12:2,ST
../../testinbox//cur/57:2,ST
../../testinbox//cur/20:2,ST
../../testinbox//cur/25:2,ST
../../testinbox//cur/93:2,ST
../../testinbox//cur/28:2,ST
../../testinbox//cur/92:2,ST
../../testinbox//cur/44:2,ST
../../testinbox//cur/65:2,SRT
../../testinbox//cur/73:2,ST
../../testinbox//cur/84:2,ST
../../testinbox//cur/76:2,ST
../../testinbox//cur/95:2,SRT
../../testinbox//cur/61:2,SRT
../../testinbox//cur/34:2,SRT
../../testinbox//cur/35:2,SRT
../../testinbox//cur/63:2,SRT
../../testinbox//cur/77:2,SRT
100 mails tested, 19 picked.
mblaze@obsdtst$ mlist ../../testinbox/| ./mpick :S >/dev/null
100 mails tested, 100 picked.
larryhynes commented 7 years ago

Hrm. Curiouser and curiouser! I'll poke around some more here and see if I can twiddle any knobs that might make it more consistent locally. I'll close this, but update it with anything of interest.

larryhynes commented 7 years ago

I'm stumped. Here's a recent session:

$ mlist .mail/logs/ | mpick :S
.mail/logs//cur/1490538571.34923_0.sutro.larryhynes.com:2,ST
.mail/logs//cur/1490538572.34927_0.sutro.larryhynes.com:2,ST
.mail/logs//cur/1490599359.38154_0.sutro.larryhynes.com:2,ST
.mail/logs//cur/1490599360.38155_0.sutro.larryhynes.com:2,ST
.mail/logs//cur/1490599363.38162_0.sutro.larryhynes.com:2,ST
.mail/logs//cur/1490599364.38163_0.sutro.larryhynes.com:2,ST
.mail/logs//cur/1490599367.38179_0.sutro.larryhynes.com:2,ST
.mail/logs//cur/1490621617.72838_0.sutro.larryhynes.com:2,ST
.mail/logs//cur/1490636797.97654_0.sutro.larryhynes.com:2,ST
.mail/logs//cur/1490655671.30102_0.sutro.larryhynes.com:2,ST
.mail/logs//cur/1490670255.54261_0.sutro.larryhynes.com:2,ST
.mail/logs//cur/1490670256.54263_0.sutro.larryhynes.com:2,ST
.mail/logs//cur/1490691433.86409_0.sutro.larryhynes.com:2,ST
.mail/logs//cur/1490707815.12383_0.sutro.larryhynes.com:2,ST
.mail/logs//cur/1490724452.39777_0.sutro.larryhynes.com:2,ST
.mail/logs//cur/1490743006.68936_0.sutro.larryhynes.com:2,ST
.mail/logs//cur/1490777528.22147_0.sutro.larryhynes.com:2,ST
.mail/logs//cur/1490777528.22148_0.sutro.larryhynes.com:2,ST
.mail/logs//cur/1490777537.22177_0.sutro.larryhynes.com:2,ST
.mail/logs//cur/1490777538.22181_0.sutro.larryhynes.com:2,ST
.mail/logs//cur/1490792589.47480_0.sutro.larryhynes.com:2,ST
.mail/logs//cur/1490794453.50531_0.sutro.larryhynes.com:2,ST
22 mails tested, 22 picked.

$ mlist .mail/logs/ | mpick :T
22 mails tested, 0 picked.

$ mlist .mail/logs/ | mpick :S
22 mails tested, 0 picked.
larryhynes commented 7 years ago

And here's another one, where i was playing around with stripping off the domain part, just to see if that was confusing anything (it wasn't).

$ mlist ob | mpick :T
ob/cur/1490806929.71524_0.sutro:2,ST
1 mails tested, 1 picked.

$ mlist ob | mpick :S
1 mails tested, 0 picked.

$ mlist ob | mpick :T
1 mails tested, 0 picked.

(I also tried building against pcre, and not regex.h to see if that did anything; it didn't. I am clutching - merrily - at straws.)

larryhynes commented 7 years ago

Making a sequence and operating on that has a similar effect i.e. :T works first, then doesn't

$ mlist ob | mseq -S

 $ mscan
 x  1   Wed  17:57 Stefan Sperling     Re: WPA2 dhcp fails on iwi after 3/1/17 security fix (#018)
1 mails scanned

$ mpick :T
ob/cur/1:2,ST
1 mails tested, 1 picked.

$ mpick :S
1 mails tested, 0 picked.

$ mpick :T
1 mails tested, 0 picked.
larryhynes commented 7 years ago

Clear all flags, flag it - as T only, and try to pick: :T doesn't work, -t trashed does, then doesn't.

$ mflag -st 1
ob/cur/1:2,

$ mlist ob
ob/cur/1:2,

$ mlist ob | mseq -S | mscan
 .  1   Wed  17:57 Stefan Sperling     Re: WPA2 dhcp fails on iwi after 3/1/17 security fix (#018)
1 mails scanned

$ mflag -T 1
ob/cur/1:2,T

$ mlist ob | mpick :T
1 mails tested, 0 picked.

$ mlist ob | mpick -t 'trashed'
ob/cur/1:2,T
1 mails tested, 1 picked.

$ mlist ob | mpick -t 'trashed'
1 mails tested, 0 picked.
larryhynes commented 7 years ago

Came back to my 'test' maildir today:

$ mlist ob | mpick :T
ob/cur/1:2,T
1 mails tested, 1 picked.

$ mlist ob | mpick :T
1 mails tested, 0 picked.

$ mlist ob | mpick -t 'trashed'
ob/cur/1:2,T
1 mails tested, 1 picked.

$ mlist ob | mpick -t 'trashed'
1 mails tested, 0 picked.

(In each case the 0 picked result will stay for subsequent retries - until I leave it overnight and come back to it, that is!)

I might try to hack together a minimum working example of mpick, that just operates on trashed / T to see if I can possibly isolate what the hell is going on.

Duncaen commented 7 years ago

We worked on a small test suit, I used travis to run tests on OS X and your problem reproducible. https://travis-ci.org/Duncaen/mblaze/jobs/216795380 Not sure yet what the problem is, but it will be fixed.

larryhynes commented 7 years ago

That's great, Duncan, thanks.

Duncaen commented 7 years ago

It should be fixed now with 2ed0a0ea6032ce8d47bf1ab71c1641a731fcf858. This was a stupid case of uninitialized memory, it checks if extra is set and changes the matching behavior based on it. On Linux and OpenBSD it seems like it was zero most of the time, if not every time. OS X did not zero the memory and that's why it was randomly failing.

Thanks for testing mpick and reporting the issues.

larryhynes commented 7 years ago

Thanks yourself, for the hard work and effort, Duncan; I've pulled this locally now and it's working as expected.