search

kannibalox / pyrosimple

An overhauled fork of the pyrocore tools for rTorrent
https://kannibalox.github.io/pyrosimple/
GNU General Public License v3.0
48 stars 5 forks source link

Custom field doesn't return results with another condition #28

Closed BristleBeard closed 1 year ago

BristleBeard commented 1 year ago

When I run rtcontrol custom_1=tvshows I get output like I'd expect. If I run rtcontrol completed=+2w I also get output as expected. However when I combine these two using rtcontrol custom_1=tvshows completed=+2w I get no results.

I believe the syntax is correct on combining the two conditions, but anytime I use custom_1 and an additional conditional it returns no results.

kannibalox commented 1 year ago

Since the two conditions are ANDed together by default, it's entirely possible that nothing actually matches both at once. custom_1=tvshows OR completed=+2w would let you do an OR if that's what you're looking for. If that's not the case, can you run this command and paste an entry or two here that should also match custom_1=tvshows completed=+2w?

rtcontrol completed=+2w -o custom_1,completed.delta,completed.raw
BristleBeard commented 1 year ago

I absolutely should have matches for both at once, I double checked that! ;) When I ran rtcontrol completed=+2w -o custom_1,completed.delta,completed.raw I did not get any results back at all.

kannibalox commented 1 year ago

Hm I'm a little confused, the only difference between rtcontrol completed=+2w and rtcontrol completed=+2w -o custom_1,completed.delta,completed.raw is the output format, but the former gives the expected output while the latter doesn't?

I did find a hopefully unrelated bug with prefiltering on completed when fast_query is enabled, I created #29 to track it.

BristleBeard commented 1 year ago

Actually after a little playing around I'm finding that rtcontrol completed=+2w also doesn't give me any output, and it definitely should. If I use rtcontrol seedtime=+2w then I get results, but not with completed.

I should've mentioned earlier my versions as well. rtcontrol 2.6.1 on Python 3.9.16

kannibalox commented 1 year ago

That's very strange, since seedtime uses calculations based on completed to figure itself out. Can you post a couple lines of output from rtcontrol // -o seedtime.delta,seedtime.raw,completed.delta,completed.raw for items that you'd expect to match completed=+2w

BristleBeard commented 1 year ago

This output looks a little odd. An enormous amount is coming back w/ basically no data, the N/A None lines reflect that. Sorry the formatting is a little off. I'm having trouble getting the spacing to behave.

2728w 1d ago 25731276.0 64w ago 1637019990 2648w 1d ago 74133382.0 237w 6d ago 1531877567 N/A None never 0 N/A None never 0 N/A None never 0 2662w 5d ago 65252749.0 216w 3d ago 1544836569 N/A None never 0 2723w 2d ago 28668651.0 82w 6d ago 1625601536 2753w 3d ago 10448368.0 32w 2d ago 1656132216 2658w ago 68154533.0 224w 2d ago 1540048278 2769w 5d ago 541740.0 16w ago 1666038844 2769w 5d ago 537539.0 16w ago 1666043045 2728w 1d ago 25731276.0 62w 4d ago 1637881631

kannibalox commented 1 year ago

No worries, that gives me enough to go off of.

kannibalox commented 1 year ago

Try giving the latest commit a shot, should be be fixed now.

BristleBeard commented 1 year ago

Sorta fixed. rtcontrol completed+=2w style commands work again, however if I combine them with the custom field, nothing is returned again. ex. rtcontrol custom_1=tvshows completed=+2w

kannibalox commented 1 year ago

Can you post a couple lines from rtcontrol completed=+2w -o custom_1.raw,completed.raw that you think should be matching the combined query?

BristleBeard commented 1 year ago

I have 423 that are completely currently. Approximately 268 of those were completed over 2 weeks ago. I have 7 different labels applied across all of those.

When I run that command, I see some odd things, only 55 results are returned and those results contain a mix of only 2 labels. The number of results is very suspicious because it matches exactly the number of torrents that are attached to a specific label (misc).

kannibalox commented 1 year ago

Try running this command to make sure pyrosimple on the latest commit:

pip install git+https://github.com/kannibalox/pyrosimple.git -U --force-reinstall

I'm not able to reproduce it locally, so if the issue persists I'll need specific examples.

BristleBeard commented 1 year ago

Same results I'm afraid. How can I give you specific examples? I tried a rtcontrol is_complete -o custom_1.raw,completed.raw to pick some that aren't showing up but it gives this error: WARNING:pyrosimple.torrent.rtorrent.RtorrentEngine:Method 'strings.contains_i' does not exist!Fast query 'string.contains_i=$d.name=,"is_complete"' would return an empty list, disabling fast query.

kannibalox commented 1 year ago

Try is_complete=yes, otherwise rtcontrol thinks you're looking for a torrent with the literal name is_complete, but yes that would be a good way to get examples.

The warning is just a heads up that fast_query won't work properly with strings on your rTorrent version, so it's been disabled. It doesn't effect the results, just the speed.

BristleBeard commented 1 year ago

I should've realized that field need a yes/no! Using rtcontrol is_complete=yes -o seedtime.delta,seedtime.raw,completed.delta,completed.raw,custom_1.raw,name I get this line back for it:

2648w 2d ago 74133262.0 238w ago 1531877687 misc exmaplefilename.pdf N/A None never 0 tvshows examplefilename2.pdf

The first line is an example that is getting returned by completed=+2w. The seconds line should be getting returned but isn't. This particular torrent completed 9 weeks, 4 days ago. It looks like the first line has an additional column of data in it?

kannibalox commented 1 year ago

Basically it looks like the tm_completed custom field isn't set for some torrents, which means completed won't be able to match against them. I'm not sure how you're determining when they're completed, but you could do some quick scripting to fill them. pyroadmin has a basic command to backfill it based on the modification time of the path if that's useful: pyroadmin backfill --dry-run (it'll also try to fix any missing loaded values). If you try pyroadmin, I just pushed up some small fixes, so you'll want to install the latest commit again.

BristleBeard commented 1 year ago

That fixed everything that I can tell. I'm curious though what would be causing the missing data? Looking over the torrents that didn't need the backfill, they were all very old. I'm missing data on brand new ones as well.

kannibalox commented 1 year ago

Has your rtorrent.rc changed recently? The lines mentioned in https://kannibalox.github.io/pyrosimple/rtorrent-config/#timestamps are required for a bunch of the timestamp fields. It's also rare but possible for things like fast_resumes and backup restores to bypass the event.download.finished event.

BristleBeard commented 1 year ago

It has not, however I switched over to your fork when my provider killed python2 support. I was scrambling and missed ALL of that. Most likely things kept working well for a while, but as timed marched on the problem got worse and worse.

kannibalox commented 1 year ago

Glad to hear the issue got solved, I'm going to close this but feel free to ask away if you have any other questions.