search

jamesmeneghello / pynab

Newznab-compliant Usenet Indexer written in Python, using PostgreSQL/MySQL-like.
Other
209 stars 44 forks source link

t=tv tvsearch and movie are broken #281

Closed ghost closed 8 years ago

ghost commented 8 years ago

Sonarr uses the following for searching and testing before you can add a newznab source:

api?t=tvsearch&cat=5030,5040&extended=1&apikey=MYAPIKEY&offset=0&limit=100

This is broken as it returns no results. However, if you change this query to simply be t=search it returns results. There's something broken with the aliasing of tv, tvsearch, and movie to "search" function.

brookesy2 commented 8 years ago

How odd, Sonarr works fine for testing and searching here. What version of Sonarr are you using? I am on the dev branch for pynab though.

ghost commented 8 years ago

I'm on the latest release of Sonarr, 2.0.0.4146. I just changed to your dev branch and I have the same issue. This is very odd.

ghost commented 8 years ago

If I do a manual sql query for category 5030 I get 795 rows, so the database is populated and the releases are correctly categorized:

pynab=# select count(*) from releases where category_id = '5030';
count 
-------
   795
(1 row)
ghost commented 8 years ago

manual api queries have no results:

$ curl "http://172.16.1.124/api?t=tvsearch&cat=5030,5040&extended=1&apikey=344088558018b88c56ea624e462ec6be&offset=0&limit=100"
<?xml version="1.0" encoding="UTF-8" ?>
<rss version="2.0" xmlns:newznab="http://www.newznab.com/DTD/2010/feeds/attributes/">
<channel>
    <title>pynab</title>
    <description>a pynab api</description>
    <link>http://172.16.1.124</link>
    <newznab:response offset="0" total="0"/>
    </channel> 
</rss>
brookesy2 commented 8 years ago

Using the built in web server or uwsgi with apache/nginx?

ghost commented 8 years ago

I originally was using the built-in but I'm now using uwsgi with nginx. Same results.

brookesy2 commented 8 years ago

Same sonarr and setup as me. Will hopefully be able to take a look this week. Not an area I am super familiar with though!

ghost commented 8 years ago

Here's an example of a working query when just using "search":

$ curl "http://172.16.1.124/api?t=search&cat=5030&apikey=344088558018b88c56ea624e462ec6be&limit=2"
<?xml version="1.0" encoding="UTF-8" ?>
<rss version="2.0" xmlns:newznab="http://www.newznab.com/DTD/2010/feeds/attributes/">
<channel>
    <title>pynab</title>
    <description>a pynab api</description>
    <link>http://172.16.1.124</link>
    <newznab:response offset="0" total="2"/>

        <item>
            <title>Antiques Roadshow UK S38E26 HDTV x264 C4TV</title>
            <guid isPermaLink="true">http://172.16.1.124/details/3890</guid>
            <link>http://172.16.1.124/api?t=g&amp;guid=3890&amp;apikey=344088558018b88c56ea624e462ec6be</link>
            <pubDate>Mon, 16 May 2016 13:26:31 -0000</pubDate>
            <category>TV &gt; SD</category>
            <description>Antiques Roadshow UK S38E26 HDTV x264 C4TV</description>
            <posted>Mon, 16 May 2016 20:00:18 -0000</posted>
            <group>alt.binaries.teevee</group>
            <enclosure url="http://172.16.1.124/api?t=g&amp;guid=3890&amp;apikey=344088558018b88c56ea624e462ec6be" length="562740132" type="application/x-nzb"></enclosure>
            <grabs>0</grabs>

            <newznab:attr name="category" value="5030"/>
            <newznab:attr name="category" value="5000"/>
            <newznab:attr name="guid" value="3890"/>
            <newznab:attr name="poster" value="provide@4u.net (yeahsure)"/>
            <newznab:attr name="usenetdate" value="Mon, 16 May 2016 20:00:18 -0000"/>
            <newznab:attr name="grabs" value="0"/>
            <newznab:attr name="group" value="alt.binaries.teevee"/>
            <newznab:attr name="size" value="562740132"/>
        <size>562740132</size>
        </item>

        <item>
            <title>Burmas Secret Jungle War With Joe Simpson S01E02 HDTV x264 C4TV</title>
            <guid isPermaLink="true">http://172.16.1.124/details/3861</guid>
            <link>http://172.16.1.124/api?t=g&amp;guid=3861&amp;apikey=344088558018b88c56ea624e462ec6be</link>
            <pubDate>Mon, 16 May 2016 13:20:55 -0000</pubDate>
            <category>TV &gt; SD</category>
            <description>Burmas Secret Jungle War With Joe Simpson S01E02 HDTV x264 C4TV</description>
            <posted>Mon, 16 May 2016 19:48:48 -0000</posted>
            <group>alt.binaries.teevee</group>
            <enclosure url="http://172.16.1.124/api?t=g&amp;guid=3861&amp;apikey=344088558018b88c56ea624e462ec6be" length="828734659" type="application/x-nzb"></enclosure>
            <grabs>0</grabs>

            <newznab:attr name="category" value="5030"/>
            <newznab:attr name="category" value="5000"/>
            <newznab:attr name="guid" value="3861"/>
            <newznab:attr name="poster" value="canuck@su.ck (ilikeboys)"/>
            <newznab:attr name="usenetdate" value="Mon, 16 May 2016 19:48:48 -0000"/>
            <newznab:attr name="grabs" value="0"/>
            <newznab:attr name="group" value="alt.binaries.teevee"/>
            <newznab:attr name="size" value="828734659"/>
        <size>828734659</size>
        </item>
    </channel>
</rss>
brookesy2 commented 8 years ago

Just did a quick curl with your original url, 100% getting results.

        <item>
            <title>Flophouse S01E02 720p HDTV x264 YesTV</title>
            <guid isPermaLink="true">http://xxx/details/214136</guid>
            <link>http://localhost:6667/api?t=g&amp;guid=214136&amp;apikey=xxx</link>
            <pubDate>Mon, 16 May 2016 00:39:29 -0000</pubDate>
            <category>TV &gt; HD</category>
            <description>Flophouse S01E02 720p HDTV x264 YesTV</description>
            <posted>Mon, 16 May 2016 00:57:51 -0000</posted>
            <group>alt.binaries.teevee</group>
            <enclosure url="http://xxx/api?t=g&amp;guid=214136&amp;apikey=xxx" length="534908721" type="application/x-nzb"></enclosure>
            <grabs>0</grabs>

            <newznab:attr name="category" value="5040"/>
            <newznab:attr name="category" value="5000"/>
            <newznab:attr name="TVMAZE" value="13233"/>
            <newznab:attr name="guid" value="214136"/>
            <newznab:attr name="poster" value="provide@4u.net (yeahsure)"/>
            <newznab:attr name="usenetdate" value="Mon, 16 May 2016 00:57:51 -0000"/>
            <newznab:attr name="grabs" value="0"/>
            <newznab:attr name="group" value="alt.binaries.teevee"/>
            <newznab:attr name="size" value="534908721"/>
            <size>534908721</size>
        </item>
    </channel>
</rss>
brookesy2 commented 8 years ago

Both search and tvsearch return results :(

ghost commented 8 years ago

I enabled full query logging with postgres and hit this URL:

curl "http://172.16.1.124/api?t=tvsearch&cat=5030,5040&extended=1&apikey=344088558018b88c56ea624e462ec6be&offset=0&limit=100"

LOG:  statement: BEGIN
LOG:  statement: SELECT 1
LOG:  statement: SELECT users.id AS users_id, users.api_key AS users_api_key, users.email AS users_email, users.grabs AS users_grabs 
        FROM users 
        WHERE users.api_key = '344088558018b88c56ea624e462ec6be' 
         LIMIT 1
LOG:  statement: COMMIT
LOG:  statement: BEGIN
LOG:  statement: SELECT 1
LOG:  statement: SELECT count(*) AS count_1 
        FROM (SELECT releases.id AS releases_id, releases.uniqhash AS releases_uniqhash, releases.added AS releases_added, releases.posted AS releases_posted, releases.name AS releases_name, releases.search_name AS releases_search_name, releases.original_name AS releases_original_name, releases.posted_by AS releases_posted_by, releases.status AS releases_status, releases.grabs AS releases_grabs, releases.size AS releases_size, releases.passworded AS releases_passworded, releases.unwanted AS releases_unwanted, releases.group_id AS releases_group_id, releases.category_id AS releases_category_id, releases.regex_id AS releases_regex_id, releases.tvshow_id AS releases_tvshow_id, releases.tvshow_metablack_id AS releases_tvshow_metablack_id, releases.movie_id AS releases_movie_id, releases.movie_metablack_id AS releases_movie_metablack_id, releases.nzb_id AS releases_nzb_id, releases.rar_metablack_id AS releases_rar_metablack_id, releases.nfo_id AS releases_nfo_id, releases.nfo_metablack_id AS releases_nfo_metablack_id, releases.sfv_id AS releases_sfv_id, releases.sfv_metablack_id AS releases_sfv_metablack_id, releases.episode_id AS releases_episode_id, releases.pre_id AS releases_pre_id 
        FROM releases JOIN tvshows ON tvshows.id = releases.tvshow_id JOIN categories ON categories.id = releases.category_id 
        WHERE categories.id IN (5000) OR categories.parent_id IN (5000) ORDER BY releases.posted DESC 
         LIMIT 100 OFFSET 0) AS anon_1
LOG:  statement: SELECT releases.id AS releases_id, releases.uniqhash AS releases_uniqhash, releases.added AS releases_added, releases.posted AS releases_posted, releases.name AS releases_name, releases.search_name AS releases_search_name, releases.original_name AS releases_original_name, releases.posted_by AS releases_posted_by, releases.status AS releases_status, releases.grabs AS releases_grabs, releases.size AS releases_size, releases.passworded AS releases_passworded, releases.unwanted AS releases_unwanted, releases.group_id AS releases_group_id, releases.category_id AS releases_category_id, releases.regex_id AS releases_regex_id, releases.tvshow_id AS releases_tvshow_id, releases.tvshow_metablack_id AS releases_tvshow_metablack_id, releases.movie_id AS releases_movie_id, releases.movie_metablack_id AS releases_movie_metablack_id, releases.nzb_id AS releases_nzb_id, releases.rar_metablack_id AS releases_rar_metablack_id, releases.nfo_id AS releases_nfo_id, releases.nfo_metablack_id AS releases_nfo_metablack_id, releases.sfv_id AS releases_sfv_id, releases.sfv_metablack_id AS releases_sfv_metablack_id, releases.episode_id AS releases_episode_id, releases.pre_id AS releases_pre_id 
        FROM releases JOIN tvshows ON tvshows.id = releases.tvshow_id JOIN categories ON categories.id = releases.category_id 
        WHERE categories.id IN (5000) OR categories.parent_id IN (5000) ORDER BY releases.posted DESC 
         LIMIT 100 OFFSET 0
LOG:  statement: COMMIT

I'd be curious to know if you have the same query structure. It does bother me that it's hijacking the request for category 5030 and 5040 and replacing them with IN (5000), which is what it appears it is supposed to be doing according to pynab/api.py line 139:

cat_ids.append(5000)
ghost commented 8 years ago

And for a final point of reference, here's this API query with search instead of tvsearch:

curl "http://172.16.1.124/api?t=search&cat=5030,5040&apikey=344088558018b88c56ea624e462ec6be&limit=2"

LOG:  statement: BEGIN
LOG:  statement: SELECT 1
LOG:  statement: SELECT users.id AS users_id, users.api_key AS users_api_key, users.email AS users_email, users.grabs AS users_grabs 
        FROM users 
        WHERE users.api_key = '344088558018b88c56ea624e462ec6be' 
         LIMIT 1
LOG:  statement: COMMIT
LOG:  statement: BEGIN
LOG:  statement: SELECT 1
LOG:  statement: SELECT count(*) AS count_1 
        FROM (SELECT releases.id AS releases_id, releases.uniqhash AS releases_uniqhash, releases.added AS releases_added, releases.posted AS releases_posted, releases.name AS releases_name, releases.search_name AS releases_search_name, releases.original_name AS releases_original_name, releases.posted_by AS releases_posted_by, releases.status AS releases_status, releases.grabs AS releases_grabs, releases.size AS releases_size, releases.passworded AS releases_passworded, releases.unwanted AS releases_unwanted, releases.group_id AS releases_group_id, releases.category_id AS releases_category_id, releases.regex_id AS releases_regex_id, releases.tvshow_id AS releases_tvshow_id, releases.tvshow_metablack_id AS releases_tvshow_metablack_id, releases.movie_id AS releases_movie_id, releases.movie_metablack_id AS releases_movie_metablack_id, releases.nzb_id AS releases_nzb_id, releases.rar_metablack_id AS releases_rar_metablack_id, releases.nfo_id AS releases_nfo_id, releases.nfo_metablack_id AS releases_nfo_metablack_id, releases.sfv_id AS releases_sfv_id, releases.sfv_metablack_id AS releases_sfv_metablack_id, releases.episode_id AS releases_episode_id, releases.pre_id AS releases_pre_id 
        FROM releases JOIN categories ON categories.id = releases.category_id 
        WHERE categories.id IN ('5030', '5040') OR categories.parent_id IN ('5030', '5040') ORDER BY releases.posted DESC 
         LIMIT 2 OFFSET 0) AS anon_1
LOG:  statement: SELECT releases.id AS releases_id, releases.uniqhash AS releases_uniqhash, releases.added AS releases_added, releases.posted AS releases_posted, releases.name AS releases_name, releases.search_name AS releases_search_name, releases.original_name AS releases_original_name, releases.posted_by AS releases_posted_by, releases.status AS releases_status, releases.grabs AS releases_grabs, releases.size AS releases_size, releases.passworded AS releases_passworded, releases.unwanted AS releases_unwanted, releases.group_id AS releases_group_id, releases.category_id AS releases_category_id, releases.regex_id AS releases_regex_id, releases.tvshow_id AS releases_tvshow_id, releases.tvshow_metablack_id AS releases_tvshow_metablack_id, releases.movie_id AS releases_movie_id, releases.movie_metablack_id AS releases_movie_metablack_id, releases.nzb_id AS releases_nzb_id, releases.rar_metablack_id AS releases_rar_metablack_id, releases.nfo_id AS releases_nfo_id, releases.nfo_metablack_id AS releases_nfo_metablack_id, releases.sfv_id AS releases_sfv_id, releases.sfv_metablack_id AS releases_sfv_metablack_id, releases.episode_id AS releases_episode_id, releases.pre_id AS releases_pre_id 
        FROM releases JOIN categories ON categories.id = releases.category_id 
        WHERE categories.id IN ('5030', '5040') OR categories.parent_id IN ('5030', '5040') ORDER BY releases.posted DESC 
         LIMIT 2 OFFSET 0
LOG:  statement: SELECT categories.id AS categories_id, categories.name AS categories_name, categories.parent_id AS categories_parent_id 
        FROM categories 
        WHERE categories.id = 5040
LOG:  statement: SELECT categories.id AS categories_id, categories.name AS categories_name, categories.parent_id AS categories_parent_id 
        FROM categories 
        WHERE categories.id = 5000
LOG:  statement: SELECT groups.id AS groups_id, groups.active AS groups_active, groups.first AS groups_first, groups.last AS groups_last, groups.name AS groups_name 
        FROM groups 
        WHERE groups.id = 82
LOG:  statement: COMMIT

Notice it correctly handles &cat=5030,5040 ?

gkoh commented 8 years ago

I just tried the t=tvsearch query against my instance and definitely get results.

ghost commented 8 years ago

I'm on FreeBSD and using the python modules from OS packages, not PIP. I had to create a couple packages for missing python modules but I've met the requirements.txt. Of course since I'm not using pip my install is a bit different than yours. Here is the output of all installed packages on my system. Please browse the list and let me know if you see anything that stands out. Perhaps I have something slightly newer or older than your typical install. Do note that I have played with using different versions of SQLAlchemy without any difference. I have not yet played with the version of psycopg2, but that is my next suspicion since something seems awry with the SQL queries.

# pkg info
binutils-2.25.1_2,1            GNU binary tools
blas-3.5.0_3                   Basic Linear Algebra Subroutines
ca_root_nss-3.22.2             Root certificate bundle from the Mozilla Project
cblas-1.0_4                    Reference implementation of the C interface to the legacy Fortran BLAS
ccache-3.2.5                   Tool to minimize the compile time of C/C++ programs
curl-7.48.0_2                  Non-interactive tool to get files from FTP, GOPHER, HTTP(S) servers
expat-2.1.0_3                  XML 1.0 parser written in C
fontconfig-2.11.1_2,1          XML-based font configuration API for X Windows
freetype2-2.6.3                Free and portable TrueType font rendering engine
gcc-4.8.5_2                    GNU Compiler Collection 4.8
gcc-ecj-4.5                    Eclipse Java Compiler used to build GCC Java
gettext-runtime-0.19.7         GNU gettext runtime libraries and programs
git-lite-2.7.4_1               Distributed source code management tool (lite package)
gmp-5.1.3_3                    Free library for arbitrary precision arithmetic
indexinfo-0.2.4                Utility to regenerate the GNU info page index
kbproto-1.0.7                  KB extension headers
lapack-3.5.0                   A library of Fortran 77 subroutines for linear algebra
libX11-1.6.3,1                 X11 library
libXScrnSaver-1.2.2_3          The XScrnSaver library
libXau-1.0.8_3                 Authentication Protocol library for X11
libXdmcp-1.1.2                 X Display Manager Control Protocol library
libXext-1.3.3_1,1              X11 Extension library
libXft-2.3.2_1                 Client-sided font API for X applications
libXrender-0.9.9               X Render extension library
libffi-3.2.1                   Foreign Function Interface
libgcrypt-1.7.0                General purpose crypto library based on code used in GnuPG
libgpg-error-1.22              Common error values for all GnuPG components
libpthread-stubs-0.3_6         This library provides weak aliases for pthread functions
libxcb-1.11.1                  The X protocol C-language Binding (XCB) library
libxml2-2.9.3                  XML parser library for GNOME
libxslt-1.1.28_8               The XSLT C library for GNOME
mpc-1.0.3                      Library of complex numbers with arbitrarily high precision
mpfr-3.1.4                     Library for multiple-precision floating-point computations
nginx-1.10.0_3,2               Robust and small WWW server
openblas-0.2.18,1              Optimized BLAS library based on GotoBLAS2
openssl-1.0.2_12               SSL and crypto library
p5-Authen-SASL-2.16_1          Perl5 module for SASL authentication
p5-Digest-HMAC-1.03_1          Perl5 interface to HMAC Message-Digest Algorithms
p5-Error-0.17024               Error/exception handling in object-oriented programming style
p5-GSSAPI-0.28_1               Perl extension providing access to the GSSAPIv2 library
p5-IO-Socket-IP-0.37           Drop-in replacement for IO::Socket::INET supporting IPv4 and IPv6
p5-IO-Socket-SSL-2.024         Perl5 interface to SSL sockets
p5-Mozilla-CA-20160104         Perl extension for Mozilla CA cert bundle in PEM format
p5-Net-SMTP-SSL-1.03           SSL support for Net::SMTP
p5-Net-SSLeay-1.72             Perl5 interface to SSL
p5-Socket-2.021                Networking constants and support functions
pcre-8.38_1                    Perl Compatible Regular Expressions library
perl5-5.20.3_13                Practical Extraction and Report Language
pkg-1.7.2                      Package manager
postgresql93-client-9.3.12     PostgreSQL database (client)
py27-meld3-0.6.8_1             HTML/XML templating system for Python
py27-setuptools27-20.0         Python packages installer
py27-supervisor-3.0_1,1        System to monitor and control a number of processes on UNIX-like OS
py34-Babel-2.3.3               Collection of tools for internationalizing Python applications
py34-MarkupSafe-0.23           Implements XML/HTML/XHTML Markup safe string for Python
py34-Pympler-0.4.2             Measure, monitor and analyze the memory behavior of Python objects
py34-alembic-0.8.6             Database migration tool for SQLAlchemy
py34-beaker-1.8.0_1            Session and Caching library with WSGI Middleware
py34-beautifulsoup-4.4.1       HTML/XML Parser for Python
py34-bottle-0.12.7             Fast and simple WSGI-framework for small web-applications
py34-bottleneck-1.0.0          Collection of fast NumPy array functions written in Cython
py34-colorama-0.3.3            Cross-platform API to print colored terminal text from Python apps
py34-colorlog-2.6.1            Python module for log formatting with colors
py34-daemonize-2.4.2           Enables code to run as a daemon process on Unix-like systems
py34-dateutil-2.5.0            Extensions to the standard Python datetime module
py34-docopt-0.6.2              Pythonic argument parser, that will make you smile
py34-eventlet-0.18.4           Concurrent networking library for Python
py34-funcsigs-0.4              Python function signatures from PEP362
py34-greenlet-0.4.9            Light-weight microthreads for Python
py34-intspan-1.5.2             FInite element Automatic Tabulator
py34-irc-13.1                  IRC protocol client library for Python
py34-jaraco.timing-1.1.5       Routines pertinent to the fourth dimension
py34-jaraco.util-11.0          General utility modules that supply commonly-used functionality
py34-libxml2-2.9.3             Python interface for XML parser library for GNOME
py34-lxml-3.5.0                Pythonic binding for the libxml2 and libxslt libraries
py34-mako-1.0.4                Super-fast templating language in Python
py34-more-itertools-2.2        More routines for operating on iterables, beyond itertools
py34-nltk-3.2.1,2              Natural language toolkit written in python
py34-numexpr-2.5.2             Fast numerical array expression evaluator for Python and NumPy
py34-numpy-1.11.0,1            The New Numeric Extension to Python
py34-pandas-0.17.0             Flexible, high-performance data analysis in Python
py34-psutil-4.1.0              Process utilities module for Python
py34-psycopg2-2.6.1_1          High performance Python adapter for PostgreSQL
py34-pySmartDL-1.2.4           Smart download manager for Python
py34-pyhashxx-0.1.3            Python wrapper for xxHash algorithm
py34-pymongo-3.2.2             Python driver for MongoDB
py34-pymysql-0.7.2             Pure Python MySQL Driver
py34-pynzb-0.1.0_2             Unified API for parsing NZB files
py34-python-editor-0.5         Python interface to your system's EDITOR
py34-pytvmaze-1.5.1            Python interface to the TV Maze API
py34-pytz-2016.4,1             World Timezone Definitions for Python
py34-regex-2016.02.25          Alternative regular expression module, to replace re
py34-requests-2.9.1            HTTP library written in Python for human beings
py34-requests-futures-0.9.7    HTTP library written in Python for human beings
py34-roman-1.4.0_1             Integer to Roman numerals converter
py34-setuptools34-20.0         Python packages installer
py34-setuptools_scm-1.10.1     Setuptools plugin to manage your versions by scm tags
py34-six-1.10.0                Python 2 and 3 compatibility utilities
py34-sleekxmpp-1.3.1_1         XMPP Library
py34-sqlalchemy-1.0.12         Python SQL toolkit and Object Relational Mapper
py34-sqlite3-3.4.4_7           Standard Python binding to the SQLite3 library (Python 3.4)
py34-tkinter-3.4.4_6           Python bindings to the Tk widget set (Python 3.4)
py34-xmltodict-0.9.2           XML parser that returns python dictionaries
py34-yaml-3.11                 Python YAML parser
python2-2_3                    The "meta-port" for version 2 of the Python interpreter
python27-2.7.11_2              Interpreted object-oriented programming language
python3-3_3                    The "meta-port" for version 3 of the Python interpreter
python34-3.4.4_2               Interpreted object-oriented programming language
renderproto-0.11.1             RenderProto protocol headers
scrnsaverproto-1.2.2           ScrnSaver extension headers
sqlite3-3.12.2                 SQL database engine in a C library
suitesparse-4.0.2_4            SuiteSparse is a set of packages for sparse matrices calculation
tcl86-8.6.5_1                  Tool Command Language
tk86-8.6.5                     Graphical toolkit for Tcl
unrar-5.31,5                   Extract, view & test RAR archives
uwsgi-2.0.13                   Developer-friendly WSGI server which uses uwsgi protocol
xextproto-7.3.0                XExt extension headers
xproto-7.0.28                  X11 protocol headers
gkoh commented 8 years ago

Good to see another non-Linux user! My configuration is more esoteric than yours, but actually not that different. I run on a Solaris derivative (OmniOS), using mostly pkgsrc packages with the remainder filled in by pip.

I'm running the same version of python (3.4) and the same version of psycopg2 (2.6.1). I was running PostgreSQL 9.3 recently before upgrading to PostgreSQL 9.4, but have not seen this yet.

Is this search the only one that is broken? Or do other searches exhibit similar oddity? Also, just to confirm, when you say no results, I assume you get the standard template with no results (likely the query or data is awry as you suspect) as opposed to an empty page of nothing (likely the API hit an error)?

A little more, I ran the SQL query on my DB from your previous comment (as per dumped by the full query log for t=tvsearch) and it found 100 entries as expected. Does it do the same for you?

ghost commented 8 years ago

Correct, just the standard template with no results.

<?xml version="1.0" encoding="UTF-8" ?>
<rss version="2.0" xmlns:newznab="http://www.newznab.com/DTD/2010/feeds/attributes/">
<channel>
    <title>pynab</title>
    <description>a pynab api</description>
    <link>http://172.16.1.124</link>
    <newznab:response offset="0" total="0"/>
    </channel>
</rss>

I don't know what other queries to try, but everything I've tried with the api functions tv/tvseries/m/movie produce zero results.

I'm getting suspicious about whether other problems might exist because the nfos table is always empty and so are a couple others, but the nzbs and releases table have data and I can get results if I use the api function "search".

edit: episodes table stays empty too...

gkoh commented 8 years ago

I see now that you already mentioned that in a previous comment, I should read more carefully. It feels like there's something not right with the database or the postprocessing for reasons unknown.

If you manually run the SQL query as I did, what do you get? I get this:

# SELECT count(*) AS count_1 FROM (SELECT releases.id AS releases_id, releases.uniqhash AS releases_uniqhash, releases.added AS releases_added, releases.posted AS releases_posted, releases.name AS releases_name, releases.search_name AS releases_search_name, releases.original_name AS releases_original_name, releases.posted_by AS releases_posted_by, releases.status AS releases_status, releases.grabs AS releases_grabs, releases.size AS releases_size, releases.passworded AS releases_passworded, releases.unwanted AS releases_unwanted, releases.group_id AS releases_group_id, releases.category_id AS releases_category_id, releases.regex_id AS releases_regex_id, releases.tvshow_id AS releases_tvshow_id, releases.tvshow_metablack_id AS releases_tvshow_metablack_id, releases.movie_id AS releases_movie_id, releases.movie_metablack_id AS releases_movie_metablack_id, releases.nzb_id AS releases_nzb_id, releases.rar_metablack_id AS releases_rar_metablack_id, releases.nfo_id AS releases_nfo_id, releases.nfo_metablack_id AS releases_nfo_metablack_id, releases.sfv_id AS releases_sfv_id, releases.sfv_metablack_id AS releases_sfv_metablack_id, releases.episode_id AS releases_episode_id, releases.pre_id AS releases_pre_id FROM releases JOIN tvshows ON tvshows.id = releases.tvshow_id JOIN categories ON categories.id = releases.category_id WHERE categories.id IN (5000) OR categories.parent_id IN (5000) ORDER BY releases.posted DESC LIMIT 100 OFFSET 0) AS anon_1;
 count_1 
---------
     100
(1 row)
ghost commented 8 years ago

I just found it. Postprocess was broken because of roman, but wasn't logging any errors. This seems like a bug. I now have postprocess working correctly and results in tvsearch.

# python3
Python 3.4.4 (default, May  7 2016, 01:23:11) 
[GCC 4.2.1 Compatible FreeBSD Clang 3.4.1 (tags/RELEASE_34/dot1-final 208032)] on freebsd10
Type "help", "copyright", "credits" or "license" for more information.
>>> import roman
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.4/site-packages/roman.py", line 43
    raise OutOfRangeError, "number out of range (must be 1..4999)"
                         ^
SyntaxError: invalid syntax

The version of roman on my system was 1.4.0 which apparently builds with python 3.4 but is broken. Version 2.0.0 is required. I'll make a pull request to update requirements.txt to make it more obvious for people in similar situations (ie, not using pip)

gkoh commented 8 years ago

Nice find, I'm not surprised postprocess didn't log anything, it likely wouldn't have made it that far. I'd expect crash output like that is typically found in your OS's daemon logs or similar.

ghost commented 8 years ago

supervisor was only logging:

2016-05-16 13:21:10,401 INFO exited: postproc (exit status 1; not expected)
gkoh commented 8 years ago

I don't actually use supervisor myself, but reading over the docs, and assuming you only use it for pynab and there's no other configuration file, I'm expecting the stdout and stderr of the failed process to end up somewhere in /tmp (or whereever your python thinks the temp dir should be). Note that supervisor cleans up the /tmp logs on startup by default, so if you've successfully restarted it, it probably wiped the evidence for you.

brookesy2 commented 8 years ago

@feld Awesome find, thank you! :)

ghost commented 8 years ago

Closing this and considering it fixed.