Newznab support? - Githubissues

evenreven commented 11 years ago

I have a Newznab install that indexes comics, and it would be wonderful if Mylar could use this for searching for nzbs. Sick Beard, Couchpotato and Maraschino can do this already, so you might be able to nick some code from there.

Thank you for a really promising piece of software. I wish I knew enough Python to help out.

evilhero commented 11 years ago

Yep, it's on my to-do list, especially since alot of the main nzb sites have closed or restricted access. I had it working initially, but removed it before I had even done the first release, so it won't be a problem to readd it...expect an onslaught of commits to address this and several other outstanding issues in the next couple of days :)

evenreven commented 11 years ago

Nice! Looking forward to it. :D

Koire commented 11 years ago

Created an account to add my support! -- Using newznab+ on my home network right now and would love this feature, I just hope that the stock regex for releases will work with Mylar. -- For now I'm just pointing nzb.su to my localhost newznab.

bebetech commented 11 years ago

+1, this would be awesome. I'm about to set one up as well.

evilhero commented 11 years ago

Check out the development branch - newznab provider option is now added and working (at least with my newznab its working which might not mean much lol). Currently only one newznab provider can be given, but multiple newznab will be allowed in the near future...it was a quick code that I stole from headphones so I have to rework it abit for Mylar. If you're not sure how to switch branches do the following : Git pull Git checkout development

evenreven commented 11 years ago

That was fast! I'll test the dev branch later today. Thanks!

AtariBaby commented 11 years ago

I could add a newznab-based provider, if this thing is go. (NZBs.in)

hypyke commented 11 years ago

It lets you add more than one newznab provider but if you do it stops working against newznab altogether. Strange. It does seem to work if you only add one.

evilhero commented 11 years ago

I have 2 newznabs added and I'm not having any problems with the newznab portion as it searches both for me and I'm running the latest commit.

If you're having problems with the newznabs, can you paste your mylar.log file to show what the issue is and I can look at it - if I'm not having the problem, it's hard for me to figure things out without something to show me :)

hypyke commented 11 years ago

I'll have to re-add both of them to recreate the problem. I also noticed that without a trailing slash in the url the search urls fail of course.

On Mon, Jan 7, 2013 at 11:29 PM, evilhero notifications@github.com wrote:

I have 2 newznabs added and I'm not having any problems with the newznab portion as it searches both for me.

If you're having problems with the newznabs, can you paste your mylar.log file to show what the issue is and I can look at it - if I'm not having the problem, it's hard for me to figure things out without something to show me :)

— Reply to this email directly or view it on GitHubhttps://github.com/evilhero/mylar/issues/98#issuecomment-11984289.

hypyke commented 11 years ago

Ok it works. I think the trailing slash issue was causing all of my problems. I now have nzb.su, nzbs.in and two other private servers working. The only thing that is quirky is the appearance/disappearance of the category field that pops up when you add a new server. Not sure what that is about. Before adding

No category boxes.

After Adding

Category box is there until you save and go back in.

evilhero commented 11 years ago

Haha - yeah the category I threw in there, then obviously forgot about - it was meant to be in case the newznab provider didn't follow the typical category mapping for comics (newznab defaults to 7030 for comics) it would allow a user to properly map so they could use the newznab properly.

Prymal commented 11 years ago

I see the trailing slash issue as well, but I don't see what i need to edit to get a newznab instance searching properly.

evilhero commented 11 years ago

Make sure you have a trailing slash at the end of your newznab host address in Mylar. Not doing it will cause some issues because it will string the address with the api call and not having a slash to break it up is bad : http://newznab.r-us.orgapi? t= vs. http://newznab.r-us.org/api? t=

Will try to fix this later tonight..

llpowwowll commented 11 years ago

Was this ever fixed? My newznab local host used to work now it doesnt find anything...

evilhero commented 11 years ago

Nothing has changed in the newznab portion, so if you're having problems you need to be much more specific. What comics/issues? Do the comics exist on the provider? Is it occuring for all of the providers you have? Paste your mylar.log file showing the search/failed searches results to confirm exactly what's happening.

llpowwowll commented 11 years ago

I deleted the cache and it brought everything back...It seems to be windows issues I'm having. Im running my own local newznab plus server and alot of issues is as small as "\" or "/". I'm having fun and getting frustrated at the same time learning your f@#$ing awesome application. I think I have everything functioning for the mean time (knock on wood)...Just started fresh and readding new titles now. Thanks for all your hard work on this.

evilhero commented 11 years ago

From some previous discussions if you have sphinx running with newznab you'll get better results because of the searching power of sphinx.You'll get stuff you never hit on prior to having it running ;)

llpowwowll commented 11 years ago

OK I have been trying to setup sphinx with my local newznab plus and mylar with zero luck and its driving me crazy. Do you have any setup docs or links with some tutorials with newznab as the main focus of the sphinx search engine...Any help on this would be greatly appreciated.

alt3red commented 11 years ago

Try this:

http://newznab.readthedocs.org/en/latest/misc/sphinx/

kRAyZZed commented 11 years ago

I had sooo many issues doing this on Windows as well - as the guide is pretty shockingly poor on the newznab site - and completely overlooks windows lol.

You need to download Sphinx install it, but NOT as a service yet.

Follow the steps up to step 4 in the above guide, this will have created the config file for you - you need this config file to install sphinx as a service - as the default config file for sphinx is completely wrong.

You then need to load up the command prompt as administrator - and run the following command C:\Sphinx\bin> C:\Sphinx\bin\searchd --install --config C:\wherevertheconfigyoujustcreatedis --servicename sphinxsearch.

Then you can continue with the rest of the steps in the guide. There is a catch however...you will need to run your batch scripts as administrator going forward - or they cannot utilise sphinxsearch...so make sure you are running command prompt as admin when you are doing your updating scripts.

I hope that helps a little bit, I am kinda posting this from memory, as it took me so long to figure out that despite the guide saying in step 1 to install sphinx, that wasnt actually what I wanted.

Also make sure you dont have any other instances of sphinx running as services already - if you do you will have to remove them.

llpowwowll commented 11 years ago

kRAyZZed:

Can you post your contents of your sphinx.conf file? and this file is to be located in the (if using xampp which I am) path of C:\xampp\htdocs\nnplus\db\sphinxdata? Lastly is the file titled (with extension) sphinx.conf or sphinx.conf.in? sorry for noob questions just brand new to sphinx and newznab for that matter...

Oh yeah and just to be safe which version of sphinx did you use and what was your install path?

kRAyZZed commented 11 years ago

for me the file is just titled sphinx.conf - and I dont have my install in the default location - however the place you mentioned is correct if you are using the default localtion, with xampp.

And you just need to follow the steps in the guide that was posted by radjp to generate that config file - once I figured out that you had to use the config file they generated when installing as a service, and then following the instructions, things started to become plain sailing.

Below is a paste of my config file, but as I mentioned the php nnindexer.php generate command created that for me.

###############################################################################


indexer settings
###############################################################################
indexer
{
You can tweak these settings as you like and/or add your own
mem_limit = 256M
}
###############################################################################
searchd settings
###############################################################################
searchd
{
listen                  = localhost:9312
listen                  = localhost:9306:mysql41
read_timeout            = 5
client_timeout          = 300
max_children            = 30
pid_file                = C:\xampp\htdocs/db/sphinxdata/searchd.pid
log                     = C:\xampp\htdocs/db/sphinxdata/searchd.log
binlog_path             = C:\xampp\htdocs/db/sphinxdata/binlog
max_matches             = 10000
seamless_rotate         = 1
preopen_indexes         = 1
unlink_old              = 1
workers                 = threads
compat_sphinxql_magics  = 0
rt_flush_period         = 3600
}
###############################################################################
source definitions
###############################################################################
source base
{
type     = mysql
sql_host = localhost
sql_user = 
sql_pass = **
sql_db   = newznab
}
[---------------------------------------------------------------------------]
|       Don't modify below here unless you know what you are doing          |
[---------------------------------------------------------------------------]
source releases_base : base
{
Full-text fields - these are searchable
# releases
sql_field_string   = name
sql_field_string   = searchname
sql_field_string   = fromname
sql_field_string   = tvtitle
sql_field_string   = season
sql_field_string   = episode
sql_field_string   = guid
sql_field_string   = seriesfull
# bookinfo
sql_field_string   = bookinfo_title
sql_field_string   = bookinfo_author
sql_field_string   = bookinfo_publisher
sql_field_string   = bookinfo_review
sql_field_string   = bookinfo_asin
sql_field_string   = bookinfo_url
sql_field_string   = bookinfo_dewey
sql_field_string   = bookinfo_ean
sql_field_string   = bookinfo_isbn

# consoleinfo
sql_field_string   = consoleinfo_title
sql_field_string   = consoleinfo_publisher
sql_field_string   = consoleinfo_review
sql_field_string   = consoleinfo_platform
sql_field_string   = consoleinfo_esrb
sql_field_string   = consoleinfo_asin
sql_field_string   = consoleinfo_url

# episodeinfo
sql_field_string   = episodeinfo_showtitle
sql_field_string   = episodeinfo_eptitle
sql_field_string   = episodeinfo_director
sql_field_string   = episodeinfo_writer
sql_field_string   = episodeinfo_gueststars
sql_field_string   = episodeinfo_overview
sql_field_string   = episodeinfo_fullep
sql_field_string   = episodeinfo_link

# musicinfo
sql_field_string   = musicinfo_title
sql_field_string   = musicinfo_review
sql_field_string   = musicinfo_artist
sql_field_string   = musicinfo_publisher
sql_field_string   = musicinfo_tracks
sql_field_string   = musicinfo_asin
sql_field_string   = musicinfo_year

# movieinfo
sql_field_string   = movieinfo_title
sql_field_string   = movieinfo_tagline
sql_field_string   = movieinfo_plot
sql_field_string   = movieinfo_director
sql_field_string   = movieinfo_actors
sql_field_string   = movieinfo_genre
sql_field_string   = movieinfo_language

# predb
sql_field_string   = predb_dirname
sql_field_string   = predb_filename
sql_field_string   = predb_category
sql_field_string   = predb_nuketype
sql_field_string   = predb_nukereason

## Attributes - not searchable but can be used for filtering ##
# releases
sql_attr_bigint    = size
sql_attr_uint      = groupID
sql_attr_uint      = categoryID
sql_attr_uint      = totalpart
sql_attr_uint      = grabs
sql_attr_uint      = completion
sql_attr_uint      = regexID
sql_attr_uint      = rageID
sql_attr_uint      = tvdbID
sql_attr_uint      = imdbID
sql_attr_uint      = episodeinfoID
sql_attr_uint      = musicinfoID
sql_attr_uint      = consoleinfoID
sql_attr_uint      = bookinfoID
sql_attr_uint      = preID
sql_attr_uint      = anidbID
sql_attr_uint      = reqID
sql_attr_uint      = comments
sql_attr_uint      = passwordstatus
sql_attr_uint      = rarinnerfilecount
sql_attr_uint      = haspreview
sql_attr_timestamp = postdate
sql_attr_timestamp = adddate
sql_attr_timestamp = tvairdate

# bookinfo
sql_attr_uint      = bookinfo_genreID
sql_attr_uint      = bookinfo_pages
sql_attr_uint      = bookinfo_cover
sql_attr_timestamp = bookinfo_publishdate

# consoleinfo
sql_attr_uint      = consoleinfo_salesrank
sql_attr_uint      = consoleinfo_genreID
sql_attr_timestamp = consoleinfo_releasedate
sql_attr_uint      = consoleinfo_cover

# episodeinfo
sql_attr_uint      = episodeinfo_rageID
sql_attr_uint      = episodeinfo_tvdbID
sql_attr_uint      = episodeinfo_imdbID
sql_attr_uint      = episodeinfo_epabsolute
sql_attr_float     = episodeinfo_rating
sql_attr_timestamp = episodeinfo_airdate

# musicinfo
sql_attr_uint      = musicinfo_salesrank
sql_attr_uint      = musicinfo_genreID
sql_attr_uint      = musicinfo_cover
sql_attr_timestamp = musicinfo_releasedate

# movieinfo
sql_attr_uint      = movieinfo_imdbID
sql_attr_uint      = movieinfo_tmdbID
sql_attr_uint      = movieinfo_year
sql_attr_uint      = movieinfo_cover
sql_attr_uint      = movieinfo_backdrop
sql_attr_float     = movieinfo_rating

# predb
sql_attr_uint      = predb_ctime
sql_attr_uint      = predb_nuketime
sql_attr_float     = predb_filesize
sql_attr_uint      = predb_filecount
}
source releases : releases_base
{
sql_query_pre    = UPDATE sphinx \
SET maxID = (SELECT MAX(id) FROM releases), updatedate = NOW() \
WHERE ID = 1
sql_query       =   SELECT \
releases.ID AS ID, \
releases.name as name, \
releases.searchname as searchname, \
releases.fromname as fromname, \
releases.tvtitle as tvtitle, \
releases.season as season, \
releases.episode as episode, \
bookinfo.title as bookinfo_title, \
bookinfo.author as bookinfo_author, \
bookinfo.publisher as bookinfo_publisher, \
bookinfo.review as bookinfo_review, \
consoleinfo.title as consoleinfo_title, \
consoleinfo.publisher as consoleinfo_publisher, \
consoleinfo.review as consoleinfo_review, \
episodeinfo.showtitle as episodeinfo_showtitle, \
episodeinfo.eptitle as episodeinfo_eptitle, \
episodeinfo.director as episodeinfo_director, \
episodeinfo.writer as episodeinfo_writer, \
episodeinfo.gueststars as episodeinfo_gueststars, \
episodeinfo.overview as episodeinfo_overview, \
musicinfo.title as musicinfo_title, \
musicinfo.review as musicinfo_review, \
musicinfo.artist as musicinfo_artist, \
musicinfo.publisher as musicinfo_publisher, \
musicinfo.tracks as musicinfo_tracks, \
movieinfo.title as movieinfo_title, \
movieinfo.tagline as movieinfo_tagline, \
movieinfo.plot as movieinfo_plot, \
movieinfo.director as movieinfo_director, \
movieinfo.actors as movieinfo_actors, \
movieinfo.genre as movieinfo_genre, \
predb.dirname as predb_dirname, \
predb.filename as predb_filename, \
releases.size as size, \
releases.groupID as groupID, \
releases.categoryID as categoryID, \
releases.totalpart as totalpart, \
releases.grabs as grabs, \
releases.completion as completion, \
releases.regexID as regexID, \
releases.rageID as rageID, \
releases.tvdbID as tvdbID, \
releases.imdbID as imdbID, \
releases.episodeinfoID as episodeinfoID, \
releases.musicinfoID as musicinfoID, \
releases.consoleinfoID as consoleinfoID, \
releases.bookinfoID as bookinfoID, \
releases.preID as preID, \
releases.anidbID as anidbID, \
releases.reqID as reqID, \
releases.comments as comments, \
releases.passwordstatus as passwordstatus, \
releases.rarinnerfilecount as rarinnerfilecount, \
releases.haspreview as haspreview, \
releases.guid as guid, \
releases.seriesfull as seriesfull, \
UNIX_TIMESTAMP(releases.adddate) AS adddate, \
UNIX_TIMESTAMP(releases.postdate) AS postdate, \
UNIX_TIMESTAMP(releases.tvairdate) AS tvairdate, \
bookinfo.genreID as bookinfo_genreID, \
bookinfo.pages as bookinfo_pages, \
bookinfo.cover as bookinfo_cover, \
bookinfo.asin as bookinfo_asin, \
bookinfo.url as bookinfo_url, \
bookinfo.dewey as bookinfo_dewey, \
bookinfo.ean as bookinfo_ean, \
bookinfo.isbn as bookinfo_isbn, \
UNIX_TIMESTAMP(bookinfo.publishdate) as bookinfo_publishdate, \
consoleinfo.asin as consoleinfo_asin, \
consoleinfo.url as consoleinfo_url, \
consoleinfo.salesrank as consoleinfo_salesrank, \
consoleinfo.platform as consoleinfo_platform, \
consoleinfo.genreID as consoleinfo_genreID, \
consoleinfo.esrb as consoleinfo_esrb, \
UNIX_TIMESTAMP(consoleinfo.releasedate) as consoleinfo_releasedate, \
consoleinfo.cover as consoleinfo_cover, \
episodeinfo.rageID as episodeinfo_rageID, \
episodeinfo.tvdbID as episodeinfo_tvdbID, \
episodeinfo.imdbID as episodeinfo_imdbID, \
episodeinfo.epabsolute as episodeinfo_epabsolute, \
episodeinfo.rating as episodeinfo_rating, \
episodeinfo.fullep as episodeinfo_fullep, \
episodeinfo.link as episodeinfo_link, \
UNIX_TIMESTAMP(episodeinfo.airdate) as episodeinfo_airdate, \
musicinfo.salesrank as musicinfo_salesrank, \
musicinfo.genreID as musicinfo_genreID, \
musicinfo.cover as musicinfo_cover, \
musicinfo.asin as musicinfo_asin, \
musicinfo.year as musicinfo_year, \
UNIX_TIMESTAMP(musicinfo.releasedate) as musicinfo_releasedate, \
movieinfo.imdbID as movieinfo_imdbID, \
movieinfo.tmdbID as movieinfo_tmdbID, \
movieinfo.year as movieinfo_year, \
movieinfo.cover as movieinfo_cover, \
movieinfo.backdrop as movieinfo_backdrop, \
movieinfo.rating as movieinfo_rating, \
movieinfo.language as movieinfo_language, \
predb.ctime as predb_ctime, \
predb.nuketime as predb_nuketime, \
predb.filesize as predb_filesize, \
predb.filecount as predb_filecount, \
predb.category as predb_category, \
predb.nuketype as predb_nuketype, \
predb.nukereason as predb_nukereason \
FROM releases \
LEFT JOIN bookinfo \
ON releases.bookinfoID = bookinfo.ID \
LEFT JOIN consoleinfo \
ON releases.consoleinfoID = consoleinfo.ID \
LEFT JOIN episodeinfo \
ON releases.episodeinfoID = episodeinfo.ID \
LEFT JOIN musicinfo \
ON releases.musicinfoID = musicinfo.ID \
LEFT JOIN movieinfo \
ON releases.imdbID = movieinfo.imdbID \
LEFT JOIN predb \
ON releases.preID = predb.ID \
WHERE releases.ID >= $start AND releases.ID <= $end \
GROUP BY releases.ID
sql_range_step   = 10000
sql_query_range  = SELECT MIN(id), MAX(id) FROM releases
# Uncomment for debugging with "search" utility
#sql_query_info  = SELECT * FROM releases WHERE id=$id
}
source releases_delta : releases_base
{
sql_query       =   SELECT \
releases.ID AS ID, \
releases.name as name, \
releases.searchname as searchname, \
releases.fromname as fromname, \
releases.tvtitle as tvtitle, \
releases.season as season, \
releases.episode as episode, \
bookinfo.title as bookinfo_title, \
bookinfo.author as bookinfo_author, \
bookinfo.publisher as bookinfo_publisher, \
bookinfo.review as bookinfo_review, \
consoleinfo.title as consoleinfo_title, \
consoleinfo.publisher as consoleinfo_publisher, \
consoleinfo.review as consoleinfo_review, \
episodeinfo.showtitle as episodeinfo_showtitle, \
episodeinfo.eptitle as episodeinfo_eptitle, \
episodeinfo.director as episodeinfo_director, \
episodeinfo.writer as episodeinfo_writer, \
episodeinfo.gueststars as episodeinfo_gueststars, \
episodeinfo.overview as episodeinfo_overview, \
musicinfo.title as musicinfo_title, \
musicinfo.review as musicinfo_review, \
musicinfo.artist as musicinfo_artist, \
musicinfo.publisher as musicinfo_publisher, \
musicinfo.tracks as musicinfo_tracks, \
movieinfo.title as movieinfo_title, \
movieinfo.tagline as movieinfo_tagline, \
movieinfo.plot as movieinfo_plot, \
movieinfo.director as movieinfo_director, \
movieinfo.actors as movieinfo_actors, \
movieinfo.genre as movieinfo_genre, \
predb.dirname as predb_dirname, \
predb.filename as predb_filename, \
releases.size as size, \
releases.groupID as groupID, \
releases.categoryID as categoryID, \
releases.totalpart as totalpart, \
releases.grabs as grabs, \
releases.completion as completion, \
releases.regexID as regexID, \
releases.rageID as rageID, \
releases.tvdbID as tvdbID, \
releases.imdbID as imdbID, \
releases.episodeinfoID as episodeinfoID, \
releases.musicinfoID as musicinfoID, \
releases.consoleinfoID as consoleinfoID, \
releases.bookinfoID as bookinfoID, \
releases.preID as preID, \
releases.anidbID as anidbID, \
releases.reqID as reqID, \
releases.comments as comments, \
releases.passwordstatus as passwordstatus, \
releases.rarinnerfilecount as rarinnerfilecount, \
releases.haspreview as haspreview, \
releases.guid as guid, \
releases.seriesfull as seriesfull, \
UNIX_TIMESTAMP(releases.adddate) AS adddate, \
UNIX_TIMESTAMP(releases.postdate) AS postdate, \
UNIX_TIMESTAMP(releases.tvairdate) AS tvairdate, \
bookinfo.genreID as bookinfo_genreID, \
bookinfo.pages as bookinfo_pages, \
bookinfo.cover as bookinfo_cover, \
bookinfo.asin as bookinfo_asin, \
bookinfo.url as bookinfo_url, \
bookinfo.dewey as bookinfo_dewey, \
bookinfo.ean as bookinfo_ean, \
bookinfo.isbn as bookinfo_isbn, \
UNIX_TIMESTAMP(bookinfo.publishdate) as bookinfo_publishdate, \
consoleinfo.asin as consoleinfo_asin, \
consoleinfo.url as consoleinfo_url, \
consoleinfo.salesrank as consoleinfo_salesrank, \
consoleinfo.platform as consoleinfo_platform, \
consoleinfo.genreID as consoleinfo_genreID, \
consoleinfo.esrb as consoleinfo_esrb, \
UNIX_TIMESTAMP(consoleinfo.releasedate) as consoleinfo_releasedate, \
consoleinfo.cover as consoleinfo_cover, \
episodeinfo.rageID as episodeinfo_rageID, \
episodeinfo.tvdbID as episodeinfo_tvdbID, \
episodeinfo.imdbID as episodeinfo_imdbID, \
episodeinfo.epabsolute as episodeinfo_epabsolute, \
episodeinfo.rating as episodeinfo_rating, \
episodeinfo.fullep as episodeinfo_fullep, \
episodeinfo.link as episodeinfo_link, \
UNIX_TIMESTAMP(episodeinfo.airdate) as episodeinfo_airdate, \
musicinfo.salesrank as musicinfo_salesrank, \
musicinfo.genreID as musicinfo_genreID, \
musicinfo.cover as musicinfo_cover, \
musicinfo.asin as musicinfo_asin, \
musicinfo.year as musicinfo_year, \
UNIX_TIMESTAMP(musicinfo.releasedate) as musicinfo_releasedate, \
movieinfo.imdbID as movieinfo_imdbID, \
movieinfo.tmdbID as movieinfo_tmdbID, \
movieinfo.year as movieinfo_year, \
movieinfo.cover as movieinfo_cover, \
movieinfo.backdrop as movieinfo_backdrop, \
movieinfo.rating as movieinfo_rating, \
movieinfo.language as movieinfo_language, \
predb.ctime as predb_ctime, \
predb.nuketime as predb_nuketime, \
predb.filesize as predb_filesize, \
predb.filecount as predb_filecount, \
predb.category as predb_category, \
predb.nuketype as predb_nuketype, \
predb.nukereason as predb_nukereason \
FROM releases \
LEFT JOIN bookinfo \
ON releases.bookinfoID = bookinfo.ID \
LEFT JOIN consoleinfo \
ON releases.consoleinfoID = consoleinfo.ID \
LEFT JOIN episodeinfo \
ON releases.episodeinfoID = episodeinfo.ID \
LEFT JOIN musicinfo \
ON releases.musicinfoID = musicinfo.ID \
LEFT JOIN movieinfo \
ON releases.imdbID = movieinfo.imdbID \
LEFT JOIN predb \
ON releases.preID = predb.ID \
WHERE releases.updatedate >= (SELECT lastmergedate \
FROM sphinx WHERE ID = 1) \
GROUP BY releases.ID
}
source releasefiles : base
{
Can't do this index using a "range" because it appears to cause duplicate
# documents to be indexed.
sql_query_pre    = UPDATE sphinx \
                   SET maxID = (SELECT MAX(id) FROM releasefiles), updatedate = NOW() \
                   WHERE ID = 2
sql_query        = SELECT \
                    releasefiles.releaseID AS ID, \
                    GROUP_CONCAT(releasefiles.name) as name, \
                    UNIX_TIMESTAMP(releases.postdate) AS postdate, \
                    releases.categoryID, \
                    releases.size, \
                    releases.grabs, \
                    releases.totalpart \
                   FROM releasefiles \
                   INNER JOIN releases ON releases.ID = releasefiles.releaseID \
                   GROUP BY releasefiles.releaseID \
                   ORDER BY releasefiles.releaseID ASC
sql_field_string   = name

sql_attr_bigint    = size
sql_attr_uint      = grabs
sql_attr_timestamp = postdate
sql_attr_uint      = totalpart
sql_attr_uint      = categoryID
}
source releasefiles_delta : base
{
sql_query        = SELECT \
releasefiles.releaseID as ID, \
GROUP_CONCAT(releasefiles.name) as name, \
UNIX_TIMESTAMP(releases.postdate) AS postdate, \
releases.categoryID, \
releases.size, \
releases.grabs, \
releases.totalpart \
FROM releasefiles \
INNER JOIN releases ON releases.ID = releasefiles.releaseID \
WHERE releasefiles.ID > (SELECT maxID \
FROM sphinx WHERE ID = 2) \
GROUP BY releasefiles.releaseID \
ORDER BY releasefiles.releaseID ASC
sql_field_string   = name

sql_attr_bigint    = size
sql_attr_uint      = grabs
sql_attr_timestamp = postdate
sql_attr_uint      = totalpart
sql_attr_uint      = categoryID
}
source releasenfo : base
{
sql_query_pre    = UPDATE sphinx \
SET maxID = (SELECT MAX(id) FROM releasenfo), updatedate = NOW() \
WHERE id = 3
sql_query        = SELECT \
releaseID as ID, \
UNCOMPRESS(releasenfo.nfo) as nfo, \
releases.name, \
UNIX_TIMESTAMP(releases.postdate) AS postdate, \
releases.categoryID, \
releases.size, \
releases.grabs, \
releases.totalpart \
FROM releasenfo \
INNER JOIN releases ON releases.ID = releasenfo.releaseID \
WHERE releasenfo.id >= $start \
AND releasenfo.id <= $end \
GROUP BY releaseID \
ORDER BY releaseID ASC
sql_range_step   = 10000
sql_query_range  = SELECT MIN(releaseID), MAX(releaseID) FROM releasenfo
sql_field_string   = nfo
sql_field_string   = name

sql_attr_bigint    = size
sql_attr_uint      = grabs
sql_attr_timestamp = postdate
sql_attr_uint      = totalpart
sql_attr_uint      = categoryID
}
source releasenfo_delta : base
{
sql_query        = SELECT \
releaseID as ID, \
UNCOMPRESS(releasenfo.nfo) as nfo, \
releases.name, \
UNIX_TIMESTAMP(releases.postdate) AS postdate, \
releases.categoryID, \
releases.size, \
releases.grabs, \
releases.totalpart \
FROM releasenfo \
INNER JOIN releases ON releases.ID = releasenfo.releaseID \
WHERE releasenfo.id > (SELECT maxID \
FROM sphinx WHERE ID = 3) \
GROUP BY releaseID \
ORDER BY releaseID ASC
sql_field_string   = nfo
sql_field_string   = name

sql_attr_bigint    = size
sql_attr_uint      = grabs
sql_attr_timestamp = postdate
sql_attr_uint      = totalpart
sql_attr_uint      = categoryID
}
source predb : base
{
sql_query_pre    = UPDATE sphinx \
SET maxID = (SELECT MAX(id) FROM predb), updatedate = NOW() \
WHERE id = 4
sql_query       =   SELECT \
predb.ID AS ID, \
predb.ctime as ctime, \
predb.dirname as dirname, \
predb.category as category, \
predb.nuketype as nuketype, \
predb.nukereason as nukereason, \
predb.filesize as filesize, \
predb.filecount as filecount, \
releases.guid as guid, \
releasenfo.ID as nfoID \
FROM predb \
LEFT JOIN releases \
ON releases.preID = predb.ID \
LEFT JOIN releasenfo \
ON releasenfo.releaseID = releases.ID \
WHERE predb.ID >= $start AND predb.ID <= $end \
GROUP BY predb.ID
sql_range_step   = 10000
sql_query_range  = SELECT MIN(id), MAX(id) FROM predb
sql_field_string = dirname
sql_field_string = category
sql_field_string = nuketype
sql_field_string = guid

sql_attr_uint    = ctime
sql_attr_uint    = nfoID
}
source predb_delta : base
{

sql_query       = SELECT \
predb.ID AS ID, \
predb.ctime as ctime, \
predb.dirname as dirname, \
predb.category as category, \
predb.nuketype as nuketype, \
predb.nukereason as nukereason, \
predb.filesize as filesize, \
predb.filecount as filecount, \
releases.guid as guid, \
releasenfo.ID as nfoID \
FROM predb \
LEFT JOIN releases \
ON releases.preID = predb.ID \
LEFT JOIN releasenfo \
ON releasenfo.releaseID = releases.ID \
WHERE predb.updatedate >= (SELECT lastmergedate \
FROM sphinx WHERE ID = 4) \
GROUP BY predb.ID
sql_field_string = dirname
sql_field_string = category
sql_field_string = nuketype
sql_field_string = guid

sql_attr_uint    = ctime
sql_attr_uint    = nfoID
}
###############################################################################
index definition
###############################################################################
index releases
{
type              = plain
path              = C:\xampp\htdocs/db/sphinxdata/releases
source            = releases
morphology        = stem_en
blend_chars       = -, (, ), [, ], {, }, +, &, U+23, /, \, <, >
charset_type      = utf-8
enable_star       = 1
min_infix_len     = 3
infix_fields      = name, searchname
index_exact_words = 1
}
index releases_delta
{
type              = plain
path              = C:\xampp\htdocs/db/sphinxdata/releases_delta
source            = releases_delta
morphology        = stem_en
blend_chars       = -, (, ), [, ], {, }, +, &, U+23, /, \, <, >
charset_type      = utf-8
enable_star       = 1
min_infix_len     = 3
infix_fields      = name, searchname
index_exact_words = 1
}
index releasefiles
{
type              = plain
path              = C:\xampp\htdocs/db/sphinxdata/releasefiles
source            = releasefiles
morphology        = stem_en
blend_chars       = -, (, ), [, ], {, }, +, &, U+23, /, \, <, >
charset_type      = utf-8
index_exact_words = 1
enable_star       = 1
min_infix_len     = 3
}
index releasefiles_delta
{
type              = plain
path              = C:\xampp\htdocs/db/sphinxdata/releasefiles_delta
source            = releasefiles_delta
morphology        = stem_en
blend_chars       = -, (, ), [, ], {, }, +, &, U+23, /, \, <, >
charset_type      = utf-8
index_exact_words = 1
enable_star     = 1
min_infix_len   = 3
}
index releasenfo
{
type              = plain
path              = C:\xampp\htdocs/db/sphinxdata/releasenfo
source            = releasenfo
morphology        = stem_en
blend_chars       = -, (, ), [, ], {, }, +, &, U+23, /, \, <, >
charset_type      = utf-8
index_exact_words = 1
enable_star     = 1
#min_infix_len   = 3
}
index releasenfo_delta
{
type              = plain
path              = C:\xampp\htdocs/db/sphinxdata/releasenfo_delta
source            = releasenfo_delta
morphology        = stem_en
blend_chars       = -, (, ), [, ], {, }, +, &, U+23, /, \, <, >
charset_type      = utf-8
index_exact_words = 1
enable_star     = 1
#min_infix_len   = 3
}
index nzbs
{
type            = rt
path            = C:\xampp\htdocs/db/sphinxdata/nzbs
rt_field        = file_names
rt_attr_uint    = file_count
rt_mem_limit    = 256M
# If you really have a death wish:
#min_infix_len   = 3
}
index predb
{
type              = plain
path              = C:\xampp\htdocs/db/sphinxdata/predb
source            = predb
morphology        = stem_en
blend_chars       = -, (, ), [, ], {, }, +, &, U+23, /, \, <, >
charset_type      = utf-8
index_exact_words = 1
enable_star       = 1
min_infix_len     = 3
infix_fields      = dirname
}

index predb_delta { type = plain path = C:\xampp\htdocs/db/sphinxdata/predb_delta source = predb_delta morphology = stem_en blend_chars = -, (, ), [, ], {, }, +, &, U+23, /, \, <, > charset_type = utf-8 index_exact_words = 1 enable_star = 1 min_infix_len = 3 infix_fields = dirname }

llpowwowll commented 11 years ago

Holy shit you guys weren't kidding...sphinx helps. All I can say is wow and thanks for your help guys. I think its working properly.

evilhero / mylar

Newznab support? #98

indexer settings

You can tweak these settings as you like and/or add your own

searchd settings

source definitions

[---------------------------------------------------------------------------]

| Don't modify below here unless you know what you are doing |

[---------------------------------------------------------------------------]

Full-text fields - these are searchable

Can't do this index using a "range" because it appears to cause duplicate

index definition

enable_star = 1

enable_star = 1