search

isohuntto / openbay

Other
3.52k stars 902 forks source link

Working Kickass api data dump to Openbay DB converter and DB updater #174

Open nicoboss opened 9 years ago

nicoboss commented 9 years ago

I made a very nice Kickass api data dumps (Full data dump and Latest 24 hours data dump) to Openbay DB converter and updater written in shell and MySQL. Now it's possible to have an own Torrent site with nearly the same DB like the kickass site. And the best thing is that you get every hour an update. I wrote two working shell scripts that will do all work for you.

How it works:

  1. Installing mysql on your Linux server (sudo apt-get install mysql-server)
  2. Download the two shell scripts that you find at the bottom of this text
  3. Edit the shell script and fill in your MySQL password in the second line
  4. Open the KickassFullDump.sh once to get the full Kickass DB and open KickassUpdater.sh to update your DB with root permissions (or change cd /root path in the script on line 5). For automatically updates you can use something like crontab.

Downloadlinks: https://raw.githubusercontent.com/nicoboss/KickassCopy/master/Kickass%20api%20to%20Openbay/KickassFullDump.sh https://raw.githubusercontent.com/nicoboss/KickassCopy/master/Kickass%20api%20to%20Openbay/KickassUpdater.sh

If you have any questions don't be afraid to ask.

TPBT-OFFICIAL commented 9 years ago

ERROR 1148 (42000) at line 32: The used command is not allowed with this MySQL version

nicoboss commented 9 years ago

Yes it's because LOAD DATA LOCAL is disabled in some MySQL versions by default due to security. Reed more under http://dev.mysql.com/doc/refman/5.0/en/load-data-local.html or Google under "Enabling LOAD DATA LOCAL INFILE mysql" and "ERROR 1148 (42000)". You will find al lot about this in the Internet.

The latest version of my script that I've uploaded some minutes ago tries to enabling LOAD DATA LOCAL INFILE by his own. But on some server this value must be changed manually in the /etc/my.cnf file by setting:

[mysqld] local-infile = 1

[mysql] local-infile = 1

milezzz commented 9 years ago

Thanks for doing this =)

How long does it take to complete the full data dump?

nicoboss commented 9 years ago

A full dump takes between 10 and 50 minutes depending on your server speed. On extremely slow Server I can also take over one hour.

nicoboss commented 9 years ago

As you may know from https://github.com/isohuntto/openbay/issues/175 kickass offer their API only for registered users after sending a request (https://github.com/isohuntto/openbay-db-dump/issues/7 and https://github.com/isohuntto/openbay/issues/141 still works without registration). So I changed my script for supporting this closed API. First register and sending a well-founded request and wait some minutes until they approved it. Next you have to copy your userhash into the 3rd line of both scripts you can see if you copy the hourly dump download link. The Link will look like https://kat.cr/api/get_dump/hourly/?userhash=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx your hash is he thing after userhash= and contains 32 characters. After this my script should work like before. The Downloadlinks are the same as for the first version because I'm using GitHub links.