niqdev / packtpub-crawler

Download your daily free Packt Publishing eBook https://www.packtpub.com/packt/offers/free-learning
MIT License
755 stars 178 forks source link

Error when getting config file #49

Closed trancen closed 7 years ago

trancen commented 7 years ago

Updated to the latest version.

I have the following in a crontab, been using this for a few weeks working correctly. I noticed that I wasn't getting any new books so I ran the commend manually :

python /home/david/packtpub-crawler/script/spider.py --config /home/david/packtpub-crawler/config/prod.cfg -t pdf --extras

But I'm getting the following error.

[*] 2017-01-31 08:34 - fetching today's eBooks
[-] <type 'exceptions.IOError'> file not found! | spider.py@89
Traceback (most recent call last):
  File "/home/david/packtpub-crawler/script/spider.py", line 89, in main
    config = config_file(dir_path + args.config)
  File "/home/david/packtpub-crawler/script/utils.py", line 24, in config_file
    raise IOError('file not found!')
IOError: file not found!
[*] done

But if I move to the packtpub-crawler directory it works:

david@server:~/packtpub-crawler$ python script/spider.py --config config/prod.cfg -t pdf --extras


[*] 2017-01-31 08:39 - fetching today's eBooks
[*] configuration file: /home/david/packtpub-crawler/config/prod.cfg
[*] getting daily free eBook
[*] fetching url... 200 | https://www.packtpub.com/packt/offers/free-learning
[*] fetching url... 200 | https://www.packtpub.com/packt/offers/free-learning
[*] fetching url... 200 | https://www.packtpub.com/account/my-ebooks
[+] book successfully claimed
juzim commented 7 years ago

There were some issues with relative paths in cron: https://github.com/niqdev/packtpub-crawler/pull/36/commits/ce03d4fc5cb14bd8f36671db81d6d443c2fc6324

Please try using relative paths (--config config/prod.cfg). You can also look at the readme file for working examples.

niqdev commented 7 years ago

@trancen using relatives path solves your problem? Thanks

trancen commented 7 years ago

Ya it did.. I had to use the following on my crontab

cd /home/david/packtpub-crawler/ && python script/spider.py --config config/prod.cfg -t pdf --extras