guanquann / Stocksera

Finance application that provides more than 60 different alternative data to retail investors
MIT License
659 stars 107 forks source link

IndexError with reddit crypto scrape #13

Closed ElegantEgotist closed 3 years ago

ElegantEgotist commented 3 years ago

When trying to scrape crypto:

Traceback (most recent call last): File "tasks_to_run.py", line 83, in scrape_reddit_crypto.main() File "/home/testadmin/Stocksera/scheduled_tasks/reddit/get_reddit_trending_crypto.py", line 317, in main

This line from get_reddit_trending_crypto.py gives an index error:

db.execute("SELECT * FROM cryptocurrency WHERE date_updated LIKE '%{}%'".format(dates[-1][0].split()[0], )) IndexError: list index out of range

CodeInFilth commented 3 years ago

When trying to scrape crypto:

Traceback (most recent call last): File "tasks_to_run.py", line 83, in scrape_reddit_crypto.main() File "/home/testadmin/Stocksera/scheduled_tasks/reddit/get_reddit_trending_crypto.py", line 317, in main

This line from get_reddit_trending_crypto.py gives an index error:

db.execute("SELECT * FROM cryptocurrency WHERE date_updated LIKE '%{}%'".format(dates[-1][0].split()[0], )) IndexError: list index out of range

Check you database files, index out of range means the varible its trying to grab is non existant. Use the google drive database he supplied in Scheduled_Tasks and then run the procceses. Make sure you update all the settings needed and run collectscript

guanquann commented 3 years ago

I have fixed the bug. Replace it with:

if dates:
    last_date = dates[-1][0].split()[0]
else:
    last_date = ""
db.execute("SELECT * FROM cryptocurrency WHERE date_updated LIKE '%{}%'".format(last_date, ))
ElegantEgotist commented 3 years ago

The fix worked. Also I didn't notice @spartan737 had updated the sample drive database since I hadn't checked it in a while. I tried with those as well and it works great. Thank you both, closing this now.