search

laughingclouds / Scrapia-World

A web scraper for scraping wuxiaworld. Written in python, using selenium and python cmd for an interactive shell experience with a command line utility to work with text along with a database to store information.
MIT License
2 stars 1 forks source link

can we switch to sqlite? #7

Closed laughingclouds closed 2 years ago

laughingclouds commented 2 years ago

What the title says. I never knew we could use sqlite (I don't think I even knew about it much) when I started with this project. I'm aiming for working with as less dependencies as I can.

My rationale: It improves portability (in a weird, probably-unnecessary way).

tl:dr; Switch from mysql to sqlite

laughingclouds commented 2 years ago

Okay, I made changes so that we use sqlite instead of mysql. And frankly, this is a lot less painful than mysql. At least that's what I feel right now.

laughingclouds commented 2 years ago

Won't close this for now, I need to test whether it's working...

laughingclouds commented 2 years ago

I believe this is where this issue resolves bdb5dba9efd61d341515f87bfeda9742a1c041d3 And this 4c06df835ca5e424ae1bbcc2ab67e80f6a22c778...makes this issue unneeded.

Basically, our objective can be solved with json instead. Which is definitely a little wasteful (repeatedly creating a file object), but it "works". Maybe later we can better use the database, in conjunction with the json.

But yes, this is issue is resolved.