search

chihacknight / chihacknight-yt-interface

Chi Hack Night Youtube captions scraper/search interface
MIT License
4 stars 4 forks source link

Keyword search/Database Structure #3

Open easherma opened 7 years ago

easherma commented 7 years ago

Need a way to search keywords, KISS, TBD

This goes along with determining a database structure

easherma commented 7 years ago

For now we're using Jupyter/iPython Notebooks and reading the flat files we wrote to disk.

The slowest part is having Beautiful Soup parse the xml, but loading transcripts into Pandas and then searching them seems pretty snappy. We're not dealing with a ton of data here and the transcripts won't really change, so I'm contemplating going with a NoSQL/document based solution over something more heavyweight.