issues
search
Alhajras
/
webscraper
Configurable search engine written in Python and Angular. It supports indexing as well.
1
stars
0
forks
source link
Milestone #1
#11
Open
Alhajras
opened
1 year ago
Alhajras
commented
1 year ago
[x] Indexing:
[x] Use an Inverted Index.
[x] Crawler:
[x] Basic UI that allows users to crawl one website.
[x] Multi-threaded (Crawling happens in patches).
[x] Not distributed (Only running on one machine).
[x] UI:
[x] Dashboard: Basic configurations that allows simple crawling.
[x] Dashboard: Basic configurations that allows simple indexing.
[x] Dashboard: Basic monitoring progress bar (Shows the current url, how many urls are done).
[x] Add actions: start, rerun, pause and continue crawling.
[x] Search page that contain a searching bar, and a result list based on the indexed data.