search

trokam / step2

Trokam Step2 -- objective: fast online search over 100 millon web pages.
GNU General Public License v3.0
1 stars 2 forks source link

Mechanism to control the crunchers. #3

Open nicolas-slusarenko opened 6 years ago

nicolas-slusarenko commented 6 years ago

The objective is have a mechanism to pause all the crunchers to allow some operations in the database and then activate them all to continue crunching.

nicolas-slusarenko commented 6 years ago

The crunchers polls the new database, control, to pause the crunching or exit. The option to crunch a specific amount of pages is still valid.

nicolas-slusarenko commented 6 years ago

Te database control has only the table command. I created inside the database pagesInfo the same table, to create one database for both functions, text search and control the crunchers. Also, I updated the URLs seeds to include more areas of software development.