search

N0taN3rd / Squidwarc

Squidwarc is a high fidelity, user scriptable, archival crawler that uses Chrome or Chromium with or without a head
https://n0tan3rd.github.io/Squidwarc/
Apache License 2.0
168 stars 26 forks source link

Make scalable #17

Open N0taN3rd opened 6 years ago

N0taN3rd commented 6 years ago

Are you submitting a bug report or a feature request?

Feature Request

What is the current behavior?

Does not scale

What is the expected behavior?

To scale

What's your environment?

Other information

machawk1 commented 6 years ago

@N0taN3rd Did you mean scalable?

N0taN3rd commented 5 years ago

ref https://github.com/peterk/warcworker/issues/5