Add an endpoint for consolidated spider updates

TorSpider / TorSpider-Backend

The database backend with which the spiders share their discoveries.

BSD 3-Clause "New" or "Revised" License

1 stars 0 forks source link

Add an endpoint for consolidated spider updates #39

Open artagel opened 6 years ago

artagel commented 6 years ago

Instead of the spider posting a million times to update each link/page/spider. We could simply post a json blob with all the found data on a scrape and let the backend handle it.

haxys commented 6 years ago

Offload_to_backend should be just about done. We just need to test and merge.