add a crawling defaults on the Org to allow setting certain crawl workflow fields as defaults:

webrecorder / browsertrix

Browsertrix is the hosted, high-fidelity, browser-based crawling service from Webrecorder designed to make web archiving easier and more accessible for all!

https://webrecorder.net/browsertrix

GNU Affero General Public License v3.0

201 stars 35 forks source link

add a crawling defaults on the Org to allow setting certain crawl workflow fields as defaults: #2031

Closed ikreymer closed 3 months ago

ikreymer commented 3 months ago

add POST /orgs//defaults/crawling API to update all defaults (defaults unset are cleared)
defaults returned as 'crawlingDefaults' object on Org, if set
fixes #2016

SuaYoo commented 3 months ago

It looks like we're missing maxCrawlSizeGB and crawlTimeout.

ikreymer commented 3 months ago

It looks like we're missing maxCrawlSizeGB and crawlTimeout.

Added, replacing limit (page limit). Just maxCrawlSize to be consistent with how its stored in the workflow, but could do just GB if that's easier.