The crawl.php code as it is, at line 160 (line 153 for crawl-manual) could use these bits:
$spacer_size = 8; // increment me until it works echo str_pad('', (1024 * $spacer_size), "\n"); if(ob_get_level()) ob_end_clean();
It will prevent a inevitable 524 Timeout if you are using Cloudflare services (ran into this issue myself)
The crawl.php code as it is, at line 160 (line 153 for crawl-manual) could use these bits:
$spacer_size = 8; // increment me until it works echo str_pad('', (1024 * $spacer_size), "\n"); if(ob_get_level()) ob_end_clean();It will prevent a inevitable 524 Timeout if you are using Cloudflare services (ran into this issue myself)
Source of fix: https://github.com/marcialpaulg/Fixing-Cloudflare-Error-524
Example in action: https://github.com/safesploit/doogle/commit/dd72700c3919c6d3b1150f06ef726ff081fe479c