Closed JohnDavid07 closed 4 years ago
``exclusions = ['__MACOSX/']
destination = "/content/drive/My Drive/" download_tasks = [ { 'folder': 'gdppci', 'url': 'https://........................workers.dev/0:/..................................' (something private url) }, ]
print('##################################') print('# Crawling all downloadable urls #') print('##################################', end='\n\n') tasks = [] for task in download_tasks: tasks += crawler_v2(task['url'], [], os.path.join(destination, task['folder']), 0, exclusions, verbose=False)
total_size = get_filesize(sum([int(task['size']) for task in tasks]))
print(json.dumps(tasks, indent=2)) print('\nTotal Task:', len(tasks)) print('Total size: %.3fGB' % total_size, end='\n\n')``
Can you please give/post a guide on how to use.
@atlonxp can you please look into this? i can't find the problem with "tasks"
@JohnDavid07 @NullBruce could you provide me the goindex link I will try when I have time
@atlonxp literally any link.
Crawling all downloadable urls # ##################################
https://tutnetflix.mlwdl.workers.dev/FrontEndMasters%20-%20Complete%20Intro%20to%20Containers/ retry #2 https://tutnetflix.mlwdl.workers.dev/FrontEndMasters%20-%20Complete%20Intro%20to%20Containers/ retry #3 https://tutnetflix.mlwdl.workers.dev/FrontEndMasters%20-%20Complete%20Intro%20to%20Containers/ retry #4 https://tutnetflix.mlwdl.workers.dev/FrontEndMasters%20-%20Complete%20Intro%20to%20Containers/ retry #5 https://tutnetflix.mlwdl.workers.dev/FrontEndMasters%20-%20Complete%20Intro%20to%20Containers/
use terminal CURL - Nah, something went wrong!
JSONDecodeError Traceback (most recent call last)
Huh! You don't seem to aware that tutflix (aka tutnetflix) has been banned from Cloudflare. The links you provided were not available long ago.
Easy way to check if the link working is to visit the GoIndex website.
@atlonxp here's a link that doesn't work, also i tried with multiple ones that are up.
#################################
##################################
https://manga.td-index.workers.dev/0:/ retry #2 https://manga.td-index.workers.dev/0:/ retry #3 https://manga.td-index.workers.dev/0:/ retry #4 https://manga.td-index.workers.dev/0:/ retry #5 https://manga.td-index.workers.dev/0:/
use terminal CURL - Nah, something went wrong!
JSONDecodeError Traceback (most recent call last)
@NullBruce Acrous index is not yet supported. See #7 I tried to poke around a bit but nothing seems to work :(
@Rudo2204 @NullBruce i need to have a look around how Acrous working. I think it is just a theme but there might as well be some script for a dynamic content generation (which is causing the problem)