Always hangs up in different sections without any error - Githubissues

kiliman / tailwindui-crawler

tailwindui-crawler downloads the component HTML files locally

MIT License

756 stars 94 forks source link

Always hangs up in different sections without any error #40

Closed tobkle closed 3 years ago

tobkle commented 3 years ago

Is there a trick to run it stable? Or is the crawler outdated meanwhile? Has anyone tried with current version of tailwind ui site?

Running with

EMAIL=yyy
PASSWORD=xxxx
OUTPUT=./output
HTMLMODE=comments 
TRANSFORMERS=prefixSrc

Process hangs up always in one of the first 4 marketing sections.

tobkle commented 3 years ago

Also in the debugger I cannot find the problem. It sometimes hangs during const html = response.text() without raising an error.

Then I decided to change to puppeteer instead. With that changed version it worked for me: https://github.com/tobkle/tailwindui-crawler.git