kiliman / tailwindui-crawler

tailwindui-crawler downloads the component HTML files locally
MIT License
756 stars 94 forks source link

Is this broken now? #83

Open sujeetpillai opened 6 months ago

sujeetpillai commented 6 months ago

I'm unable to use it to download.

First I got the "Not logged in error" when downloading a section. Since it was looking for my email address in the HTML, the email address is not present as is on that page html anymore. It's in a modal dialogue.

When I bypassed that, I got an error where "component.snippets" was null. The data-page attribute has a javascript object there doesn't have those fields anymore. In fact the iframes for each component now has srcdoc.

tecio-first commented 6 months ago

Interesting what exactly is not working ? I used this repo for the first time yesterday and it works locally and as a download action.

muesli-dev commented 4 months ago

ref: https://github.com/kiliman/tailwindui-crawler/issues/84 @sujeetpillai

toshsan commented 2 months ago

Not logged in error after logging in.

tailwindui-crawler@4.1.0 start node index.mjs

🔐 Logging into tailwindui.com... 🔍 Fetching https://tailwindui.com/login ⏱ 235ms (200) ✅ Success!

🗂 Output is ./output 🔍 Fetching https://tailwindui.com/components ⏱ 182ms (200) ⏳ Processing /components/marketing/sections/heroes... 🔍 Fetching https://tailwindui.com/components/marketing/sections/heroes ⏱ 178ms (200) 🚫 Not logged in

kiliman commented 2 months ago

I haven't checked recently. Let me see.

kiliman commented 2 months ago

Ok, I tried it locally

image

And here's my GitHub Action which runs daily

image
toshsan commented 2 months ago

Hmm. I have tried again on my repo. Same error as i reported above. Could you please share your commit hash to make sure we are on the same codebase?

kiliman commented 2 months ago

It's the latest master d3cc6ee9cf2c87145aa2943ff9c7806e4b80855c

Do you have the all-access subscription? That's what I have. I wonder if that's the difference.

muesli-dev commented 2 months ago

Hmm. I have tried again on my repo. Same error as i reported above. Could you please share your commit hash to make sure we are on the same codebase?

Maybe this works for you also https://github.com/kiliman/tailwindui-crawler/issues/84

abdussamadbello commented 1 month ago

@kiliman The issue I noticed are that tailwind is now blocking non-origin access from its CDN (https://tailwindui.com/build/assets/). So we need to download the sources from the headers and iframe and serve them locally.

kiliman commented 1 month ago

Yes, I noticed that too. I have a version running locally that I'm still testing. I'll try to get a preview version that you can try out by tomorrow.

abdussamadbello commented 1 month ago

Thanks @kiliman ... Youre awesome!

abdussamadbello commented 1 month ago

Yes, I noticed that too. I have a version running locally that I'm still testing. I'll try to get a preview version that you can try out by tomorrow.

Eagerly waiting for updates

VIEWVIEWVIEW commented 1 month ago

Yes, I noticed that too. I have a version running locally that I'm still testing. I'll try to get a preview version that you can try out by tomorrow.

Would be great if you could release the preview version. I copied almost all components by hand in the meantime to have them available when I'm on a plane. However, they updated many, many components to headlessui 2.1, and I definitely don't want to copy all of them by hand again xD