puppeteer / examples

Use case-driven examples for using Puppeteer and headless chrome
https://developers.google.com/web/tools/puppeteer/
Apache License 2.0
2.37k stars 302 forks source link

Added sitemap.xml gen #20

Open forrest321 opened 6 years ago

forrest321 commented 6 years ago

Added sitemap.xml gen

googlebot commented 6 years ago

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

:memo: Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here (e.g. I signed it!) and we'll verify it.


What to do if you already signed the CLA

Individual signers
Corporate signers
forrest321 commented 6 years ago

I signed it!

forrest321 commented 6 years ago

I signed it!

googlebot commented 6 years ago

CLAs look good, thanks!

abdonrd commented 5 years ago

Interested in this!

forrest321 commented 5 years ago

Requested changed are done.

forrest321 commented 5 years ago

This one fell off my radar, thought I'd get it done. Hope that helps.

Kiina commented 4 years ago

Just a note, the sitemap generator might need a filter for anchor same page links (aka example.com/#home). I'm kinda sure they shouldn't be included in a sitemap in basically any case but they get included in the current implementation.

My quick and dirty fix would be to just add page.url = page.url.replace(/#.*$/,''); before the if (crawledPages.has(page.url)) { so it removed the anchors but i'm not sure how that interferes with the original crawl function when people wanna build the graph