search

random-mdn / random-mdn-bot

Serverless functions tweeting/sending/... random MDN articles
MIT License
14 stars 5 forks source link

Fetch MDN sitemap #3

Closed stefanjudis closed 6 years ago

stefanjudis commented 6 years ago

@malgosiastp I set up the basic structure and I think it's good enough for now. As a first step we would have to find a way to fetch and parse the MDN sitemap.

The project now includes on function which is defined in handler.js. This function is also deployed already to AWS but we don't have to worry about the online version yet because I only put it there to see it running.

handler.js uses a random npm dependency to see dependencies working, too.

To fetch the MDN sitemap I believe there are two things that need to be done:

  1. fetch the sitemap
  2. parse it to JSON

To fetch files in Node.js packages like node-fetch (which I use on regular basis) or ky (everything by Sindre Sorhus is high quality) might be a good fit. You could add one of them to the package.json and then use the package of choice in the handler function.

When you have the data available a XML parser like xml2j (I didn't test it though) can parse the sitemap into json. :)

For the scope of this ticket I believe it's good enough to log the "JSON sitemap" to the console. You should see the result when you run npm run dev. :)

If you have any questions, I'm happy to help.

stefanjudis commented 6 years ago

Couldn't hold it and had a free weekend. Sorry. :(