Support websites

[reaper47]

To be done right before release. Also check TODO comments.

Create a new migration with the following newly-supported websites:

• americastestkitchen
• nigella.com
• smittenkitchen.com
• downshiftology.com
• wellplated.com
• avocadoskillet.com
• beyondkimchee.com
• chatelaine.com
• chejorge.com
• cuisineandtravel.com
• daringgourmet.com
• dreenaburton.com
• elephantasticvegan.com
• entertainingwithbeth.com
• feastingathome.com
• foodal.com
• foodbymaria.com
• foodiecrush.com
• foolproofliving.com
• gastroplant.com
• goodto.com
• greenevi.com
• healthylittlefoodies.com
• jaroflemons.com
• natashaskitchen.com

Support these websites:

• [x] https://chinesecookingdemystified.substack.com/p/lao-gan-ma-fried-rice
• [ ] https://eda.ru/recepty/salaty/klassicheskij-krabovij-salat-s-ogurcom-47909
• [ ] http://www.etenvaneefke.nl/pastasalade/
• [ ] https://gurki.no/tomatsuppe
• [ ] https://www.cestmafournee.com/2024/02/crumble-banane-citron-vert-sauce.html
• [ ] https://food-guide.canada.ca/en/recipes/muhammara-dip-red-bell-pepper-walnut
• [ ] https://www.colruyt.be/nl/lekker-koken/recept/scampi-torpedo-met-chimichurri
• [ ] https://aniagotuje.pl/przepis/zupa-pomidorowa
• [ ] https://glutenfreetables.com/recipe/gluten-free-bread-loaf-bread-buns-with-schar-mix-b-flour/
• [ ] https://www.madewithlau.com/recipes/chicken-congee
• [ ] https://www.hellofresh.se/recipes/vietnamesisk-notfars-64143ec27bee82308d551397
• [ ] https://www.britishbakels.co.uk/recipes/multiseed-rolls/
• [ ] https://www.aldi.com.au/recipes/breakfast-recipes/spinach-crepes-with-smoked-salmon-recipe/
• [ ] https://www2.whole30.com/recipes/whole30-smash-burger-taco
• [x] https://drinkoteket.se/recept/margarita
• [ ] https://www.jocooks.com/recipes/korean-fried-chicken
• [ ] https://inbloombakery.com/the-best-cinnamon-rolls-ever
• [ ] https://www.leukerecepten.nl/recepten/pasta-met-paddenstoelen
• [ ] https://kitchenaid.com.au/blogs/kitchenthusiast/hot-cross-buns
• [ ] https://www.myjewishlearning.com/the-nosher/foolproof-sourdough-challah-recipe
• [ ] https://breadtopia.com/kubaneh-jewish-yemeni-bread
• [ ] https://kuchynalidla.sk/recepty/rychla-jablkova-rolada-s-karamelovou-polevou
• [ ] https://www.womensweeklyfood.com.au/recipe/baking/classic-lamingtons-17017
• [ ] https://joythebaker.com/2023/01/jambalaya-biscuits
• [ ] https://www.moulinex.fr/recette/detail/PRO/B%C5%93uf%20bourguignon/%2Fv1%2Frecipes%2F108166%2F
• [ ] https://www.ploetzblog.de/rezepte/roggenschrotbrot-2021/id=61a7372928ae7154616ab843
• [ ] https://www.bestrecipes.com.au/recipes/aussie-meat-pie-recipe-2/34l4qr5q
• [ ] https://www.ah.be/producten/product/wi576128/ah-rigatoni-met-meatballs-verspakket-bel
• [ ] https://vegan-pratique.fr/recettes/banana-bread/

[mblennegard]

@reaper47 What help is needed regarding this issue? The issue mentions create a new migration but I assume this is connected to scraping websites using the Fetch from website function? Is there any documentation for how to help adding new websites (I want to remember I saw something around this somewhere, but can't find anything now when searching GitHub and looking through the documentation)?

[reaper47]

The migration part of the issue enumerates websites recently supported that shall not be forgotten in the migration file: https://github.com/reaper47/recipya/blob/main/internal%2Fservices%2Fmigrations%2Ffor-release-v1.2.0

This file is where all newly-supported websites for v1.2.0 are dumped. The .sql extension will be added it before release for goose to pick it up.

The guide on how to support websites is here: https://recipes.musicavis.ca/guide/docs/development/workflow/import-website/. I need to update it a bit. What is left to do for this issue is to support the unchecked items above.

[mblennegard]

@reaper47 Oh, okay, so all websites mentioned in this ticket are already implemented as a supported website from a scraping perspective?

Note that the link https://recipes.musicavis.ca/guide/docs/development/workflow/ and its child pages are broken.

[reaper47]

Thank you for pointing the demo being down. It was caused by a panic in internal\services\file_apps:L337.

All unchecked websites mentioned in this issue are to be verified/supported. Often times, websites already have the LD+JSON script tag with all the recipe's information so there's not much to do except add a unit test and add the website to the SQL. When this tag is not present, the "domain not implemented" error is returned and elements from the website need to be scraped manually. There are plenty of examples in the scraper package.

[mblennegard]

@reaper47 The documentation site seems to be down altogether right now.

[reaper47]

Thank you for letting me know. Someone fetched a website that caused a panic:

Jun 26 17:19:17 musicavis recipya[1848782]:         recipya/internal/scraper/websites.go:195 +0x4105                                           

I will try to find the anomalous website from the logs.

Edit: It's one of these. I will fix it tomorrow.

https://www.maangchi.com/recipe/dububuchim-yangnyeomjang
 https://www.maangchi.com/recipe/yangnyeom-gejang
 https://www.maangchi.com/recipe/kkeopjilkong-maneul-bokkeum
 https://www.maangchi.com/recipe/asparagus-muchim
 https://www.maangchi.com/recipe/haemul-pajeon
 https://www.maangchi.com/recipe/gamja-haem-bokkeum
 https://www.maangchi.com/recipe/corn-cheese
 https://www.maangchi.com/recipe/rose-tteokbokki

 https://www.maangchi.com/recipe/oijangajji

 https://www.maangchi.com/recipe/wanja-jeon