search

jgstew / jgstew-recipes

For JGStew's AutoPkg Recipes and Processors
MIT License
8 stars 2 forks source link

improve URLTextSearcher to take an array of re_patterns to run against the same url #6

Open jgstew opened 3 years ago

jgstew commented 3 years ago

The idea is to get the contents of a URL once, but then run multiple RegEx parses against the content to extract different things. Currently URLTextSearcher has to be used multiple times to achieve this and will download the web page multiple times to do so.

Example: https://github.com/jgstew/jgstew-recipes/blob/main/Microsoft/SQLServerManagementStudio-Win.download.recipe.yaml

jgstew commented 1 year ago

One workaround for this is to just download the webpage itself to the filesystem, then use URLTextSearcher multiple times on the cached copy.