Closed Teufelchen1 closed 1 year ago
First of all:
🌶️
As the example config gets bigger [..] write an src/example-config.yaml file exactly as the total result should be
Good idea.
and include it with include_str!("example-config.yaml")
TIL: include_str!
, Thanks!
The example can then be verified with a test like its currently being done already so its still ensured to be working.
There were a lot of ways to do that - at the moment, only one test build the example as before. This could be outsourced to an helper function? Not sure, trying to KISS.
Maybe also name it website-stalker.yaml and place it in the root folder while adding the sites folder to the .gitignore. Then local testing can just be cargo run -- run --all too.
That's the way to go! Please note that any change to the config results in a rebuild now. This is a bit annoying but okay for me.
Hi 👋
While messing around, I noticed that the example config tries to
css_select
stuff that is not present on the example website. To be precise:https://edjopato.de/post/
no longer has anarticle
class. This resulted in the following error:ERROR: https://edjopato.de/post/ css_selector (article) selected nothing
Further, the regex example seemed also outdated.This PR aims to fix both issues:
article
is replaced with.content
The regex now replaces a date in iso format with the string
ISO8601
As well as adding the following feature:
Additionally a refactoring of the code base was attempted:
const
variablemain.rs
to the top ofconfig.rs
There is a test for the ISO8601 regex in this PR - I think it should be removed before merge as it does not test behaviour of the website-stalker. I left it in for this / the first review only.