everywall / ladder

Selfhosted alternative to 12ft.io. and 1ft.io bypass paywalls with a proxy ladder and remove CORS headers from any URL
GNU General Public License v3.0
4.41k stars 175 forks source link

refactor rulesets into separate files and add a ruleset compiler cli … #41

Closed deoxykev closed 10 months ago

deoxykev commented 10 months ago

This PR does the following:

  1. Break out the monolithic ruleset.yaml into an organized directory of smaller rulesets.
    rulesets
    ├── ca
    │  └── _multi-metroland-media-group.yaml
    ├── ch
    │  └── nzz-ch.yaml
    ├── de
    │  └── tagesspiegel-de.yaml
    ├── gb
    │  └── ft-com.yaml
    └── us
    ├── _multi-conde-nast.yaml
    ├── americanbanker-com.yaml
    ├── medium-com.yaml
    ├── nytimes-com.yaml
    ├── usatoday-com.yaml
    └── washingtonpost-com.yaml

Create a CLI option --merge-ruleset and --merge-ruleset-gzip that optionally merges the rulesets into a single file for release.


I'm not familiar enough with Github CI/CD and goreleaser to make changes with the releaser, but ideally the ruleset.yaml should be distributed with the release files.

Something like this in .github/workflows/release-binaries.yaml

        name: Build rulesets for release
        run: |
          go run cmd/main.go --ruleset "./rulesets" --merge-rulesets --merge-rulesets-output "ruleset.yaml"
mms-gianni commented 10 months ago

My first intention was to create a bash script. But this is way better.

  1. It ships with the app.
  2. it aligns automatically with the app.
  3. It is typesafe

Writing an action to install the binary and generate the rules is a challenge. But I think I can handle that.