search

truecharts / charts

Community Helm Chart Repository
https://truecharts.org
GNU Affero General Public License v3.0
1.13k stars 622 forks source link

[traefik]: unknown plugin type traefik-modsecurity-plugin #11797

Closed polarstack closed 1 year ago

polarstack commented 1 year ago

App Name

traefik

SCALE Version

22.02.3

App Version

2.10.4_21.0.0

Application Events

2023-08-22 19:43:16 Created container drawio
2023-08-22 19:43:16 Started container drawio
2023-08-22 19:43:15 Add eth0 [172.16.68.5/16] from ix-net
2023-08-22 19:43:15 Container image "tccr.io/truecharts/drawio:v21.6.8@sha256:bafb01a7356b6251343ef11ed9e2b1b616d0c9a4f7dfafdd189acda62c21a482" already present on machine
2023-08-22 19:43:14 Scaled up replica set drawio-c74d7469d to 1
2023-08-22 19:43:14 Created pod: drawio-c74d7469d-vczkv
2023-08-22 19:43:14 Successfully assigned ix-drawio/drawio-c74d7469d-vczkv to ix-truenas
2023-08-22 19:43:13 Scaled down replica set drawio-85c494c9f to 0 from 1
2023-08-22 19:43:13 Deleted pod: drawio-85c494c9f-vwmqz
2023-08-22 19:43:13 Stopping container drawio
2023-08-22 19:43:13 Successfully updated Certificate "drawio-tls-0"

Application Logs

time="2023-08-22T16:16:45+02:00" level=error msg="Skipping service: no endpoints found" providerName=kubernetes serviceName=drawio servicePort="&ServiceBackendPort{Name:,Number:10214,}" ingress=drawio namespace=ix-drawio
time="2023-08-22T16:16:45+02:00" level=error msg="Skipping service: no endpoints found" ingress=drawio serviceName=drawio servicePort="&ServiceBackendPort{Name:,Number:10214,}" namespace=ix-drawio providerName=kubernetes
time="2023-08-22T16:16:51+02:00" level=error msg="Skipping service: no endpoints found" serviceName=drawio namespace=ix-drawio servicePort="&ServiceBackendPort{Name:,Number:10214,}" providerName=kubernetes ingress=drawio
time="2023-08-22T16:17:07+02:00" level=error msg="plugin: unknown plugin type: traefik-modsecurity-plugin" entryPointName=websecure routerName=ix-drawio-drawio-diagrams-SUB-DOMAIN-TLD@kubernetes

Application Configuration

(modsecurity-crs): image

(traefik): image

(drawio as example, applicable to others as well) image

Describe the bug

image

To Reproduce

  1. Deploy whoami as custom-app
  2. Deploy modsecurity-crs and point BACKEND= to svc of whoami App
  3. Edit Traefik and add modsecurity plugin with defaults except for modsecurityUrl which has to be pointed to e.g http://modsecurity-crs.ix-modsecurity-crs.svc.cluster.local:8081
  4. Edit App of choice and add the configured modsecurity in middleware
  5. Go to traefik dashboard and inspect the ingress which shows

Expected Behavior

Plugin traefik-modsecurity-plugin should be loaded in the traefik router

Screenshots

image

Additional Context

Functionality of the traefik modsecurity plugin is described here: https://blog.kvak.net/post/2023-06-05_traefik-proxy-with-web-application-firewall-waf/

I've read and agree with the following

PrivatePuffin commented 1 year ago

This has nothing to do with modsecurity-crs chart, please edit the issue and write down the traefik chart version instead :)

polarstack commented 1 year ago

This has nothing to do with modsecurity-crs chart, please edit the issue and write down the traefik chart version instead :)

done :)

PrivatePuffin commented 1 year ago

Thx!

polarstack commented 1 year ago

I see in charts/enterprise/traefik/values.yaml an emptydir as plugin persistance:

persistence:
  plugins:
    enabled: true
    mountPath: "/plugins-storage"
    type: emptyDir

but when I shell into the traefik container, the folder is empty:

~ $ ls -l /plugins-storage
total 0

Am I misunderstanding something or has the plugin script a bug? https://github.com/truecharts/containers/blob/master/mirror/traefik/clone-plugins.sh

polarstack commented 1 year ago

This is how the process looks like inside traefik container with modsecurity middleware configured:

traefik traefik 
--global.checknewversion
--entryPoints.main.address=:9000/tcp
--entryPoints.metrics.address=:9180/tcp
--entryPoints.web.address=:80/tcp
--entryPoints.websecure.address=:443/tcp
--api.dashboard=true
--ping=true
--metrics.prometheus=true
--metrics.prometheus.entrypoint=metrics
--providers.kubernetescrd
--providers.kubernetesingress
--providers.kubernetesingress.ingressendpoint.publishedservice=ix-traefik/traefik-tcp
--entrypoints.web.http.redirections.entryPoint.to=:443
--entrypoints.web.http.redirections.entryPoint.scheme=https
--entrypoints.websecure.http.tls=true
--log.format=common
--log.level=WARN
--accesslog=true
--accesslog.format=common
--accesslog.filters.statuscodes=200,300-302
--accesslog.filters.retryattempts
--accesslog.filters.minduration=10ms
--accesslog.fields.defaultmode=keep
--accesslog.fields.headers.defaultmode=drop
--serverstransport.insecureskipverify=true
--providers.kubernetesingress.allowexternalnameservices=true

If I add the extra args in the app questions, the /plugin-storage folder gets populated with the corresponding module:

extra args:

--experimental.plugins.traefik-modsecurity-plugin.modulename=github.com/acouvreur/traefik-modsecurity-plugin
--experimental.plugins.traefik-modsecurity-plugin.version=v1.3.0

process inside container

traefik traefik 
--global.checknewversion
...<some lines omitted>...
--experimental.plugins.traefik-modsecurity-plugin.modulename=github.com/acouvreur/traefik-modsecurity-plugin
--experimental.plugins.traefik-modsecurity-plugin.version=v1.3.0

on storage side:

~ $ ls -l /plugins-storage/sources/gop-2564854005/src/github.com/acouvreur/traefik-modsecurity-plugin/
total 46
-rw-r--r--    1 568      568          11357 Aug 24 14:35 LICENSE
-rw-r--r--    1 568      568            101 Aug 24 14:35 Makefile
-rw-r--r--    1 568      568           2742 Aug 24 14:35 README.md
-rw-r--r--    1 568      568           1223 Aug 24 14:35 docker-compose.local.yml
-rw-r--r--    1 568      568           1212 Aug 24 14:35 docker-compose.yml
-rw-r--r--    1 568      568            283 Aug 24 14:35 go.mod
-rw-r--r--    1 568      568           1024 Aug 24 14:35 go.sum
drwxr-sr-x    2 568      568              5 Aug 24 14:35 img
-rw-r--r--    1 568      568           3826 Aug 24 14:35 modsecurity.go
-rw-r--r--    1 568      568           3717 Aug 24 14:35 modsecurity_test.go
-rw-r--r--    1 568      568            584 Aug 24 14:35 release.config.js
drwxr-sr-x    4 568      568              5 Aug 24 14:35 vendor

Any thoughts about this?

polarstack commented 1 year ago

Think I've found the issue. Re-checked the Dockerfile and saw the plugins are placed not in /plugins-storage/ but in /plugins-local/ 

~ $ ls -l /plugins-local/src/github.com/
total 3
drwxr-xr-x    3 root     root             3 Jul 29 15:42 PascalMinder
drwxr-xr-x    3 root     root             3 Jul 29 15:42 acouvreur
drwxr-xr-x    3 root     root             3 Jul 29 15:42 maxlerebourg
drwxr-xr-x    3 root     root             3 Jul 29 15:42 packruler
drwxr-xr-x    3 root     root             3 Jul 29 15:42 soulbalz

Which is the way according to the blog post about the new feature: https://traefik.io/blog/using-private-plugins-in-traefik-proxy-2-5/

Nevertheless, if you copy a plugin you have also to start traefik with extra args --experimental.localPlugins......= which seems to happen here https://github.com/truecharts/charts/blob/10b88d86a8a7b2b98195fc1dafbe745f64d043dd/charts/enterprise/traefik/templates/_args.tpl where modsecurity is completely missing

polarstack commented 1 year ago

Hey @xstar97

Do you agree that the code below pasted in https://github.com/truecharts/charts/blob/10b88d86a8a7b2b98195fc1dafbe745f64d043dd/charts/enterprise/traefik/templates/_args.tpl after line 178 would fix the issue? 

  {{/* ModSecurity */}}
  {{- if .Values.middlewares.modsecurity }}
  - "--experimental.localPlugins.traefik-modsecurity-plugin.modulename=github.com/acouvreur/traefik-modsecurity-plugin"
  {{- end }}
  {{/* End of ModSecurity */}}

Additionally https://github.com/truecharts/charts/blob/master/.github/scripts/updateTraefikMiddlewareVersions.sh

# ModSecurity
update_plugin "acouvreur/traefik-modsecurity-plugin" "modsecurityVersion" "ModSecurity"

maybe if you could check for typos or if .Values.middlewares.modsecurity is the right path.

From my perspective this fix would make the https://github.com/truecharts/charts/pull/11791 obsolete - agree?

PrivatePuffin commented 1 year ago

Cleaning all the hacks and unsupported setups here.

PrivatePuffin commented 1 year ago

Hey @xstar97

Do you agree that the code below pasted in https://github.com/truecharts/charts/blob/10b88d86a8a7b2b98195fc1dafbe745f64d043dd/charts/enterprise/traefik/templates/_args.tpl after line 178 would fix the issue?

  {{/* ModSecurity */}}
  {{- if .Values.middlewares.modsecurity }}
  - "--experimental.localPlugins.traefik-modsecurity-plugin.modulename=github.com/acouvreur/traefik-modsecurity-plugin"
  {{- end }}
  {{/* End of ModSecurity */}}

Additionally https://github.com/truecharts/charts/blob/master/.github/scripts/updateTraefikMiddlewareVersions.sh

# ModSecurity
update_plugin "acouvreur/traefik-modsecurity-plugin" "modsecurityVersion" "ModSecurity"

maybe if you could check for typos or if .Values.middlewares.modsecurity is the right path.

From my perspective this fix would make the #11791 obsolete - agree?

Please don't tag staff members not involved, XStar is not a maintainer for either stable or enterprise trains

truecharts-admin commented 1 year ago

This issue is locked to prevent necro-posting on closed issues. Please create a new issue or contact staff on discord of the problem persists