crowdsecurity / example-docker-compose

Example integration of crowdsec in docker-compose
106 stars 25 forks source link

is Lepresidente custom image still needed? #17

Closed maxdd closed 3 months ago

maxdd commented 4 months ago

Hello, i have seen now that openresty is included in official npm. So if i had to integrate NPM with crowdsec, do we still need the custom NPM image lepresidente/nginxproxymanager:latest as reported here https://github.com/crowdsecurity/example-docker-compose/tree/main/npm?

LaurenceJJones commented 4 months ago

Hello, i have seen now that openresty is included in official npm. So if i had to integrate NPM with crowdsec, do we still need the custom NPM image lepresidente/nginxproxymanager:latest as reported here https://github.com/crowdsecurity/example-docker-compose/tree/main/npm?

Yes, even though openresty is included it doesnt include any of our lua files so they still need to be packaged together

maxdd commented 4 months ago

Is it possible to include an sh script via a volume to collect them and configure the original image perhaps?

LaurenceJJones commented 4 months ago

Is it possible to include an sh script via a volume to collect them and configure the original image perhaps?

No idea, we (crowdsec) have very limited experience with NPM hence why we rely on @LePresidente to compile the image for the community to use

LePresidente commented 4 months ago

Is it possible to include an sh script via a volume to collect them and configure the original image perhaps?

it might be possible with a cont init script, though its out of scope what I wanted to achieve, my release is basically just NPM with my two PR's included.

If they ever get merged i'll delete my releases.

https://github.com/NginxProxyManager/docker-nginx-full/pull/8/files https://github.com/NginxProxyManager/nginx-proxy-manager/pull/2677

maxdd commented 4 months ago

The issue for me is that jc21/nginx-proxy-manager:latest is still bugged in the hsts, see https://github.com/NginxProxyManager/nginx-proxy-manager/issues/3512

so i just wanted to apply it on top of the pr image jc21/nginx-proxy-manager:github-pr-3478

I admit it is a secondary issue and as long as you are still maintaining that's ok for me. Can i replace it as is without corrupting anything (in a normal scenario)?

LePresidente commented 4 months ago

it should be fixed in the latest release.

https://github.com/NginxProxyManager/nginx-proxy-manager/pull/3581

was removed in this PR and released as 2.11.2

I also nuked that line in my builds as well.

maxdd commented 4 months ago

yeah i was editing the post above, so can i use your version as a drop-in now, is it based on 2.11.2? I admit i still saw a 2.10.4 somewhere

LePresidente commented 4 months ago

Lepresidente/nginx-proxy-manager = jlesange/nginx-proxy-manager (up-to-date) (unraid fork I use) Lepresidente/nginxproxymanager = jc21/nginx-proxy-manager (up-to-date)

make sure you use the right image, I currently release two

maxdd commented 4 months ago

is Lepresidente/nginxproxymanager = jc21/nginx-proxy-manager (up-to-date) available on github? or did you only host it?

LePresidente commented 4 months ago

is Lepresidente/nginxproxymanager = jc21/nginx-proxy-manager (up-to-date) available on github? or did you only host it?

https://github.com/NginxProxyManager/docker-nginx-full/pull/8/files https://github.com/NginxProxyManager/nginx-proxy-manager/pull/2677

its just these PR's being built into the image.

my repo where those PR's are coming from, I have a manual shell script to do my builds on my VM. from these two repos. https://github.com/LePresidente/docker-nginx-full https://github.com/LePresidente/nginx-proxy-manager/tree/develop-crowdsec

maxdd commented 4 months ago

I'm getting

❯ Configuring npm user ...
useradd warning: npm's uid 0 outside of the UID_MIN 1000 and UID_MAX 60000 range.
❯ Configuring npm group ...
❯ Checking paths ...
❯ Setting ownership ...
❯ Dynamic resolvers ...
❯ IPv6 ...
Disabling IPV6 in hosts in: /etc/nginx/conf.d
- /etc/nginx/conf.d/crowdsec_openresty.conf
- /etc/nginx/conf.d/default.conf
- /etc/nginx/conf.d/production.conf
- /etc/nginx/conf.d/include/ssl-ciphers.conf
- /etc/nginx/conf.d/include/force-ssl.conf
- /etc/nginx/conf.d/include/ip_ranges.conf
- /etc/nginx/conf.d/include/block-exploits.conf
- /etc/nginx/conf.d/include/proxy.conf
- /etc/nginx/conf.d/include/assets.conf
- /etc/nginx/conf.d/include/letsencrypt-acme-challenge.conf
- /etc/nginx/conf.d/include/resolvers.conf
Disabling IPV6 in hosts in: /data/nginx
- /data/nginx/default_host/site.conf
- /data/nginx/proxy_host/5.conf
- /data/nginx/proxy_host/4.conf
- /data/nginx/proxy_host/6.conf
❯ Docker secrets ...
-------------------------------------
 _   _ ____  __  __
| \ | |  _ \|  \/  |
|  \| | |_) | |\/| |
| |\  |  __/| |  | |
|_| \_|_|   |_|  |_|
-------------------------------------
User:  npm PUID:0 ID:0 GROUP:0
Group: npm PGID:0 ID:0
-------------------------------------
Deploy Crowdsec Openresty Bouncer..
❯ Starting backend ...
sed: -e expression #1, char 29: unknown option to `s'
s6-rc: warning: unable to start service cs-crowdsec-bouncer: command exited 1
/run/s6/basedir/scripts/rc.init: warning: s6-rc failed to properly bring all the services up! Check your logs (in /run/uncaught-logs/current if you have in-container logging) for more information.
[5/7/2024] [2:03:43 PM] [Global   ] › ℹ  info      Using MySQL configuration
[5/7/2024] [2:03:44 PM] [Migrate  ] › ℹ  info      Current database version: 20211108145214
[5/7/2024] [2:03:44 PM] [Setup    ] › ℹ  info      Logrotate Timer initialized
[5/7/2024] [2:03:44 PM] [Global   ] › ⬤  debug     CMD: logrotate /etc/logrotate.d/nginx-proxy-manager
[5/7/2024] [2:03:44 PM] [Setup    ] › ℹ  info      Logrotate completed.
[5/7/2024] [2:03:44 PM] [IP Ranges] › ℹ  info      Fetching IP Ranges from online services...
[5/7/2024] [2:03:44 PM] [IP Ranges] › ℹ  info      Fetching https://ip-ranges.amazonaws.com/ip-ranges.json
[5/7/2024] [2:03:44 PM] [IP Ranges] › ℹ  info      Fetching https://www.cloudflare.com/ips-v4
[5/7/2024] [2:03:44 PM] [IP Ranges] › ℹ  info      Fetching https://www.cloudflare.com/ips-v6
[5/7/2024] [2:03:44 PM] [SSL      ] › ℹ  info      Let's Encrypt Renewal Timer initialized
[5/7/2024] [2:03:44 PM] [SSL      ] › ℹ  info      Renewing SSL certs expiring within 30 days ...
[5/7/2024] [2:03:44 PM] [IP Ranges] › ℹ  info      IP Ranges Renewal Timer initialized
[5/7/2024] [2:03:44 PM] [Global   ] › ℹ  info      Backend PID 164 listening on port 3000 ...
[5/7/2024] [2:03:44 PM] [SSL      ] › ℹ  info      Completed SSL cert renew process

with 'lepresidente/nginxproxymanager:latest'

i've also tried with a brand new folder without using my current setup (which btw works with latest jc21 image)

I didn't have much time to look into but seems like something here is complaining https://github.com/LePresidente/nginx-proxy-manager/blob/develop-crowdsec/docker/rootfs/etc/s6-overlay/s6-rc.d/cs-crowdsec-bouncer/script.sh

what exactly is this pipe?? https://github.com/crowdsecurity/example-docker-compose/blob/main/npm/docker-compose.yml#L21C35-L21C36 it seems that the pipe together with the string parsing are not right (tried with and without pipe)

CROWDSEC_OPENRESTY_BOUNCER:
        ENABLED=true
        API_URL=http://crowdsec:8080
        API_KEY=${CROWDSEC_BOUNCER_APIKEY}

For sure API_URL and API_KEY are not getting updated in /defaults/crowdsec/crowdsec-openresty-bouncer.conf

also in a cleaned deploy once i remove the pipe (which is still not making the conf right) i get a complaint here not creating the templates folder. To me /data is from npm and it is root

LePresidente commented 4 months ago

Yea looking at this now, fixed the script that parses the CROWDSEC_OPENRESTY_BOUNCER variable in the image and also found an issue in the example compose file.

LaurenceJJones commented 4 months ago

Note: If no further questions or issues arise, the issue will be closed within 5 days

pittbull commented 1 month ago

@LePresidente - I having been using 'lepresidente/nginx-proxy-manager' for many months now, and it still says "v2.10.4" when I log in. Is this correct, or should I be chasing down some other image to use?

LaurenceJJones commented 1 month ago

@LePresidente - I having been using 'lepresidente/nginx-proxy-manager' for many months now, and it still says "v2.10.4" when I log in. Is this correct, or should I be chasing down some other image to use?

Maybe best to open an issue directly on LePresidente fork, but have you brought down the running containers, then image pulled the latest and recreated the containers?

https://github.com/LePresidente/docker-nginx-proxy-manager

pittbull commented 1 month ago

I have indeed - tried with :latest and :dev as well. No change. But at this moment I am unsure which version of NPM I should be seeing with LePresidente's container.

pittbull commented 1 month ago

According to the dockerfile NPM should be at version 2.11.3, but this is not reflected in my container.

LaurenceJJones commented 1 month ago

According to the dockerfile NPM should be at version 2.11.3, but this is not reflected in my container.

Well can you link which image you are pulling? as it confusing as there 2 NPM's that are both forked for LePresidente

LePresidente commented 1 month ago

So the jlesange branch i'll try update today, i did a build a few hours ago but haven't tested it (reason its not tagged latest)

the official one should be on the latest image 2.11.3 (lepresidente/nginxproxymanager) used in this example

pittbull commented 1 month ago

According to the dockerfile NPM should be at version 2.11.3, but this is not reflected in my container.

Well can you link which image you are pulling? as it confusing as there 2 NPM's that are both forked for LePresidente

lepresidente/nginx-proxy-manager

pittbull commented 1 month ago

Lepresidente/nginx-proxy-manager = jlesange/nginx-proxy-manager (up-to-date) (unraid fork I use) Lepresidente/nginxproxymanager = jc21/nginx-proxy-manager (up-to-date)

@LePresidente, thanks for the quick feedback. According to this previous post I am using the jlesange-image. I have tried pulling both but the jc21-image fails to work with my config. Bascially just stopping here:


[7/24/2024] [3:01:37 PM] [Setup    ] › ℹ  info      Default settings added
[7/24/2024] [3:01:37 PM] [Setup    ] › ℹ  info      Logrotate Timer initialized
[7/24/2024] [3:01:37 PM] [Global   ] › ⬤  debug     CMD: logrotate /etc/logrotate.d/nginx-proxy-manager
[7/24/2024] [3:01:38 PM] [Setup    ] › ℹ  info      Logrotate completed.
[7/24/2024] [3:01:38 PM] [IP Ranges] › ℹ  info      Fetching IP Ranges from online services...
[7/24/2024] [3:01:38 PM] [IP Ranges] › ℹ  info      Fetching https://ip-ranges.amazonaws.com/ip-ranges.json
[7/24/2024] [3:01:38 PM] [IP Ranges] › ℹ  info      Fetching https://www.cloudflare.com/ips-v4
[7/24/2024] [3:01:38 PM] [IP Ranges] › ℹ  info      Fetching https://www.cloudflare.com/ips-v6
[7/24/2024] [3:01:38 PM] [SSL      ] › ℹ  info      Let's Encrypt Renewal Timer initialized
[7/24/2024] [3:01:38 PM] [SSL      ] › ℹ  info      Renewing SSL certs expiring within 30 days ...
[7/24/2024] [3:01:38 PM] [IP Ranges] › ℹ  info      IP Ranges Renewal Timer initialized
[7/24/2024] [3:01:38 PM] [Global   ] › ℹ  info      Backend PID 160 listening on port 3000 ...
[7/24/2024] [3:01:38 PM] [SSL      ] › ℹ  info      Completed SSL cert renew process

Based on your post a few seconds ago it is correct that I'm seeing 2.10.4 as jlesange has not been updated?

LePresidente commented 1 month ago

Yea, upstream was updated 3 weeks ago to 2.11.3, i just didn't notice so I haven't touched that branch.

https://github.com/jlesage/docker-nginx-proxy-manager

i'll try get it done today.

pittbull commented 1 month ago

Aha... Then I understand. Was getting confused as I thought something was wrong on my side.

Thank you kindly for provding support and updates.

pittbull commented 1 month ago

@LePresidente Thank you for recompiling 👍