Temporary Files Accumulating in /tmp Folder After Starting AppSec

TokuiNico commented 5 months ago

What happened?

I started to use AppSec component 1 month ago and everything went well. However, I found After starting AppSec, the /tmp folder will quickly increase with a large number of files, the file names will be in a format like crzmp1015053877.

In less than an hour, there will be an increase of over 200 MB of temporary files.

What are these files? Is there any way to remove them?

Thanks

What did you expect to happen?

I expected the AppSec component to run without generating a large number of temporary files in the /tmp folder.

How can we reproduce it (as minimally and precisely as possible)?

Start the AppSec component with these config.

# acquis.d/appsec.yaml
source: appsec
listen_addr: 0.0.0.0:7422
appsec_config: crowdsecurity/virtual-patching
labels:
type: appsec

# deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: crowdsec-appsec
  namespace: crowdsec
  labels:
    k8s-app: crowdsec
    type: appsec
spec:
  replicas: 3
  selector:
    matchLabels:
      k8s-app: crowdsec
      type: appsec
  template:
    metadata:
      labels:
        k8s-app: crowdsec
        type: appsec
    spec:
      containers:
      - name: appsec
        image: crowdsecurity/crowdsec:v1.6.1-2
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 7422

        env:
        - name: COLLECTIONS
          value: >-
            crowdsecurity/appsec-virtual-patching
            crowdsecurity/appsec-generic-rules

        - name: DISABLE_LOCAL_API
          value: "true"
        - name: DISABLE_ONLINE_API
          value: "true"

set appsec to get http request
Monitor the /tmp folder for an increase in files.
Observe the accumulation of files named in the format crzmp[digits].

Anything else we need to know?

No response

Crowdsec version

docker image: crowdsecurity/crowdsec:v1.6.1-2 ```console $ cscli version 2024/06/05 15:00:49 version: v1.6.1-c6e40191 2024/06/05 15:00:49 Codename: alphaga 2024/06/05 15:00:49 BuildDate: 2024-04-18_14:50:12 2024/06/05 15:00:49 GoVersion: 1.21.9 2024/06/05 15:00:49 Platform: docker 2024/06/05 15:00:49 libre2: C++ 2024/06/05 15:00:49 Constraint_parser: >= 1.0, <= 3.0 2024/06/05 15:00:49 Constraint_scenario: >= 1.0, <= 3.0 2024/06/05 15:00:49 Constraint_api: v1 2024/06/05 15:00:49 Constraint_acquis: >= 1.0, < 2.0 ```

OS version

```console # On Linux: $ cat /etc/os-release NAME="Alpine Linux" ID=alpine VERSION_ID=3.19.1 PRETTY_NAME="Alpine Linux v3.19" HOME_URL="https://alpinelinux.org/" BUG_REPORT_URL="https://gitlab.alpinelinux.org/alpine/aports/-/issues" $ uname -a Linux crowdsec-appsec-5ff6c7b45d-s2m5l 5.10.214-202.855.amzn2.aarch64 #1 SMP Tue Apr 9 06:57:15 UTC 2024 aarch64 Linux ```

Enabled collections and parsers

```console $ cscli hub list -o raw name,status,version,description,type crowdsecurity/appsec-logs,enabled,0.5,Parse Appsec events,parsers crowdsecurity/cri-logs,enabled,0.1,CRI logging format parser,parsers crowdsecurity/dateparse-enrich,enabled,0.2,,parsers crowdsecurity/docker-logs,enabled,0.1,docker json logs parser,parsers crowdsecurity/geoip-enrich,enabled,0.3,"Populate event with geoloc info : as, country, coords, source range.",parsers crowdsecurity/sshd-logs,enabled,2.3,Parse openSSH logs,parsers crowdsecurity/syslog-logs,enabled,0.8,,parsers crowdsecurity/whitelists,enabled,0.2,Whitelist events from private ipv4 addresses,parsers crowdsecurity/appsec-vpatch,enabled,0.5,Identify attacks flagged by CrowdSec AppSec,scenarios crowdsecurity/ssh-bf,enabled,0.3,Detect ssh bruteforce,scenarios crowdsecurity/ssh-slow-bf,enabled,0.4,Detect slow ssh bruteforce,scenarios crowdsecurity/appsec_base,enabled,0.2,,contexts crowdsecurity/bf_base,enabled,0.1,,contexts crowdsecurity/appsec-default,enabled,0.1,,appsec-configs crowdsecurity/generic-rules,enabled,0.3,,appsec-configs crowdsecurity/virtual-patching,enabled,0.4,,appsec-configs crowdsecurity/base-config,enabled,0.1,,appsec-rules crowdsecurity/generic-freemarker-ssti,enabled,0.3,Generic FreeMarker SSTI,appsec-rules crowdsecurity/vpatch-connectwise-auth-bypass,enabled,0.3,Detect exploitation of auth bypass in ConnectWise ScreenConnect,appsec-rules crowdsecurity/vpatch-CVE-2017-9841,enabled,0.3,PHPUnit RCE (CVE-2017-9841),appsec-rules crowdsecurity/vpatch-CVE-2018-1000861,enabled,0.1,Jenkins - RCE (CVE-2018-1000861),appsec-rules crowdsecurity/vpatch-CVE-2018-10562,enabled,0.2,Dasan GPON RCE (CVE-2018-10562),appsec-rules crowdsecurity/vpatch-CVE-2019-1003030,enabled,0.1,Jenkins - RCE (CVE-2019-1003030),appsec-rules crowdsecurity/vpatch-CVE-2019-12989,enabled,0.3,Citrix SQLi (CVE-2019-12989),appsec-rules crowdsecurity/vpatch-CVE-2020-11738,enabled,0.6,Wordpress Snap Creek Duplicator - Path Traversal (CVE-2020-11738),appsec-rules crowdsecurity/vpatch-CVE-2020-17496,enabled,0.1,vBulletin RCE (CVE-2020-17496),appsec-rules crowdsecurity/vpatch-CVE-2021-22941,enabled,0.3,Citrix RCE (CVE-2021-22941),appsec-rules crowdsecurity/vpatch-CVE-2021-3129,enabled,0.4,Laravel with Ignition Debug Mode RCE (CVE-2021-3129),appsec-rules crowdsecurity/vpatch-CVE-2022-22954,enabled,0.2,VMWare Workspace ONE Access RCE (CVE-2022-22954),appsec-rules crowdsecurity/vpatch-CVE-2022-22965,enabled,0.2,Spring4Shell - RCE (CVE-2022-22965),appsec-rules crowdsecurity/vpatch-CVE-2022-27926,enabled,0.4,Zimbra Collaboration XSS (CVE-2022-27926),appsec-rules crowdsecurity/vpatch-CVE-2022-35914,enabled,0.5,GLPI RCE (CVE-2022-35914),appsec-rules crowdsecurity/vpatch-CVE-2022-44877,enabled,0.2,CentOS Web Panel 7 RCE (CVE-2022-44877),appsec-rules crowdsecurity/vpatch-CVE-2022-46169,enabled,0.5,Cacti RCE (CVE-2022-46169),appsec-rules crowdsecurity/vpatch-CVE-2023-1389,enabled,0.1,TP-Link Archer AX21 - RCE (CVE-2023-1389),appsec-rules crowdsecurity/vpatch-CVE-2023-20198,enabled,0.6,CISCO IOS XE Account Creation (CVE-2023-20198),appsec-rules crowdsecurity/vpatch-CVE-2023-22515,enabled,0.4,Atlassian Confluence Privesc (CVE-2023-22515),appsec-rules crowdsecurity/vpatch-CVE-2023-22527,enabled,0.2,RCE using SSTI in Confluence (CVE-2023-22527),appsec-rules crowdsecurity/vpatch-CVE-2023-23752,enabled,0.1,Joomla! Webservice - Password Disclosure (CVE-2023-23752),appsec-rules crowdsecurity/vpatch-CVE-2023-24489,enabled,0.2,Citrix ShareFile RCE (CVE-2023-24489),appsec-rules crowdsecurity/vpatch-CVE-2023-28121,enabled,0.1,WooCommerce auth bypass (CVE-2023-28121),appsec-rules crowdsecurity/vpatch-CVE-2023-33617,enabled,0.4,Atlassian Confluence Privesc (CVE-2023-33617),appsec-rules crowdsecurity/vpatch-CVE-2023-34362,enabled,0.6,MOVEit Transfer RCE (CVE-2023-34362),appsec-rules crowdsecurity/vpatch-CVE-2023-35078,enabled,0.1,MobileIron Core Remote Unauthenticated API Access (CVE-2023-35078),appsec-rules crowdsecurity/vpatch-CVE-2023-35082,enabled,0.2,MobileIron Core Remote Unauthenticated API Access (CVE-2023-35082),appsec-rules crowdsecurity/vpatch-CVE-2023-3519,enabled,0.3,Citrix RCE (CVE-2023-3519),appsec-rules crowdsecurity/vpatch-CVE-2023-38205,enabled,0.3,Adobe ColdFusion Access Control Bypass (CVE-2023-38205),appsec-rules crowdsecurity/vpatch-CVE-2023-40044,enabled,0.3,WS_FTP .NET deserialize RCE (CVE-2023-40044),appsec-rules crowdsecurity/vpatch-CVE-2023-42793,enabled,0.3,JetBrains Teamcity Auth Bypass (CVE-2023-42793),appsec-rules crowdsecurity/vpatch-CVE-2023-46805,enabled,0.4,Ivanti Connect Auth Bypass (CVE-2023-46805),appsec-rules crowdsecurity/vpatch-CVE-2023-49070,enabled,0.1,Apache OFBiz - RCE (CVE-2023-49070),appsec-rules crowdsecurity/vpatch-CVE-2023-50164,enabled,0.6,Apache Struts2 Path Traversal (CVE-2023-50164),appsec-rules crowdsecurity/vpatch-CVE-2023-6553,enabled,0.1,Backup Migration plugin for WordPress RCE (CVE-2023-6553),appsec-rules crowdsecurity/vpatch-CVE-2023-7028,enabled,0.2,Gitlab Password Reset Account Takeover (CVE-2023-7028),appsec-rules crowdsecurity/vpatch-CVE-2024-1212,enabled,0.3,Progress Kemp LoadMaster Unauthenticated Command Injection (CVE-2024-1212),appsec-rules crowdsecurity/vpatch-CVE-2024-22024,enabled,0.1,Ivanti Connect Secure - XXE (CVE-2024-22024),appsec-rules crowdsecurity/vpatch-CVE-2024-23897,enabled,0.4,Jenkins CLI RCE (CVE-2024-23897),appsec-rules crowdsecurity/vpatch-CVE-2024-27198,enabled,0.4,Teamcity - Authentication Bypass (CVE-2024-27198),appsec-rules crowdsecurity/vpatch-CVE-2024-3273,enabled,0.1,D-LINK NAS Command Injection (CVE-2024-3273),appsec-rules crowdsecurity/vpatch-env-access,enabled,0.1,Detect access to .env files,appsec-rules crowdsecurity/vpatch-laravel-debug-mode,enabled,0.3,Detect bots exploiting laravel debug mode,appsec-rules crowdsecurity/vpatch-symfony-profiler,enabled,0.1,Detect abuse of symfony profiler,appsec-rules crowdsecurity/appsec-generic-rules,enabled,0.5,A collection of generic attack vectors for additional protection.,collections crowdsecurity/appsec-virtual-patching,enabled,2.6,"a generic virtual patching collection, suitable for most web servers.",collections crowdsecurity/linux,enabled,0.2,core linux support : syslog+geoip+ssh,collections crowdsecurity/sshd,enabled,0.3,sshd support : parser and brute-force detection,collections ```

Acquisition config

```console # On Linux: $ cat /etc/crowdsec/acquis.yaml /etc/crowdsec/acquis.d/* filenames: - /var/log/nginx/*.log - ./tests/nginx/nginx.log #this is not a syslog log, indicate which kind of logs it is labels: type: nginx --- filenames: - /var/log/auth.log - /var/log/syslog labels: type: syslog --- filename: /var/log/apache2/*.log labels: type: apache2 source: appsec listen_addr: 0.0.0.0:7422 appsec_config: crowdsecurity/virtual-patching labels: type: appsec

Config show

```console $ cscli config show Global: - Configuration Folder : /etc/crowdsec - Data Folder : /var/lib/crowdsec/data - Hub Folder : /etc/crowdsec/hub - Simulation File : /etc/crowdsec/simulation.yaml - Log Folder : /var/log - Log level : info - Log Media : stdout Crowdsec: - Acquisition File : /etc/crowdsec/acquis.yaml - Parsers routines : 1 - Acquisition Folder : /etc/crowdsec/acquis.d cscli: - Output : human - Hub Branch : API Client: - URL : http://crowdsec-service.crowdsec:8080/ - Login : p5mPeRdISfXbIKskOR63oemWoB2OWEDVCey17bKG1mKEiATX - Credentials File : /etc/crowdsec/local_api_credentials.yaml Local API Server (disabled): - Listen URL : 0.0.0.0:8080 - Listen Socket : - Profile File : /etc/crowdsec/profiles.yaml - Trusted IPs: - 127.0.0.1 - ::1 - Database: - Type : sqlite - Path : /var/lib/crowdsec/data/crowdsec.db - Flush age : 7d - Flush size : 5000 ```

Prometheus metrics

```console $ cscli metrics Appsec Metrics: ╭───────────────┬───────────┬─────────╮ │ Appsec Engine │ Processed │ Blocked │ ├───────────────┼───────────┼─────────┤ │ 0.0.0.0:7422/ │ 375.13k │ - │ ╰───────────────┴───────────┴─────────╯ ```

Related custom configs versions (if applicable) : notification plugins, custom scenarios, parsers etc.

github-actions[bot] commented 5 months ago

@TokuiNico: Thanks for opening an issue, it is currently awaiting triage.

In the meantime, you can:

Check Crowdsec Documentation to see if your issue can be self resolved.
You can also join our Discord.
Check Releases to make sure your agent is on the latest version.

Details

I am a bot created to help the [crowdsecurity](https://github.com/crowdsecurity) developers manage community feedback and contributions. You can check out my [manifest file](https://github.com/crowdsecurity/crowdsec/blob/master/.github/governance.yml) to understand my behavior and what I can do. If you want to use this for your project, you can check out the [BirthdayResearch/oss-governance-bot](https://github.com/BirthdayResearch/oss-governance-bot) repository.

victoredvardsson commented 5 months ago

Hello!

I also stumbled on this issue and it's actually a "bug" in coraza. I reported it to them and it's fixed but will be released in version 3.2

https://github.com/corazawaf/coraza/issues/922#issuecomment-2095964206

My hotfix for this is to run a scheduled cron once a day to cleanup.

find /tmp/ -type f -regex '.*/\(crzmp.*\|body.*\)' -mmin +5 -delete

TokuiNico commented 5 months ago

Thank you for your help. I will apply hotfix first.

crowdsecurity / crowdsec