search

cisagov / Malcolm

Malcolm is a powerful, easily deployable network traffic analysis tool suite for full packet capture artifacts (PCAP files), Zeek logs and Suricata alerts.
https://cisagov.github.io/Malcolm/
Other
1.91k stars 319 forks source link

Keystore: Value too large for defined data type #310

Closed malwareslayer closed 4 months ago

malwareslayer commented 4 months ago

Hello

OpenSearch instance immediately stop and error with log below with just regular docker-compose --profile malcolm up

Created opensearch keystore in /usr/share/opensearch/config/opensearch.keystore
mv: cannot create regular file './config/persist/opensearch.keystore': Value too large for defined data type
ln: failed to create symbolic link './config/opensearch.keystore': File exists

Any suggestions ?

mmguero commented 4 months ago
  1. Hmm... my guess is you should have run ./scripts/auth_setup to create certain files before starting Malcolm. You'll need to stop Malcolm (./scripts/stop) delete those files, as they've been created as directories now when docker started up, with a command like: 
    $ sudo rm -rf ./opensearch/opensearch.keystore ./logstash/certs/*.crt ./logstash/certs/*.key ./filebeat/certs/c*
  2. Then run ./scripts/auth_setup (as discussed here and here) and go through every option (select all)
  3. The suggested way to start Malcolm is to use the ./scripts/start script rather than calling docker-compose or docker compose directly, as described in the documentation. That will make sure that the necessary files are in place before docker starts and does its volume bind mounting.
malwareslayer commented 4 months ago

Still don't do anything. I do fix just by cp /usr/share/opensearch/opensearch.keystore to <malcolm>/opensearch then delete any mount to ./opensearch .

What i find with the related error is, Docker image doesn't support big file

mmguero commented 4 months ago

I don't understand your last comment, my apologies. Were you able to get it running, then?

malwareslayer commented 4 months ago

Able to get running and into dashboard, but not working, i.e: failed upload pcap in upload feature, the error still same with Value too large for defined data type

malwareslayer commented 4 months ago

Close this issues.

Solution: Its because my Docker builder that default to docker-desktop, taking the default volume options that created by docker-compose to incompatible with the host. Or making defined volume in docker-compose.yml.

volumes:
  nginx-log-path:
    driver: local
    driver_opts:
      type: ext4
      device: "${PWD}/nginx/logs"
      o: bind
  opensearch:
    driver: local
    driver_opts:
      type: ext4
      device: "${PWD}/opensearch"
      o: bind
  upload:
    driver: local
    driver_opts:
      type: ext4
      device: "${PWD}/pcap/upload"
      o: bind
  pcap:
    driver: local
    driver_opts:
      type: ext4
      device: "${PWD}/pcap"
      o: bind