mrheinen / lophiid

A distributed honeypot for monitoring large scale web attacks
GNU General Public License v2.0
6 stars 1 forks source link

Add command execution to content scripts, many bug fixes #30

Closed mrheinen closed 1 month ago

mrheinen commented 1 month ago

PR Type

Enhancement, Bug fix


Description


Changes walkthrough πŸ“

Relevant files
Configuration changes
4 files
api_server.go
Add scripting configuration and update JavaScript runner 

cmd/api/api_server.go
  • Added new 'Scripting' configuration struct with 'AllowedCommands' and
    'CommandTimeout' fields
  • Updated 'NewGojaJavascriptRunner' function call with new parameters
  • +7/-1     
    backend_main.go
    Update JavaScript runner initialization                                   

    cmd/backend/backend_main.go
  • Updated 'NewGojaJavascriptRunner' function call with new parameters
    from config
  • +1/-1     
    config.go
    Add scripting configuration to backend                                     

    pkg/backend/config.go
  • Added 'Scripting' struct with 'AllowedCommands' and 'CommandTimeout'
    fields to Config
  • +5/-0     
    backend-config.yaml
    Add scripting configuration to backend config                       

    config/backend-config.yaml - Added 'scripting' section with 'allowed_commands' configuration
    +5/-1     
    Miscellaneous
    1 files
    main.go
    Remove whois command implementation                                           

    cmd/whois/main.go - Entire file removed
    +0/-35   
    Enhancement
    7 files
    database.go
    Add body setter and fix content rule query                             

    pkg/database/database.go
  • Added 'SetBodyString' method to Request struct
  • Fixed query building in 'SearchContentRules' function
  • +3/-1     
    command_runner.go
    Add command execution functionality                                           

    pkg/javascript/command_runner.go
  • Implemented new file for command execution functionality
  • Added 'CommandRunner' interface and 'SingleCommandRunner' struct
  • Implemented 'RunCommand' method with security checks
  • +95/-0   
    crypto.go
    Add crypto utility functions                                                         

    pkg/javascript/crypto.go
  • Implemented new file for crypto operations
  • Added methods for MD5, SHA256, and SHA1 checksums
  • +34/-0   
    encoding.go
    Add encoding utility functions                                                     

    pkg/javascript/encoding.go
  • Implemented new file for encoding operations
  • Added Base64 encoding and decoding methods
  • +29/-0   
    goja.go
    Refactor JavaScript runner and add new utilities                 

    pkg/javascript/goja.go
  • Removed crypto, encoding, and time-related code (moved to separate
    files)
  • Updated 'GojaJavascriptRunner' struct and constructor
  • Modified 'RunScript' method to include new utility functions
  • +17/-149
    time.go
    Add time utility function                                                               

    pkg/javascript/time.go
  • Implemented new file for time-related operations
  • Added 'Sleep' method for JavaScript context
  • +11/-0   
    wrappers.go
    Refactor JavaScript wrappers into separate file                   

    pkg/javascript/wrappers.go
  • Implemented new file for various wrapper structs and methods
  • Moved existing wrapper code from goja.go to this file
  • +84/-0   
    Tests
    1 files
    goja_test.go
    Update JavaScript runner tests                                                     

    pkg/javascript/goja_test.go
  • Updated test cases to include new command execution functionality
  • Modified 'NewGojaJavascriptRunner' calls in tests
  • +122/-4 
    Documentation
    1 files
    SCRIPTING.md
    Update scripting documentation                                                     

    SCRIPTING.md
  • Added documentation for new command execution functionality
  • Updated overview of available scripting methods
  • +59/-0   

    πŸ’‘ PR-Agent usage: Comment /help on the PR to get a list of all available PR-Agent tools and their descriptions

    github-actions[bot] commented 1 month ago

    Preparing PR description...

    github-actions[bot] commented 1 month ago

    Preparing review...

    codiumai-pr-agent-pro[bot] commented 1 month ago

    PR Reviewer Guide πŸ”

    ⏱️ Estimated effort to review: 4 πŸ”΅πŸ”΅πŸ”΅πŸ”΅βšͺ
    πŸ§ͺ PR contains tests
    πŸ”’ Security concerns

    Command execution:
    The PR introduces command execution functionality in content scripts. This feature, if not properly restricted and sanitized, could lead to remote code execution vulnerabilities. Ensure that the `allowedCommands` list is strictly controlled and that user input is never directly used as command arguments without proper validation and sanitization.
    ⚑ Key issues to review

    Security Concern
    The command execution functionality allows running arbitrary commands. Ensure proper input validation and sanitization are in place. Potential Performance Issue
    The `NewGojaJavascriptRunner` function creates a new cache with a 30-minute timeout. Consider making this timeout configurable. SQL Injection Risk
    The `SearchContentRules` function uses dynamic SQL query building. Ensure proper parameterization is used to prevent SQL injection.
    codiumai-pr-agent-pro[bot] commented 1 month ago

    PR Code Suggestions ✨

    CategorySuggestion                                                                                                                                    Score
    Security
    Implement input validation to prevent potential command injection vulnerabilities ___ **Consider implementing input validation for the command and args parameters in the
    RunCommand method to prevent potential command injection vulnerabilities.** [pkg/javascript/command_runner.go [63-81]](https://github.com/mrheinen/lophiid/pull/30/files#diff-add6ec727371609ae146df0e03ac2124dc03934f235b6867c5d868b37bd827cdR63-R81) ```diff func (s *SingleCommandRunner) RunCommand(command string, args ...string) bool { if !util.Contains(s.allowedCommands, command) { slog.Error("command is not allowed", slog.String("command", command)) s.Error = fmt.Errorf("command %s is not allowed", command) + return false + } + + // Validate command and args + if !validateInput(command) || !validateInputs(args) { + slog.Error("invalid input detected", slog.String("command", command)) + s.Error = fmt.Errorf("invalid input detected") return false } cmd := exec.Command(command, args...) cmd.Stdout = &s.Stdout cmd.Stderr = &s.Stderr if err := cmd.Run(); err != nil { s.Error = err slog.Error("command failed", slog.String("error", err.Error()), slog.String("stderr", s.Stderr.String())) return false } return true } +func validateInput(input string) bool { + // Implement input validation logic + return true +} + +func validateInputs(inputs []string) bool { + for _, input := range inputs { + if !validateInput(input) { + return false + } + } + return true +} + ``` - [ ] **Apply this suggestion**
    Suggestion importance[1-10]: 9 Why: This suggestion addresses a critical security vulnerability by preventing potential command injection attacks.
    9
    βœ… Implement resource limits and timeout for executed commands to prevent potential denial-of-service attacks ___
    Suggestion Impact:The commit implemented a timeout mechanism for command execution, which addresses part of the suggestion's goal to prevent potential denial-of-service attacks. code diff: ```diff + ctx, cancel := context.WithTimeout(context.Background(), s.commandTimeout) + defer cancel() + + cmd := exec.CommandContext(ctx, command, args...) cmd.Stdout = &s.Stdout cmd.Stderr = &s.Stderr - if err := cmd.Run(); err != nil { + if err := cmd.Start(); err != nil { + s.Error = err + slog.Error("command failed to start", slog.String("error", err.Error()), slog.String("stderr", s.Stderr.String())) + return false + } + + if err := cmd.Wait(); err != nil { s.Error = err slog.Error("command failed", slog.String("error", err.Error()), slog.String("stderr", s.Stderr.String())) ```
    ___ **Consider implementing a mechanism to limit the resources (CPU, memory, etc.) that
    can be used by the executed commands to prevent potential denial-of-service attacks.** [pkg/javascript/command_runner.go [63-81]](https://github.com/mrheinen/lophiid/pull/30/files#diff-add6ec727371609ae146df0e03ac2124dc03934f235b6867c5d868b37bd827cdR63-R81) ```diff func (s *SingleCommandRunner) RunCommand(command string, args ...string) bool { if !util.Contains(s.allowedCommands, command) { slog.Error("command is not allowed", slog.String("command", command)) s.Error = fmt.Errorf("command %s is not allowed", command) return false } cmd := exec.Command(command, args...) cmd.Stdout = &s.Stdout cmd.Stderr = &s.Stderr - if err := cmd.Run(); err != nil { + // Set resource limits + cmd.SysProcAttr = &syscall.SysProcAttr{ + Setpgid: true, + } + + if err := cmd.Start(); err != nil { s.Error = err - slog.Error("command failed", slog.String("error", err.Error()), slog.String("stderr", s.Stderr.String())) + slog.Error("command failed to start", slog.String("error", err.Error())) + return false + } + + done := make(chan error) + go func() { done <- cmd.Wait() }() + + select { + case err := <-done: + if err != nil { + s.Error = err + slog.Error("command failed", slog.String("error", err.Error()), slog.String("stderr", s.Stderr.String())) + return false + } + case <-time.After(30 * time.Second): + if err := syscall.Kill(-cmd.Process.Pid, syscall.SIGKILL); err != nil { + slog.Error("failed to kill process", slog.String("error", err.Error())) + } + s.Error = fmt.Errorf("command timed out") return false } return true } ``` - [ ] **Apply this suggestion**
    Suggestion importance[1-10]: 9 Why: This suggestion significantly enhances security by preventing potential denial-of-service attacks through resource limitation and timeout mechanisms.
    9
    Enhancement
    βœ… Add a timeout mechanism to prevent long-running commands from blocking indefinitely ___
    Suggestion Impact:The commit implemented a timeout mechanism using context.WithTimeout, as suggested. It also added a commandTimeout field to the struct and modified the constructor to accept this timeout value. code diff: ```diff + ctx, cancel := context.WithTimeout(context.Background(), s.commandTimeout) + defer cancel() + + cmd := exec.CommandContext(ctx, command, args...) cmd.Stdout = &s.Stdout cmd.Stderr = &s.Stderr - if err := cmd.Run(); err != nil { + if err := cmd.Start(); err != nil { + s.Error = err + slog.Error("command failed to start", slog.String("error", err.Error()), slog.String("stderr", s.Stderr.String())) + return false + } + + if err := cmd.Wait(); err != nil { s.Error = err ```
    ___ **Consider adding a timeout mechanism to the RunCommand method to prevent long-running
    commands from blocking indefinitely.** [pkg/javascript/command_runner.go [63-81]](https://github.com/mrheinen/lophiid/pull/30/files#diff-add6ec727371609ae146df0e03ac2124dc03934f235b6867c5d868b37bd827cdR63-R81) ```diff func (s *SingleCommandRunner) RunCommand(command string, args ...string) bool { if !util.Contains(s.allowedCommands, command) { slog.Error("command is not allowed", slog.String("command", command)) s.Error = fmt.Errorf("command %s is not allowed", command) return false } - cmd := exec.Command(command, args...) + ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second) + defer cancel() + + cmd := exec.CommandContext(ctx, command, args...) cmd.Stdout = &s.Stdout cmd.Stderr = &s.Stderr if err := cmd.Run(); err != nil { s.Error = err slog.Error("command failed", slog.String("error", err.Error()), slog.String("stderr", s.Stderr.String())) return false } return true } ``` - [ ] **Apply this suggestion**
    Suggestion importance[1-10]: 8 Why: This suggestion addresses a potential security issue by preventing indefinite blocking, which could lead to resource exhaustion.
    8
    Improve field naming for better code readability and understanding ___ **Consider using a more descriptive name for the strCache field, such as scriptCache
    or javascriptCache, to better reflect its purpose in caching JavaScript scripts.** [pkg/javascript/goja.go [47]](https://github.com/mrheinen/lophiid/pull/30/files#diff-93ebbcb2bc5cc6ca7ef6d25c1ce739d50fc1194101ed2b4469c1941da4111828R47-R47) ```diff -strCache *util.StringMapCache[string] +scriptCache *util.StringMapCache[string] ``` - [ ] **Apply this suggestion**
    Suggestion importance[1-10]: 6 Why: The suggestion improves code readability by using a more descriptive name, but it's a minor change that doesn't address major issues or bugs.
    6
    mrheinen commented 1 month ago

    The command timeout was actually a good suggestion by the PR agent so I applied that one (via own commit) and made it configurable from the config in the backend and API

    mrheinen commented 1 month ago

    /describe

    github-actions[bot] commented 1 month ago

    Preparing PR description...

    codiumai-pr-agent-pro[bot] commented 1 month ago

    PR Description updated to latest commit (https://github.com/mrheinen/lophiid/commit/c3d8caacb478c9ab730daf2a1c6b1492b7a1c970)