PR: Intelligent Action Selection System

[csmangum]

This PR introduces a new intelligent action selection system that combines rule-based heuristics with deep Q-learning to help agents make better decisions. The system replaces the previous basic probability-based selection with a more sophisticated approach.

Key Changes

1. New Selection Module

• Added actions/select.py with dedicated selection logic
• Implements hybrid decision-making combining rules and Q-learning
• Uses 8-dimensional state representation for comprehensive agent context
• Configurable weights and multipliers for different actions

2. Agent Refactoring

• Removed old selection logic from base_agent.py
• Integrated new SelectModule into BaseAgent class
• Simplified agent learning process
• Improved state management

3. Configuration Updates

• Added selection-specific parameters to config.py and config.yaml
• Updated simulation parameters for better testing
• Added gathering and sharing module parameters
• Reduced default simulation steps from 2000 to 500

4. Documentation

• Added comprehensive docs/select_module.md
• Detailed explanation of selection process
• Usage examples and best practices
• Troubleshooting guide

Technical Details

The selection system uses:

• 8-dimensional state vector for decision making
• Epsilon-greedy exploration strategy
• Weighted combination of rules and Q-values
• GPU acceleration when available

Impact

This change should result in:

• More intelligent agent decision making
• Better resource management
• More realistic agent behavior
• Improved simulation performance

Risks and Mitigations

• Risk: Increased computational overhead
  • Mitigation: Vectorized operations and GPU support
• Risk: Learning instability
  • Mitigation: Configurable parameters and soft updates