Action System

[csmangum]

Multi-Agent System Enhancement PR

Overview

This PR introduces significant enhancements to our multi-agent system, including new action management, learning capabilities, and improved configuration handling.

Key Changes

New Modules

• Added actions/__init__.py with core Action class and behavior implementations
• Implemented Deep Q-Learning (DQN) movement module
• Added ExperimentRunner for simulation experiments
• Introduced BaseState and AgentState classes
• Added types.py for type definitions

Updates to Existing Components

• Updated base_agent.py with improved state handling and action selection
• Refactored IndependentAgent and SystemAgent classes
• Enhanced SimulationConfig with new parameters
• Updated configuration files (config.yaml)
• Modified Environment class with type hints and state management
• Updated GUI configuration defaults

Documentation

• Added comprehensive documentation for:
  • Multi-Agent Action System
  • DQN Implementation Guide
  • Experiment Running Guide
  • Move Module Documentation
  • State System Documentation

Configuration & Logging

• Added functionality to save configurations to timestamped YAML files
• Updated database path naming conventions
• Improved logging handling in SimulationGUI
• Added .csv and config_*.yaml to .gitignore

Technical Details

• Default simulation steps increased to 8000
• Implemented normalized state values [0,1]
• Added validation for position checking
• Enhanced experiment parameter handling
• Improved resource management and action selection

Testing

• Tested with multiple simulation iterations
• Verified experiment runner functionality
• Validated state normalization
• Confirmed configuration saving mechanism

Dependencies

• torch
• random
• collections.deque
• yaml

Migration Notes

Users should update their configuration files to match the new parameter structure and review the updated documentation for new features.