Future Experiments for Report

[anacarsi]

Experiments to Justify Neural Network Usage

We will try to justify the hyperparameters and architecture used based on different experiments.

Ana -> Performance and Training experiments (state representation, reward shaping, exploration strategies, time metrics)

Experiment	Key	Purpose
Baseline Comparison	Compare NN with non-NN strategies (e.g., rule-based, tabular Q-learning)	show improvements over simpler or heuristic methods
Ablation Study	remove/alter neural network components (e.g., layers, inputs)	find the most important features of the neural network and how changes affect performance
Hyperparameter Tuning	vary learning rate, hidden layers, neurons, etc.	find the optimal hyperparameters and justify the neural network's configuration
Generalization Test (Robustness)	train in one environment, test in different/altered environments	demonstrate that the neural network can generalize to unseen scenarios and is robust to environmental changes
Training time vs. Performance	track performance over time and compare with simpler methods	show the efficiency of the neural network in terms of how quickly it converges compared to other approaches
Exploration vs. Exploitation	test different exploration strategies (fixed/decaying epsilon)	impact of exploration strategies on learning and performance of the agent
Reward Shaping	modify reward structure for different objectives	show how different rewards affect the agent's behavior, e.g. focus on survival or objectives
State Representation Analysis	use different state representations (e.g., raw pixels vs. features)	validate the feature representation and show how different state inputs affect performance and learning speed
Policy Visualization	visualize agent decisions, action distributions	try to show that the neural network is making intelligent decisions

[anacarsi]

We successfullly developed the experiments and took insights from them. They are published in our final report.