Extended plotting of input/output statistics to strongermax and polymax, used similarly to consmax by specifying the stats to be visualized. Command line would look like:
python3 train.py --out_dir=out --device=cpu --eval_interval=2 --log_interval=1 --block_size=2 --batch_size=5 --n_layer=3 --n_head=4 --n_embd=16 --lr_decay_iters=2 --dtype="float32" --max_iter=20 --softmax_variant_attn="polymax" --statistic=output_mean
Extended plotting of input/output statistics to strongermax and polymax, used similarly to consmax by specifying the stats to be visualized. Command line would look like: python3 train.py --out_dir=out --device=cpu --eval_interval=2 --log_interval=1 --block_size=2 --batch_size=5 --n_layer=3 --n_head=4 --n_embd=16 --lr_decay_iters=2 --dtype="float32" --max_iter=20 --softmax_variant_attn="polymax" --statistic=output_mean