Hyperparameter

[ngoding]

Model Sequential A:

num_batch_size = 128

num_epochs = 250 Training Accuracy: 0.9536153078079224 Testing Accuracy: 0.89673912525177

num_ephocs = 200 Training Accuracy: 0.9699863791465759 Testing Accuracy: 0.907608687877655

num_epochs = 160 Training Accuracy: 0.9208731055259705 Testing Accuracy: 0.9021739363670349

num_batch_size = 5

num_ephocs = 250 Training Accuracy: 1.0 (overfitting) Testing Accuracy: 0.929347813129425

num_batch_size = 64

num_ephocs = 200 Training Accuracy: 0.9781718850135803 Testing Accuracy: 0.9239130616188049

num_ephocs = 160 Training Accuracy: 0.9754433631896973 Testing Accuracy: 0.9130434989929199

[ngoding]

Model Sequential A

[ngoding]

Model B (One layer):

Batch size 64

Epochs 160 Training Accuracy: 0.9754433631896973 Testing Accuracy: 0.9130434989929199

Epochs 200 Training Accuracy: 0.8540245294570923 Testing Accuracy: 0.8478260636329651

Batch size 128

Epochs 160 Training Accuracy: 0.8553888201713562 Testing Accuracy: 0.85326087474823

Model C (Two layers)

Batch size 64

Epochs 160 Training Accuracy: 0.8826739192008972 Testing Accuracy: 0.875

Epochs 250 Training Accuracy: 0.8908594846725464 Testing Accuracy: 0.864130437374115

[ngoding]

Model B (One Layer)

Model C (Two layers)

[ngoding]

Model D (Using 4 layers with Leaky ReLu)

Batch size: 64

200 Epochs Training Accuracy: 0.9563437700271606 Testing Accuracy: 0.9021739363670349

250 Epochs Training Accuracy: 0.9699863791465759 Testing Accuracy: 0.907608687877655

Batch size 128

200 Epochs Training Accuracy: 0.9754433631896973 Testing Accuracy: 0.91847825050354

[ngoding]

epochs 200 batch 64 Leaky ReLu: Training Accuracy: 0.9918144345283508 Training Loss: 0.03159037075738586 Testing Accuracy: 0.9239130616188049 Testing Loss: 0.5403694471587306

Without Leaky: Training Accuracy: 0.9618008136749268 Training Loss: 0.12657192920678959 Testing Accuracy: 0.907608687877655 Training Loss: 0.566461583842402

[ngoding]

Using 3 layers epochs 100 batch 64 Training Accuracy: 0.9113233089447021 Training Loss: 0.2922065547104584 Testing Accuracy: 0.885869562625885 Testing Loss: 0.36498807953751605

[ngoding]

3 Layers with Leaky ReLu epochs 100 batch 64 Training Accuracy: 0.909959077835083 Training Loss: 0.24070889859537797 Testing Accuracy: 0.907608687877655 Training Loss: 0.29709170175635297

[ngoding]

3 Layers with Leaky Relu epochs 100 batch 128 Training Accuracy: 0.9126875996589661 Training Loss: 0.19908286868956698 Testing Accuracy: 0.89673912525177 Training Loss: 0.3105538940947989

[ngoding]

3 Layers Leaky ReLu with 0.01 alpha epochs 100 batch 128 Training Accuracy: 0.8867667317390442 Training Loss: 0.3289074445649026 Testing Accuracy: 0.8804348111152649 Testing Loss: 0.3416684321735216

[ngoding]

3 Layers LeakyReLu filters (64- 128 - 128) epochs 100 batch 64 Training Accuracy: 0.9249659180641174 Training Loss: 0.19424964989679083 Testing Accuracy: 0.9130434989929199 Testing Loss: 0.32117354481116583

[ngoding]

3 Layers LeakyReLu with learning rate 0.01 epochs 100 batch 128 Training Accuracy: 0.8826739192008972 Training Loss: 0.3142617943342191 Testing Accuracy: 0.8586956262588501 Testing Loss: 0.39314839503039484

[ngoding]

3 Layers LeakyReLu filters( 64- 128 - 128) epochs 100 batch 128 Training Accuracy: 0.9631651043891907 Training Loss: 0.10386263761074774 Testing Accuracy: 0.9347826242446899 Testing Loss: 0.33088753728762915

[ngoding]

3 Layers LeakyReLu with learning rate 0.0001 epochs 100 batch 128 Training Accuracy: 0.7994542717933655 Training Loss: 0.8075553321090272 Testing Accuracy: 0.782608687877655 Testing Loss: 0.7896293220312699

[Rocksus]

So in summary the best model is the 3 Layers with LeakyReLu?

[ngoding]

Yes, 3 Layers LeakyReLu filters( 64- 128 - 128) is the best one currently

[ngoding]

3 Layers LeakyReLu filters( 64- 128 - 256) epochs 100 batch 128 Training Accuracy: 0.8867667317390442 Training Loss: 0.2652438335822029 Testing Accuracy: 0.89673912525177 Testing Loss: 0.32409622099088586