chesslablab / scanblab

Chess piece image recognizer.
MIT License
5 stars 2 forks source link

Use more samples to train the multilayer neural network #6

Open programarivm opened 1 year ago

programarivm commented 1 year ago

The process of training the multilayer neural network may require some trial and error until the final result is achieved. At this moment the results are perhaps not too accurate recognizing knights.

{
    "breakdown": {
        "overall": {
            "accuracy": 0.9994216990880235,
            "balanced accuracy": 0.9969842248240661,
            "f1 score": 0.9941602196186778,
            "precision": 0.9940627491890405,
            "recall": 0.9942795094934874,
            "specificity": 0.9996889401546447,
            "negative predictive value": 0.9996881232795347,
            "false discovery rate": 0.005937250810959638,
            "miss rate": 0.005720490506512628,
            "fall out": 0.00031105984535529675,
            "false omission rate": 0.0003118767204653641,
            "mcc": 0.9938551321222888,
            "informedness": 0.9939684496481321,
            "markedness": 0.993750872468575,
            "true positives": 19924,
            "true negatives": 239088,
            "false positives": 75,
            "false negatives": 75,
            "cardinality": 19999
        },
        "classes": {
            "1": {
                "accuracy": 0.9987468043510953,
                "balanced accuracy": 0.9814868703344161,
                "f1 score": 0.962178517397882,
                "precision": 0.9607250755287009,
                "recall": 0.9636363636363636,
                "specificity": 0.9993373770324685,
                "negative predictive value": 0.9993883168518707,
                "false discovery rate": 0.0392749244712991,
                "miss rate": 0.036363636363636376,
                "fall out": 0.000662622967531501,
                "false omission rate": 0.0006116831481293161,
                "informedness": 0.9629737406688321,
                "markedness": 0.9601133923805716,
                "mcc": 0.9615425029227576,
                "true positives": 318,
                "true negatives": 19606,
                "false positives": 13,
                "false negatives": 12,
                "cardinality": 330,
                "proportion": 0.01650082504125206
            },
            "N": {
                "accuracy": 0.997546688028839,
                "balanced accuracy": 0.9941865674490369,
                "f1 score": 0.9850290253589978,
                "precision": 0.9799392097264438,
                "recall": 0.9901719901719902,
                "specificity": 0.9982011447260835,
                "negative predictive value": 0.9991270187690965,
                "false discovery rate": 0.020060790273556228,
                "miss rate": 0.009828009828009798,
                "fall out": 0.0017988552739165442,
                "false omission rate": 0.0008729812309035134,
                "informedness": 0.9883731348980738,
                "markedness": 0.9790662284955403,
                "mcc": 0.9837086751325164,
                "true positives": 1612,
                "true negatives": 18312,
                "false positives": 33,
                "false negatives": 16,
                "cardinality": 1628,
                "proportion": 0.08140407020351018
            },
            ...
            "r": {
                "accuracy": 1,
                "balanced accuracy": 1,
                "f1 score": 1,
                "precision": 1,
                "recall": 1,
                "specificity": 1,
                "negative predictive value": 1,
                "false discovery rate": 0,
                "miss rate": 0,
                "fall out": 0,
                "false omission rate": 0,
                "informedness": 1,
                "markedness": 1,
                "mcc": 1,
                "true positives": 1622,
                "true negatives": 18302,
                "false positives": 0,
                "false negatives": 0,
                "cardinality": 1622,
                "proportion": 0.08110405520276014
            }
        }
    },
    "matrix": {
        "1": {
            "1": 318,
            "N": 0,
            "B": 8,
            "Q": 0,
            "b": 0,
            "K": 0,
            "k": 0,
            "R": 0,
            "n": 0,
            "P": 5,
            "p": 0,
            "q": 0,
            "r": 0
        },
        "N": {
            "1": 12,
            "N": 1612,
            "B": 7,
            "Q": 0,
            "b": 0,
            "K": 0,
            "k": 0,
            "R": 0,
            "n": 0,
            "P": 14,
            "p": 0,
            "q": 0,
            "r": 0
        },
        ...
        "r": {
            "1": 0,
            "N": 0,
            "B": 0,
            "Q": 0,
            "b": 0,
            "K": 0,
            "k": 0,
            "R": 0,
            "n": 0,
            "P": 0,
            "p": 0,
            "q": 0,
            "r": 1622
        }
    }
}

It'd be convenient to use 50,000 samples to train the multilayer neural network rather than 20,000 and compare the results obtained with the current ones. If the results aren't significantly improved, then the sample data may require some tweaking.

This is a few step process as described in the README file.

  1. Remove all files in the testing and the training folders.
  2. Prepare 50,000 samples for further training.
  3. Train the neural network.
  4. Prepare 50,000 samples for further testing.
  5. Make predictions.

Keep it up, and happy learning and coding.

dev0614 commented 8 months ago

Wonderful.