ac-learn

ac-learn

Where the All Contributors machine can learn about your contributions.

Install

yarn add ac-learn --save
#or
npm i -D ac-learn

Usage

const Learner = require('ac-learn')

//If you want to load a learner from a JSON export:
const learner = Learner.fromJSON(require('./your-learner.json'))
//If you want to use the default one
const learner = new Learner()
//If you want to your own dataset or customise the learner, check https://github.com/all-contributors/ac-learn#learner

//Training
learner.train() //Or
learner.train(someTrainingSet)

//Testing and getting stats
const fullStats = learner.eval()

//Cross-validation
const {microAvg, macroAvg} = learner.crossValidate()

//Confusion matrix (as string or console table)
const textualTable = learner.confusionMatrix.toString()
const cmTable = learner.confusionMatrix.toTable()

//Classifying an input
const output = learner.classify(someInput)
//Getting an input from an output
const input = learner.backClassify(someOutput)

//Saving the model to a JSON file
const savedModel = learner.toJSON()
const {writeFileSync} = require('fs')
writeFileSync('your-learner.json', JSON.stringify(jsonData))

Documentation

Learner
ConfusionMatrix

Learner

NodeJS Classification-based learner.

Parameters

opts Object Options.
- opts.dataset Array<Object> Dataset (for training and testing) (optional, default require('./conv')('io'))
- opts.splits number Dataset split percentage for the training/validation set (default: 70%/15%/15%) (optional, default [.7,.15])
- opts.classifier function (): Object Classifier builder function (optional, default classifierBuilder)
- opts.pastTrainingSamples Array<Object> Past training samples for the classifier (optional, default [])
- opts.classes Array<string> List of classes (categories) (optional, default require('./categories'))

Examples

Using pre-defined data

const learner = new Learner()

Using a custom dataset

const learner = new Learner({
  dataset: [
    {input: 'something bad', output: 'bad'},
    {input: 'a good thing', output: 'good'},
  ],
})

Using a specified classifier function

const learner = new Learner({
  classifier: myClassifierBuilderFn, //see {@link module:./classifier} for an example (or checkout `limdu`'s examples)
})

Changing the train/test split percentage

const learner = new Learner({
  splits: [0.6, 0.2],
})

(Re-)Using past-training samples

const learner = new Learner({
  pastTrainingSamples: [
    {input: 'something bad', output: 'bad'},
    {input: 'a good thing', output: 'good'},
  ],
})

train

Parameters

trainSet Array<Object> Training set (optional, default this.trainSet)

eval

Parameters

log boolean Log events (optional, default false)

Returns Object Statistics from a confusion matrix

serializeClassifier

Returns string Serialized classifier

serializeAndSaveClassifier

Parameters

file string Filename (optional, default 'classifier.json')

Returns Promise<(string | Error)> Serialized classifier

deserializeClassifier

Parameters

serializedClassifier string .

Returns Object Deserialized classifier

loadAndDeserializeClassifier

Parameters

file string Filename (optional, default 'classifier.json')

Returns Promise<(string | Error)> Deserialized classifier

classify

Parameters

data {input: any, output: any} Data to classify

Returns Array<string> Classes

crossValidate

Parameters

numOfFolds number Cross-validation folds (optional, default 5)
verboseLevel number Verbosity level on limdu's explainations (optional, default 0)
log boolean Cross-validation logging (optional, default false)

Returns {microAvg: Object, macroAvg: Object} Averages

backClassify

Parameters

category string Category name.

Returns Array<string> Labels associated with category

toJSON

JSON representation of the learner with the serialized classification model.

Returns Object JSON representation

fromJSON

Parameters

json (JSON | Object) JSON form

Returns Learner Generated learner from json

getCategoryPartition

Get the observational overall/train/validation/test count for each classes in the associated dataset.

Parameters

log boolean Log events (optional, default false)
outputFile string Filename for the output (to be used by chart.html) (optional, default '')

Returns Object<string, {overall: number, test: number, validation: number, train: number}> Partitions

getStats

Parameters

log boolean Log events (optional, default false)
categoryPartitionOutput string Filename for the output of the category partitions. (optional, default '')

Returns Object Statistics

ConfusionMatrix

Multi-class focused confusion matrix.

addEntry

Parameters

actual string Actual class
predicted string Predicted class

Returns number Updated entry

setEntry

Parameters

actual string Actual class
predicted string Predicted class
val number New entry

getEntry

Parameters

actual string Actual class
predicted string Predicted class

Returns number Entry

getTotal

Get the total count of all entries.

Returns number Total count

getTP

Number of elements in the category class correctly predicted.

Parameters

category string Class/category considered as positive

Returns number True Positives

getFP

Number of elements that aren't in the category class but predicted as such.

Parameters

category string Class/category considered as positive

Returns number False Positives

getFN

Number of elements in the category class but predicted as not being in it.

Parameters

category string Class/category considered as positive

Returns number False Negatives

getTN

Number of elements that aren't in the category class correctly predicted.

Parameters

category string Class/category considered as positive

Returns number True Negatives

getDiagonal

Diagonal of truth (top-left → bottom-right)

Returns Array<number> Numbers in the diagonal

getTrue

Number of correct (truthful) predictions.

Returns number TP

getFalse

Number of incorrect predictions.

Returns number FP + FN

getPositive

Number of real (actual) "positive" elements (i.e. elements that belong to the category class).

Parameters

category string Class/category considered as positive

Returns number TP + FN

getNegative

Number of real (actual) "negative" elements (i.e. elements that don't belong to the category class).

Parameters

category string Class/category considered as positive

Returns number TN + FP

getPredPositive

Number of predicted "positive" elements (i.e. elements guessed as belonging to the category class).

Parameters

category string Class/category considered as positive

Returns number TP + FN

getPredNegative

Number of predicted "negative" elements (i.e. elements guessed as not belonging to the category class).

Parameters

category string Class/category considered as positive

Returns number TN + FP

getSupport

Support value (count/occurrences) of category in the matrix

Parameters

category string Class/category to look at

Returns number Support value

getAccuracy

Prediction accuracy for category.

Parameters

category string Class/category considered as positive

Returns number (TP + TN) / (TP + TN + FP + FN)

getMicroAccuracy

Micro-average of accuracy.

Returns number (TP0 + ... + TPn + TN0 + ... + TNn) / (TP0 + ... + TPn + TN0 + ... + TNn + FP0 + ... + FPn + FN0 + ... + FNn)

getMacroAccuracy

Macro-average of accuracy.

Returns number (A0 + ...+ An_1) / n

getWeightedAccuracy

Weighted accuracy.

Returns number (A0 s0 + ... + An sn) / Total

getTotalPositiveRate

Predicition recall.

Parameters

category string Class/category considered as positive

Returns number TP / (TP + FN)

getMicroRecall

Micro-average of recall.

Returns number (TP0 + ... + TPn) / (TP0 + ... + TPn + FN0 + ... + FNn)

getMacroRecall

Macro-average of recall.

Returns number (R0 + R1 + ... + Rn-1) / n

getWeightedRecall

Weighted recalll.

Returns number (R0 s0 + ... + Rn sn) / Total

getPositivePredictiveValue

Prediction precision for category.

Parameters

category string Class/category considered as positive

Returns number TP / (TP + FP)

getPositivePredictiveValue

Prediction F1 score for category.

Parameters

category string Class/category considered as positive

Returns number 2 (Pr R) / (Pr + R)

getMicroPrecision

Micro-average of the precision.

Returns number (TP0 + ... + TPn) / (TP0 + ... + TPn + FP0 + ... FPn)

getMacroPrecision

Macro-average of the precsion.

Returns number (Pr0 + Pr1 + ... + Pr_n-1) / n

getWeightedPrecision

Weighted precision.

Returns number (Pr0 s0 + ... + Prn sn) / Total

getMicroF1

Micro-average of the F1 score.

Returns number 2 (TP0 + ... + TPn) / (2 (TP0 + ... + TPn) + (FN0 + ... + FNn) + (FP0 + ... + FPn))

getMacroF1

Macro-average of the F1 score.

Returns number (F0_1 + F1_1 + ... + F_n-1_1) / n

getWeightedF1

Weighted F1.

Returns number (F01 s0 + ... + Fn1 sn) / Total

getFalseNegativeRate

Miss rates on predictions for category.

Parameters

category string Class/category considered as positive

Returns number FN / (TP + FN)

getMicroMissRate

Micro-average of the miss rate.

Returns number (FN0 + ... + FNn) / (TP0 + ... + TPn + FN0 + ... FNn)

getMacroMissRate

Macro-average of the miss rate.

Returns number (M0 + M1 + ... + Mn) / n

getWeightedMissRate

Weighted miss rate.

Returns number (M0 s0 + ... + Mn sn) / Total

getFalsePositiveRate

Fall out (false alarm) on predictions for category.

Parameters

category string Class/category considered as positive

Returns number FP / (FP + TN)

getMicroFallOut

Micro-average of the fall out.

Returns number (FP0 + ... + FPn) / (FP0 + ... + FPn + TN0 + ... TNn)

getMacroFallOut

Macro-average of the fall out.

Returns number (Fo0 + Fo1 + ... + Fo_n) / n

getWeightedFallOut

Weighted fall out.

Returns number (Fo0 s0 + ... + Fon sn) / Total

getTrueNegativeRate

Specificity on predictions for category.

Parameters

category string Class/category considered as positive

Returns number TN / (FP + TN)

getMicroSpecificity

Micro-average of the specificity.

Returns number (TN0 + ... + TNn) / (FP0 + ... + FPn + TN0 + ... TNn)

getMacroSpecificity

Macro-average of the specificity.

Returns number (S0 + S1 + ... + Sn) / n

getWeightedSpecificity

Weighted specificity.

Returns number (S0 s0 + ... + Sn sn) / Total

getPrevalence

Prevalence on predictions for category.

Parameters

category string Class/category considered as positive

Returns number (TP + FN) / (TP + TN + FP + FN)

getMicroPrevalence

Micro-average of the prevalence.

Returns number (TP0 + ... + TPn + FN0 + ... + FNn) / (TP0 + ... + TPn + TN0 + ... + TNn + FP0 + ... + FPn + FN0 + ... + FNn)

getMacroPrevalence

Macro-average of the prevalence.

Returns number (Pe0 + Pe1 + ... + Pen) / n

getWeightedPrevalence

Weighted prevalence.

Returns number (Pe0 s0 + ... + Pen sn) / Total

toString

Textual tabular representation of the confusion matrix.

Parameters

opt Object Options (optional, default {})
- opt.split boolean Split the classes in half (→ 2 matrices) (optional, default false)
- opt.clean boolean Remove empty column/row pairs (optional, default false)
- opt.colours boolean Colourize cells (optional, default true)
- opt.maxValue (optional, default 100)

Examples

Example output (cf. /src/tests/confusionMatrix.js)

Actual \ Predicted bug code other

bug 5.00 0.00 1.00 code 1.00 2.00 0.00 other 0.00 3.00 8.00



Returns
**[string](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/String)**
String representation

#### toTable

`console.table` version of `confusionMatrix.toString()`.

##### Parameters

- `opt`
  **[Object](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Object)**
  Options (optional, default `{}`)
  - `opt.split`
    **[boolean](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Boolean)**
    Split the classes in half (→ 2 matrices) (optional, default `false`)
  - `opt.clean`
    **[boolean](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Boolean)**
    Remove empty column/row pairs (optional, default `false`)
  - `opt.colours`
    **[boolean](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Boolean)**
    Colourize cells (optional, default `true`)
  - `opt.maxValue` (optional, default `100`)

#### getShortStats

##### Parameters

- `type`
  **[string](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/String)**
  Type of stats (`micro`/`macro`/`weighted` average) (optional, default
  `'micro'`)

Returns
**[string](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/String)**
Short statistics (total, true, false, accuracy, precision, recall and f1)

#### getStats

Returns **{total:
[number](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Number),
correctPredictions:
[number](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Number),
incorrectPredictions:
[number](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Number),
classes:
[Array](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Array)&lt;[string](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/String)>,
microAvg:
[Object](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Object),
macroAvg:
[Object](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Object),
results:
[Object](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Object)}**
(Long) statistics

#### fromData

Creates a confusion matrix from the `actual` and `predictions` classes.

##### Parameters

- `actual`
  **[Array](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Array)&lt;[string](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/String)>**
  Actual classes
- `predictions`
  **[Array](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Array)&lt;[string](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/String)>**
  Predicted classes
- `classes`
  **[Array](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Array)&lt;[string](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/String)>**
  Classes/categories to use (optional, default `[]`)

Returns **[ConfusionMatrix](#confusionmatrix)** Filled confusion matrix

## Contributors

<!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section -->
<!-- prettier-ignore-start -->
<!-- markdownlint-disable -->
<table>
  <tbody>
    <tr>
      <td align="center" valign="top" width="14.28%"><a href="http://maxcubing.wordpress.com"><img src="https://avatars0.githubusercontent.com/u/8260834?v=4?s=100" width="100px;" alt="Maximilian Berkmann"/><br /><sub><b>Maximilian Berkmann</b></sub></a><br /><a href="https://github.com/all-contributors/ac-learn/commits?author=Berkmann18" title="Code">💻</a> <a href="https://github.com/all-contributors/ac-learn/commits?author=Berkmann18" title="Documentation">📖</a> <a href="#ideas-Berkmann18" title="Ideas, Planning, & Feedback">🤔</a> <a href="#maintenance-Berkmann18" title="Maintenance">🚧</a> <a href="#platform-Berkmann18" title="Packaging/porting to new platform">📦</a> <a href="#infra-Berkmann18" title="Infrastructure (Hosting, Build-Tools, etc)">🚇</a> <a href="https://github.com/all-contributors/ac-learn/commits?author=Berkmann18" title="Tests">⚠️</a> <a href="#security-Berkmann18" title="Security">🛡️</a></td>
      <td align="center" valign="top" width="14.28%"><a href="https://tenshiamd.com"><img src="https://avatars.githubusercontent.com/u/13580338?v=4?s=100" width="100px;" alt="Angel Aviel Domaoan"/><br /><sub><b>Angel Aviel Domaoan</b></sub></a><br /><a href="https://github.com/all-contributors/ac-learn/commits?author=tenshiAMD" title="Code">💻</a> <a href="#maintenance-tenshiAMD" title="Maintenance">🚧</a> <a href="https://github.com/all-contributors/ac-learn/pulls?q=is%3Apr+reviewed-by%3AtenshiAMD" title="Reviewed Pull Requests">👀</a></td>
      <td align="center" valign="top" width="14.28%"><a href="https://github.com/features/security"><img src="https://avatars.githubusercontent.com/u/27347476?v=4?s=100" width="100px;" alt="Dependabot"/><br /><sub><b>Dependabot</b></sub></a><br /><a href="#security-dependabot" title="Security">🛡️</a></td>
      <td align="center" valign="top" width="14.28%"><a href="https://dev.to/gr2m"><img src="https://avatars.githubusercontent.com/u/39992?v=4?s=100" width="100px;" alt="Gregor Martynus"/><br /><sub><b>Gregor Martynus</b></sub></a><br /><a href="https://github.com/all-contributors/ac-learn/pulls?q=is%3Apr+reviewed-by%3Agr2m" title="Reviewed Pull Requests">👀</a></td>
    </tr>
  </tbody>
</table>

<!-- markdownlint-restore -->
<!-- prettier-ignore-end -->

<!-- ALL-CONTRIBUTORS-LIST:END -->

This project follows the
[all-contributors](https://github.com/all-contributors/all-contributors)
specification. Contributions of any kind welcome!

all-contributors / ac-learn

readme

ac-learn

Install

Usage

Documentation

Table of Contents

Learner

Parameters

Examples

train

Parameters

eval

Parameters

serializeClassifier

serializeAndSaveClassifier

Parameters

deserializeClassifier

Parameters

loadAndDeserializeClassifier

Parameters

classify

Parameters

crossValidate

Parameters

backClassify

Parameters

toJSON

fromJSON

Parameters

getCategoryPartition

Parameters

getStats

Parameters

ConfusionMatrix

addEntry

Parameters

setEntry

Parameters

getEntry

Parameters

getTotal

getTP

Parameters

getFP

Parameters

getFN

Parameters

getTN

Parameters

getDiagonal

getTrue

getFalse

getPositive

Parameters

getNegative

Parameters

getPredPositive

Parameters

getPredNegative

Parameters

getSupport

Parameters

getAccuracy

Parameters

getMicroAccuracy

getMacroAccuracy

getWeightedAccuracy

getTotalPositiveRate

Parameters

getMicroRecall

getMacroRecall

getWeightedRecall

getPositivePredictiveValue

Parameters

getPositivePredictiveValue

Parameters

getMicroPrecision

getMacroPrecision

getWeightedPrecision