Imbalance data in DRED room level occupancy

[bontorhumala]

• Only one room is used at a time (positive vs negative = 1 vs 4)
• Livingroom is mostly used (64% of all room occupancies)

What metrics to use? How to correct imbalance?

[bontorhumala]

Accuracy is low (40-60%) due to class imbalance (1 vs 4 because only 1 room is occupied at a time).

Reference in choosing accuracy matrics:

• http://stats.stackexchange.com/questions/12702/what-are-the-measure-for-accuracy-of-multilabel-data
• scikit-learn/scikit-learn#4287
• http://scikit-learn.org/stable/modules/generated/sklearn.metrics.precision_recall_fscore_support.html

Reference in over/undersampling:

• http://contrib.scikit-learn.org/imbalanced-learn/auto_examples/index.html

[bontorhumala]

imbalanced-learn module supports binary or multiclass, but not multilabel

[bontorhumala]

See http://stackoverflow.com/questions/20461342/dealing-with-class-imbalance-in-multi-label-classification

[bontorhumala]

Turn it to a multiple binary classification problem

• Create different models for each room
• Resample training set accordingly to balance positive and negative sample
• Perform prediction separately

[bontorhumala]

Closed, the problem more likely related to quality of disaggregation