bug in eval_task()

[PixelChen24]

Hello, I've found some bugs about your code computing the accuracy after each task. After training on the new task, the learner is expected to evaluate on all seen data, and in your code it is done by calling self.eval_task(): in base.py line 84:

def eval_task(self, save_conf=False):
        y_pred, y_true = self._eval_cnn(self.test_loader)
        cnn_accy = self._evaluate(y_pred, y_true)
...

When taking a close look at the _evaluate()function:

def _evaluate(self, y_pred, y_true):
        ret = {}
        grouped = accuracy(y_pred.T[0], y_true, self._known_classes)
...

where the function accuracy() is defined as:

def accuracy(y_pred, y_true, nb_old, increment=10):
    assert len(y_pred) == len(y_true), "Data length error."
    all_acc = {}
    all_acc["total"] = np.around(
        (y_pred == y_true).sum() * 100 / len(y_true), decimals=2
    )

    # Grouped accuracy
    for class_id in range(0, np.max(y_true), increment):

Here increment is used to pick indexs of those old tasks, but in your code you don't pass the param "increment" to the accuracy() function, which means even I customize the increment in config file, the code fails to evaluate on my pattern. I think you should pass self.args["increment"] when calling the accuracy() function.

[PixelChen24]

What's more, you should consider the init_increment part, which may not be equal to increment.

[zhoudw-zdw]

Actually, we do not care about group accuracy in CIL. If group accuracy matters in your application, you can modify it according to your description.