search

arthurdouillard / incremental_learning.pytorch

A collection of incremental learning paper implementations including PODNet (ECCV20) and Ghost (CVPR-W21).
MIT License
383 stars 60 forks source link

E2E performance #8

Closed rumusan closed 5 years ago

rumusan commented 5 years ago

Thanks for your great repetition. Could you provide some configuration of E2E to get good performance?

Besides, is there much difference between single fc and multi fc?

Thank you again!

rumusan commented 5 years ago

I can not run E2E with this code. In e2e.py: " for i, ((_, idxes), inputs, targets) in enumerate(train_loader, start=1): " it seems the "train_loader" only have "inputs" and "targets".

I'd appreciate it if you could help me.

arthurdouillard commented 5 years ago

Indeed the code for e2e was not adapted since I changed data.py. I don't store anymore indexes (to log previous predictions) as it was prone to bugs, but simply call the old model in ram.

Look at iCaRL for pointers: https://github.com/arthurdouillard/incremental_learning.pytorch/blob/master/inclearn/models/icarl.py#L135

My local git is more advanced that this repo, but I won't release right now. It needs cleaning, and I am doing some experiments that are not ready to be shared. Sorry.

Anyway I was not able to reach paper performance with E2E. Their data aug scheme + their regularization is quite tricky. Their gradient noise destroys my performances for examples.

I would advise you to look for easier & better paper. See Large Scale Incremental Learning from this year CVPR for example.

rumusan commented 5 years ago

Thank you so much.

rumusan commented 5 years ago

Thank you for your advice.

In Figure 4 of "Large Scale Incremental Learning", "Classifier without bias removal" gets [85%, 72%, 59%, 51%, 40%]. However, I can only get [86%, 62%, 47%, 35%, 24%] in my experiments. Implementation Details are the same as the section 6.2. in the paper, except I implemented it with pytorch.

"Bias Correction Layer" does bring great improvement. However, I still can not get the performance presented in the paper.

I think there may be some important points that I have not noticed.

Could you reproduce the results of the paper? Could you give me some help?

Many thanks!

hardik2396 commented 5 years ago

@rumusan can you provide an implementation of large scale incremental learning?

Thanks in advanced.