yizhibaiwuya
commented
4 months ago Hi there, Thank you so much for your interest in LibContinual. We will try to address the issues you raised.
Similarities and Differences between LibContinual and PyCIL
Both LibContinual and PyCIL are an algorithm library, which aims to provide abundant algorithms for the field of continual learning, especially for the class-incremental learning scenarios. Therefore, in this sense, there are inevitably similarities in the top-level design, such as abstracting each method into a separate class and abstracting data, algorithms, and networks into separate modules. Despite these inevitable similarities, there are many differences between LibContinual and PyCIL.
Firstly, in our Libcontinual, we aim to extract more algorithm-independent components so that each file under core/model/ only needs to focus on a few functions that highlight the characteristics of the algorithm itself. To achieve this, we use a Trainer class to handle common operations, such as data preparation, optimizer creation, logging, and file saving. In contrast, PyCIL implements operations like optimizer creation, data iteration, and logging within each method. This design philosophy of LibContinual is consistent with our previous work, LibFewShot which is an algorithm library focused on few-shot learning.
Regarding the Similar Codes
As you pointed out, some codes in LibContinual are similar to PyCIL, we sincerely apologize for that. PyCIL is an excellent framework, and during the integration of algorithms in Libcontinual, we referred to many implementations available online and selected to referred to the most suitable ones. In this process, due to our oversight, some contributors may directly used some codes from PyCIL, which is indeed unacceptable. However, please allow us to explain the specific segments you mentioned:
weight_align: To our knowledge, the weight alignment operation originally comes from the paper WA. The core contribution of this algorithm is the weight alignment operation. As you can see, the implementation of this operation is very concise and has been reused by many algorithms to become a common operation in the field of continual learning, such as DER.compute_ewc: The EWC loss was proposed by the paper EWC. Its core operations involve the computation of the Fisher matrix and the EWC loss. When implementing this method, we referred to several versions of the implementation, which differ mostly in their variable names. For example: FACIL.bic: The BiC method was proposed by the paper BIC. This method inherently involves a two-stage training process, and hence, its implementation inevitably requires splitting it into two separate functions. Aside from the function names, we did not observe other similarities between the two implementations.
We must acknowledge that during the development of LibContinual, there were instances of identical code to PyCIL due to contributors' oversight, which is an unacceptable mistake. To address this, we will add comments to all sections where reference code has been used, indicating the original source of the referenced code. Additionally, we will review and reimplement the existing code as necessary.
Regarding the Issue of Reinventing the Wheel
For algorithm frameworks, which provide a library capable of running various methods, it's difficult to definitively state the necessity of each product. From a certain perspective, they can indeed replace each other. However, does that mean only the first framework developed is necessary? If that were the case, we wouldn't have so many tools to choose from :)
As mentioned earlier, PyCIL is an excellent framework. However, LibContinual still offers many features that PyCIL does not support. For example, LibContinual supports methods based on pretrained models and prompt-based approaches. While LibContinual is not perfect and has many features currently in development, we hope it will become a widely accepted tool in the future.
Lastly, we once again thank you for your attention and criticism of LibContinual. This will help us continually improve LibContinual! We will promptly add acknowledgments for the referenced code and frameworks.
Hi there,
I'm new to the field of continual learning and have been exploring various toolkits to deepen my understanding. Recently, I came across your LibContinual repository and noticed many similarities with another toolkit I'm familiar with, PyCIL (https://github.com/G-U-N/PyCIL).
While going through both repositories, I couldn't help but notice that significant portions of the code in LibContinual appear to be very similar, if not identical, to those in PyCIL. Below are some examples with screenshots for reference:
(Left: PyCIL, Right: LibContinual)
https://github.com/G-U-N/PyCIL/blob/a9b1ec8e3dbbe143cb72b008d7c8f946629d55b1/utils/inc_net.py#L138
https://github.com/RL-VIG/LibContinual/blob/5976ff2811293a0882df32f851616ce8378bb593/core/model/der.py#L181
(Left: PyCIL, Right: LibContinual)
https://github.com/G-U-N/PyCIL/blob/a9b1ec8e3dbbe143cb72b008d7c8f946629d55b1/models/ewc.py#L210
https://github.com/RL-VIG/LibContinual/blob/5976ff2811293a0882df32f851616ce8378bb593/core/model/ewc.py#L128
(It seems the indentation is directly copied from PyCIL LOL)
(Left: PyCIL, Right: LibContinual)
It seems like renaming the functions into a similar one...
These similarities raise some important questions:
Is it necessary to reinvent the wheel by creating a new repository that largely replicates the functionality and code of PyCIL? Shouldn't there be proper acknowledgment and attribution to PyCIL, considering the significant overlap in code? This includes but is not limited to:
Looking forward to your thoughts on this matter.