[dattri.benchmark] Import Issue for Complex Models

TRAIS-Lab / dattri

`dattri` is a PyTorch library for developing, benchmarking, and deploying efficient data attribution algorithms.

https://trais-lab.github.io/dattri/

MIT License

28 stars 8 forks source link

[dattri.benchmark] Import Issue for Complex Models #52

Closed tingwl0122 closed 4 months ago

tingwl0122 commented 4 months ago

For one of the experiment in dattri/benchmark, we want to train MusicTransformer on MAESTRO dataset for benchmarking. However, compared to other benchmark experiments, the model itself is too complex to be directly written in dattri/benchmark/maestro.py.

@TheaperDeng mentioned that we can either directly import it into our repo or do a submodule.

[Update: Currently, nanoGPT might also be another complex model to follow the conversion.]

tingwl0122 commented 4 months ago

Update: After the previous discussion, we will follow the convention:

We will have different folders to handle different datasets. (Ex: dattri/benchmark/maestro/...) The files in this folder should include model training, model evaluation and dataset handling.
We will have a folder to contain all the complex models we are going to consider (Ex: dattri/benchmark/models/MusicTransformer/...)

tingwl0122 commented 4 months ago

Note: Ruff checking will skipped at dattri/benchmark/models/.... This has been done in #53.

tingwl0122 commented 4 months ago

Solved by merging #53 and #56 . Will force all dataset/models added follow the same structure.