search

YugeTen / adios

63 stars 6 forks source link

Classifier Loss during training [edited] #4

Open AlejandroTL opened 2 years ago

AlejandroTL commented 2 years ago

Hello,

I have installed a new environment using the requirements.txt but I receive the following error:

Traceback (most recent call last): File "/adios/main_pretrain.py", line 5, in <module> from pytorch_lightning import Trainer, seed_everything File "/miniconda3/envs/adios/lib/python3.9/site-packages/pytorch_lightning/__init__.py", line 20, in <module> from pytorch_lightning import metrics # noqa: E402 File "/miniconda3/envs/adios/lib/python3.9/site-packages/pytorch_lightning/metrics/__init__.py", line 15, in <module> from pytorch_lightning.metrics.classification import ( # noqa: F401 File "/miniconda3/envs/adios/lib/python3.9/site-packages/pytorch_lightning/metrics/classification/__init__.py", line 14, in <module> from pytorch_lightning.metrics.classification.accuracy import Accuracy # noqa: F401 File "/miniconda3/envs/adios/lib/python3.9/site-packages/pytorch_lightning/metrics/classification/accuracy.py", line 18, in <module> from pytorch_lightning.metrics.utils import deprecated_metrics, void File "/miniconda3/envs/adios/lib/python3.9/site-packages/pytorch_lightning/metrics/utils.py", line 22, in <module> from torchmetrics.utilities.data import get_num_classes as _get_num_classes ImportError: cannot import name 'get_num_classes' from 'torchmetrics.utilities.data' (/miniconda3/envs/adios/lib/python3.9/site-packages/torchmetrics/utilities/data.py)

Any idea on what could be? If I install torchmetrics==0.6.0, some other different error appear.

Thanks!

-------- EDIT ------

The problem was solved just installing a different version of some libraries to make them compatible with torchmetrics==0.6.0.

Therefore, I use the existing open issue to ask something: why do you also use the loss of a classifier to optimise the encoder? If I read the paper, it's not clear to me the existence of that classifier in the loss function, but in the code it appears in all the models. How are the results if you don't add the classifier and the linear loss during training?

Thanks!

nax1016cs commented 1 year ago

Hi, I have same confusion, did you figure out why classifier and cross-entropy loss are used in pre-training?

YugeTen commented 1 year ago

Hi both, the classifier loss is not used to train the encoder. You will see here that the output of the encoder, feats, is detached before passing to the classifier. The classifier is only there for evaluation purposes and is not involved in training. Hope this helps.