thuml / LogME

Code release for "LogME: Practical Assessment of Pre-trained Models for Transfer Learning" (ICML 2021) and Ranking and Tuning Pre-trained Models: A New Paradigm for Exploiting Model Hubs (JMLR 2022)
MIT License
203 stars 18 forks source link

How to apply LogME on the multi-label classification tasks? #9

Closed Muyun99 closed 2 years ago

Muyun99 commented 2 years ago

Hi, thanks your excellent work and nice code style.

I notice that LogME can be applied on multi-label dataset as the paper says. But the y is not the one-hot format, I want to know how to apply it on multi-label classification tasks.

Because the label is not one-hot format, so one possible solution is to construct every feature-label pairs for each sample. Such as:

The original label is [0, 1, 1, 0, 1] as for the feature f

f: [0,1,1,0,1]

Now i need to constrcut three feature-label pairs:

f : 1
f : 2
f : 4

So I want to know if it is the right way to use it on the multi-label setting. If this is not correct, can you give me an example.

Looking forward to your reply. Thanks!

Best Regards! Yun

youkaichao commented 2 years ago

You can feed the original label directly as y, and use the following code:

logme = LogME(regression=True)
# f has shape of [N, D], y has shape [N, C] being the multi-label vector.
score = logme.fit(f, y)

multi-label classification is treated as label regression directly.

Yun Du @.***> 于2022年5月16日周一 13:12写道:

Hi, thanks your excellent work and nice code style.

I notice that LogME can be applied on multi-label dataset as the paper says. But the y is not the one-hot format, I want to know how to apply it on multi-label classification tasks.

Because the label is not one-hot format, so one possible solution is to construct every feature-label pairs for each sample. Such as:

The original label is [0, 1, 1, 0, 1] as for the feature f

f: [0,1,1,0,1]

Now i need to constrcut three feature-label pairs:

f : 1 f : 2 f : 4

So I want to know if it is the right way to use it on the multi-label setting. If this is not correct, can you give me an example.

Looking forward to your reply. Thanks!

Best Regards! Yun

— Reply to this email directly, view it on GitHub https://github.com/thuml/LogME/issues/9, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFRJAHUB4XHAY6UWGPNUN3DVKHKKNANCNFSM5WAIPVZA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

-- All my best wishes!

KaiChao YOU School of Software Tsinghua University Beijing 100084 People's Republic of China E-mail: @. / @. Homepage: https://youkaichao.github.io

Muyun99 commented 2 years ago

I get it, thanks for your reply.

I update the readme to add the usage for multi-label classification and regression tasks, you can find it in this PR.

I hope it will helps.

Best Regards! Yun

youkaichao commented 2 years ago

looks good, thanks!