search

yandex-research / rtdl-num-embeddings

(NeurIPS 2022) On Embeddings for Numerical Features in Tabular Deep Learning
https://arxiv.org/abs/2203.05556
MIT License
269 stars 31 forks source link

About periodic config #14

Closed Minhvt34 closed 10 months ago

Minhvt34 commented 10 months ago

Hi Yura!

I am trying to implement transformer-plr. I cannot find config.model.periodic and config.model.autodis in report.json file or in default config file.

I found self.model.periodic = lib.PeriodicOptions( self.model.d_num_embedding // 2, self.model.periodic_sigma, False, 'log-linear', )

But I can not find the implement of autodis.

Thank you!

Yura52 commented 10 months ago

Hi! See this line in bin/train4.py.

To clarify, our paper does not propose AutoDis. AutoDis is a baseline from prior work.

Yura52 commented 10 months ago

I am not sure that I fully understand the question, do you need help with the "periodic" part?

Minhvt34 commented 10 months ago

Thank you for your reply!

I found the implement of periodic embedding here https://github.com/Yura52/rtdl/blob/main/rtdl/nn/_embeddings.py more practical. Is the implement of numerical embedding in rtdl/nn/embedding.py as same as in the current repo - tabular-dl-num-embeddings

Yura52 commented 10 months ago

Well, I hope they are the same :) Though, currently, I recommend copying the implementation from here: link

Regarding, the rtdl project, please, see this post to learn about the status of the project.

Minhvt34 commented 10 months ago

Well, I hope they are the same :) Though, currently, I recommend copying the implementation from here: link

Regarding, the rtdl project, please, see this post to learn about the status of the project.

Thank you!