training config file

[morgsmss7]

Could you provide a sample 0.toml file? I am attempting to run the training scripts on new data and am hitting a few issues I did not hit in tune.py.

[Yura52]

Do I understand correctly that you are trying to directly use bin/train4.py without bin/tune.py and you are looking for some kind of "default" hyperparameters? In particular, there are many algorithms in this repository (and in the paper), which one exactly you are trying to run?

[morgsmss7]

Hello Yura,

A default train4.py toml file would be very helpful. Right now, I am using mlp and mlp-plrlr, so an example for those would be quite useful. We are essentially looking to repeat a few of your experiments on our datasets.

Best, Morgan

On May 16, 2023, at 6:10 AM, Yura Gorishniy @.***> wrote:

Do I understand correctly that you are trying to directly use bin/train4.py without bin/tune.py and you are looking for some kind of "default" hyperparameters? In particular, there are many algorithms in this repository (and in the paper), which one exactly you are trying to run?

— Reply to this email directly, view it on GitHub https://github.com/Yura52/tabular-dl-num-embeddings/issues/12#issuecomment-1549379732, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJQFE4TYIG7YC37CG75SR2TXGNHAHANCNFSM6AAAAAAYCWN74A. You are receiving this because you authored the thread.

[Yura52]

I see. Right now, I don't have an opportunity to build a fully working example, but you can notify me on May 18 if the help will still be needed.

Some quick comments:

• You can take inspiration from the existing 0.toml files, they are mostly located according to the following pattern: exp/<algorithm>/<dataset>/0_evaluation/0.toml. For example: exp/mlp-plr/california/0_evaluation/0.toml
• However, such files are suitable only for specific scripts in bin, and you should check the corresponding *_tuning.toml file located in the parent directory to learn what script they target. For the above example, it is exp/mlp-plr/california/0_tuning.toml. Specifically, you can see there that this file works with bin/train3.py.
• While formats for bin/train{1,2,3}.py and bin/train4.py may be different, it should be still possible to manually adjust a config for bin/train3.py to make it suitable for bin/train4.py
• There are no "default" hyperparameters, unfortunately. You can try averaging the tuned hyperparameters across all datasets for a given algorithm. While it may not be optimal for your dataset, it can serve as a starting point. Importantly, you should not average the "sigma" hyperparameter for MLP-PLRLR and similar algorithms, since it is a very special hyperparameter. The safe strategy for "sigma" is to set smaller values. Higher values may provide benefits on some dataset, but can completely break the performance on other datasets.
• I would still consider tuning algorithms with bin/tune.py if there is such an opportunity

P.S. If MLP-PLRLR will be too slow, you can try MLP-PLR

[Yura52]

Feel free to reopen the issue if needed!