Experimental details - Githubissues

tensorflow / neural-structured-learning

Training neural models with structured signals.

https://www.tensorflow.org/neural_structured_learning

Apache License 2.0

980 stars 189 forks source link

Experimental details #83

Closed Demirrr closed 3 years ago

Demirrr commented 3 years ago

Hello,

Thank you for the great work and making the implementation publicly available [1]. I do enjoy reading findings of your work. I was wondering whether you could share your opionon on the following matters:

1- Is there any plan to share pretrained models ?

2- Could you elaborate on runtimes ?

3- What are the ranges for hyperparameters of models ?

4- How many numbers of unique hyperparameter configurations are tested ?

I fail to find enough information to repeat experiments as Section A.3 reports only the number of epochs.

Cheers

[1] Low-Dimensional Hyperbolic Knowledge Graph Embeddings https://arxiv.org/pdf/2005.00545.pdf

DualityGap commented 3 years ago

@chamii22 @ines-chami for more experiment-related details and information : )

Demirrr commented 3 years ago

Cheers @DualityGap :)

ines-chami commented 3 years ago

Hi @Demirrr 1) Not at the moment but the experiments should be reproducible with the code. 2) I do not have exact runtimes but it training was taking a few hours. YAGO took around a day to train.
3) & 4) We mostly searched over the optimizer, negative sampling + learning rate. The different combinations were:

Adam + negative sampling + lr in [5e-4, 1e-3, 5e-3]
Adagrad + no negative samples + lr [5e-2, 1e-1] Batch size was searched over [100, 250, 500, 1000, 2000, 4000]. Note: we did not try all possible combinations and only ran experiments for hyper-parameter configurations that seemed promising using the validation set. All the best, Ines Chami

arjung commented 3 years ago

Closing this issue following the detailed response above. Thanks Ines!