Train the pretrained embedding for fbwq_full is time consuming using libKGE

[Xie-Minghui]

When I use the LibKGE to train the RotatE or ComplEx embeddings for fbwq dataset, it runs very slow on my server. So I wonder how much time and what devices you trained these embedding?

[Xie-Minghui]

I notice you trained about 170 epochs to get pretrained embedding , how much time did you cost? Look forward to your reply!

[apoorvumang]

what is your setup? are you using a gpu?

[Xie-Minghui]

Yes. My device is P40 , which has 22G memory

[apoorvumang]

and what config did you use? ie batch size etc.

[Xie-Minghui]

and what config did you use? ie batch size etc. batch size is 100, lr is 0.2

[apoorvumang]

can you share the text of the config file? it looks like you're not doing negative sampling, but doing K vs All entities.

you need to do neg samp when entities are this large, otherwise it will be very slow

[Xie-Minghui]

this is my setting: `job.type: train dataset.name: fbwq_full train: max_epochs: 1000 optimizer.default: type: Adagrad args: lr: 0.2 lr_scheduler: ReduceLROnPlateau lr_scheduler_args: mode: max patience: 4

model: complex lookup_embedder: dim: 100 regularizeweight: 0.8e-7 initialize: normal initializeargs: normal: mean: 0.0 std: 0.1 ` I don't set this. Maybe it use KvsALL by default. this is your setting: import: [lookup_embedder]

complex: class_name: ComplEx entity_embedder: type: lookup_embedder +++: +++ relation_embedder: type: lookup_embedder +++: +++ I can't find where to set this.

[apoorvumang]

Take a look at this config, its for WikiData5M from the official libkge github repo

http://web.informatik.uni-mannheim.de/pi1/libkge-models/wikidata5m-complex.yaml

It uses negative sampling. You should be able to modify it for your needs. If it works for you, I would request you share it here as well so I can add it to our repo

[Xie-Minghui]

Take a look at this config, its for WikiData5M from the official libkge github repo

http://web.informatik.uni-mannheim.de/pi1/libkge-models/wikidata5m-complex.yaml

It uses negative sampling. You should be able to modify it for your needs. If it works for you, I would request you share it here as well so I can add it to our repo

Thanks for quick reply. I will share it right after I succeed

[Xie-Minghui]

Take a look at this config, its for WikiData5M from the official libkge github repo

http://web.informatik.uni-mannheim.de/pi1/libkge-models/wikidata5m-complex.yaml

It uses negative sampling. You should be able to modify it for your needs. If it works for you, I would request you share it here as well so I can add it to our repo

The configuration file complex.yaml is as follows:

complex: entity_embedder: dropout: 0.44299429655075073 regularize_weight: 7.830760727899156e-12 relation_embedder: dropout: -0.4746062345802784 regularize_weight: 1.182876478423781e-10 dataset: name: fbwq_full eval: batch_size: 200 chunk_size: 25000 num_workers: 2 import:

• complex lookup_embedder: dim: 128 initializeargs: normal: mean: 0.0 std: 5.8970567449527816e-05 regularize_args: p: 1 weighted: true sparse: true model: complex negative_sampling: implementation: batch num_samples: o: 7851 s: 2176 shared: true with_replacement: false train: auto_correct: true batch_size: 128 loss_arg: .nan lr_scheduler: ReduceLROnPlateau lr_scheduler_args: factor: 0.95 mode: max patience: 1 threshold: 0.0001 max_epochs: 200 num_workers: 8 optimizer_args: lr: 0.6560544891789137 type: negative_sampling valid: early_stopping: min_threshold: epochs: 10 metric_value: 0.1 patience: 10

I have test it, it worked well.

[Xie-Minghui]

The libKGE doesnot implement Tucker, so I used the official repo to train Tucker on your fbwq_full dataset. But it's really slow, it takes 6 hours to finish one epoch. I wonder whether you tried Tucker on fbwq_full dataset ?

[apoorvumang]

Thanks for the config.

We didn't try Tucker on fbwq_full. However I think the latest version of libkge does implement Tucker (they call it Relational Tucker I think) so you can look into that.