rufex2001
commented
2 years ago How are you generating this config file? And is your model based on the ConvE code somehow? Maybe somehow the ConvE configuration key is still making an appearance somewhere?
On Sat, 25 Sep 2021, 23:57 Filip Cornell, @.***> wrote:
Hello (again, haha)
I am currently running some experiments with your package; I love it. However, I am concerned about the validity of one of my experiments. I performed an AxSearchJob on one of my models (which I have named RIE2). In my config.yaml from one of the runs, I get a conve setting, which is strange to me. I don't know why it is there, and I am concerned that it might be, in the worst case scenario, a ConvE-model training in the background (and not my model implementation).
Now, I have tried to make sure that my worries are not legitimate by introducing exceptions in all parts of the ConvE code, if it is executed, and it seems to run through without a problem. In other words, it does not seem like ConvE is used, since no code from the ConvE-part of the code seems to be used. Also, loading the model will load an RIE2-model.
I would simply like to know; why is there a ConvE part being created in this config.yaml? Even if I erase that part and re-run the experiment, it comes back into the new config.yaml-file for the new experiment. I do not believe there is an issue here, but I would just like to make sure with a 100 % certainty.
Thank you in advance, and thanks again for a very nice package!
The config file is as seen below:
1vsAll: class_name: TrainingJob1vsAllKvsAll: class_name: TrainingJobKvsAll label_smoothing: -0.10110532511025666 query_types: _po: true so: false sp: trueRIE2: class_name: RIE2 entity_embedder: +++: +++ dropout: 0.13526539038866758 normalize: p: 2.0 regularize_weight: 1.4196422325008177e-05 type: lookup_embedder l_norm: 1.0 relation_embedder: +++: +++ dropout: 0.1788316359743476 normalize: p: -1.0 regularize_weight: 0.0002284745422735363 type: lookup_embedder round_dim: false tuneable_doubletan: falseax_search: class_name: AxSearchJob num_sobol_trials: 30 num_trials: 30 parameter_constraints: [] parameters:
- name: model type: choice values:
- RIE2
- reciprocal_relations_model
- is_ordered: true name: train.batch_size type: choice values:
- 128
- 256
- 512
- 1024
- name: train.type type: fixed value: negative_sampling
- name: train.optimizer type: choice values:
- Adam
- Adagrad
- name: train.loss type: fixed value: kl
- bounds:
- 0.0003
- 1.0 log_scale: true name: train.optimizer_args.lr type: range
- name: train.lr_scheduler type: fixed value: ReduceLROnPlateau
- name: train.lr_scheduler_args.mode type: fixed value: max
- name: train.lr_scheduler_args.factor type: fixed value: 0.95
- name: train.lr_scheduler_args.threshold type: fixed value: 0.0001
- bounds:
- 0
- 10 name: train.lr_scheduler_args.patience type: range
- is_ordered: true name: lookup_embedder.dim type: choice values:
- 128
- 256
- 512
- name: lookup_embedder.initialize type: choice values:
- xaviernormal
- xavieruniform
- normal_
- uniform_
- name: lookup_embedder.initializeargs.normal.mean type: fixed value: 0.0
- bounds:
- 1.0e-05
- 1.0 log_scale: true name: lookup_embedder.initializeargs.normal.std type: range
- bounds:
- -1.0
- -1.0e-05 name: lookup_embedder.initializeargs.uniform.a type: range
- name: lookup_embedder.initialize_args.xavieruniform.gain type: fixed value: 1.0
- name: lookup_embedder.initialize_args.xaviernormal.gain type: fixed value: 1.0
- is_ordered: true name: lookup_embedder.regularize type: choice values:
- ''
- l3
- l2
- l1
- name: lookup_embedder.regularize_args.weighted type: choice values:
- true
- false
- bounds:
- 1.0e-20
- 0.1 log_scale: true name: RIE2.entity_embedder.regularize_weight type: range
- bounds:
- 1.0e-20
- 0.1 log_scale: true name: RIE2.relation_embedder.regularize_weight type: range
- bounds:
- -0.5
- 0.5 name: RIE2.entity_embedder.dropout type: range
- bounds:
- -0.5
- 0.5 name: RIE2.relation_embedder.dropout type: range
- bounds:
- -0.3
- 0.3 name: KvsAll.label_smoothing type: range
- is_ordered: true name: RIE2.l_norm type: choice values:
- 1.0
- 2.0
- name: RIE2.entity_embedder.normalize.p type: choice values:
- -1.0
- 2.0
- name: RIE2.relation_embedder.normalize.p type: choice values:
- -1.0
- 2.0
- name: negative_sampling.implementation type: fixed value: spo sobol_seed: 0console: format: {} quiet: falseconve: # <--- HERE IS THE CONVE PART 2D_aspect_ratio: 2 # <--- HERE IS THE CONVE PART class_name: ConvE # <--- HERE IS THE CONVE PART convolution_bias: true # <--- HERE IS THE CONVE PART entity_embedder: # <--- HERE IS THE CONVE PART +++: +++ # <--- HERE IS THE CONVE PART dropout: 0.2 # <--- HERE IS THE CONVE PART type: lookup_embedder # <--- HERE IS THE CONVE PART feature_map_dropout: 0.2 # <--- HERE IS THE CONVE PART filter_size: 3 # <--- HERE IS THE CONVE PART padding: 0 # <--- HERE IS THE CONVE PART projection_dropout: 0.3 # <--- HERE IS THE CONVE PART relation_embedder: # <--- HERE IS THE CONVE PART +++: +++ # <--- HERE IS THE CONVE PART dropout: 0.2 # <--- HERE IS THE CONVE PART type: lookup_embedder # <--- HERE IS THE CONVE PART round_dim: false # <--- HERE IS THE CONVE PART stride: 1 # <--- HERE IS THE CONVE PARTdataset: +++: +++ files: +++: +++ entity_ids: filename: entity_ids.del type: map entity_strings: filename: entity_strings.del type: idmap relation_ids: filename: relation_ids.del type: map relation_strings: filename: relation_ids.del type: map test: filename: test.del size: 3134 split_type: test type: triples test_without_unseen: filename: test_without_unseen.del size: 2924 split_type: test type: triples train: filename: train.del size: 86835 split_type: train type: triples train_sample: filename: train_sample.del size: 3034 split_type: train type: triples valid: filename: valid.del size: 3034 split_type: valid type: triples valid_without_unseen: filename: valid_without_unseen.del size: 2824 split_type: valid type: triples name: wnrr num_entities: 40943 num_relations: 11 pickle: trueentity_ranking: chunk_size: -1 class_name: EntityRankingJob filter_splits:
- train
- valid filter_with_test: true hits_at_k_s:
- 1
- 3
- 10
- 50
- 100
- 200
- 300
- 400
- 500
- 1000 metrics_per: argument_frequency: false head_and_tail: false relation_type: true tie_handling: rounded_mean_rankeval: batch_size: 256 num_workers: 0 pin_memory: false split: valid trace_level: epoch type: entity_rankinggrid_search: class_name: GridSearchJob parameters: +++: +++ run: trueimport:
- RIE2
- reciprocal_relations_modeljob: device: cuda type: trainlookup_embedder: class_name: LookupEmbedder dim: 128 dropout: 0.0 initialize: xavieruniform initializeargs: +++: +++ normal: mean: 0.0 std: 0.00043470941774473033 uniform_: a: -0.4313754705902655 xaviernormal: gain: 1.0 xavieruniform: gain: 1.0 normalize: p: -1.0 pretrain: ensure_all: false model_filename: '' regularize: lp regularize_args: +++: +++ p: 1 weighted: true regularize_weight: 0.0 round_dim_to: [] sparse: falsemanual_search: class_name: ManualSearchJob configurations: [] run: truemodel: RIE2modules:
- kge.model.embedder
- kge.model
- kge.jobnegative_sampling: class_name: TrainingJobNegativeSampling filtering: implementation: fast_if_available o: false p: false s: false split: '' frequency: smoothing: 1 implementation: triple num_samples: o: -1 p: 0 s: 3 sampling_type: uniform shared: false shared_type: default with_replacement: truerandom_seed: default: -1 numba: -1 numpy: -1 python: -1 torch: -1reciprocal_relations_model: base_model: +++: +++ type: RIE2 class_name: ReciprocalRelationsModelsearch: device_pool: [] num_workers: 1 on_error: abort type: ax_searchtrain: abort_on_nan: true auto_correct: true batch_size: 256 checkpoint: every: 5 keep: 3 keep_init: true loss: kl loss_arg: .nan lr_scheduler: ReduceLROnPlateau lr_scheduler_args: +++: +++ factor: 0.95 mode: max patience: 2 threshold: 0.0001 lr_warmup: 0 max_epochs: 400 num_workers: 0 optimizer: +++: +++ default: args: +++: +++ lr: 0.4969193773628293 type: Adagrad pin_memory: false split: train subbatch_auto_tune: false subbatch_size: -1 trace_level: epoch type: negative_sampling visualize_graph: falsetraining_loss: class_name: TrainingLossEvaluationJobuser: +++: +++valid: early_stopping: patience: 10 threshold: epochs: 50 metric_value: 0.05 every: 5 metric: mean_reciprocal_rank_filtered_with_test metric_expr: float("nan") metric_max: true split: valid trace_level: epoch
The new ConvE-file looks like this (so it should not be run, which it does not seem to be).
import torchfrom torch import Tensorimport math from kge import Config, Datasetfrom kge.model.kge_model import RelationalScorer, KgeModel
class ConvEScorer(RelationalScorer): r"""Implementation of the ConvE KGE scorer. Must be used with ReciprocalRelationsModel."""
def __init__(self, config: Config, dataset: Dataset, configuration_key=None): raise Exception("This model should not be used at all! This is the error! ") def score_emb(self, s_emb, p_emb, o_emb, combine: str): raise Exception("This model should not be used at all! This is the error! ")class ConvE(KgeModel): r"""Implementation of the ConvE KGE model."""
def __init__( self, config: Config, dataset: Dataset, configuration_key=None, init_for_load_only=False, ): raise Exception("This model should not be used at all! This is the error! ") def score_spo(self, s: Tensor, p: Tensor, o: Tensor, direction=None) -> Tensor: raise Exception("This model should not be used at all! This is the error! ")— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/uma-pi1/kge/issues/228, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABEWXZGKVOTTFGBE5BTSAKDUDZAUTANCNFSM5EYA6B7A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
Filco306
Hello (again, haha)
I am currently running some experiments with your package; I love it. However, I am concerned about the validity of one of my experiments. I performed an AxSearchJob on one of my models (which I have named
RIE2). In myconfig.yamlfrom one of the runs, I get aconvesetting, which is strange to me. I don't know why it is there, and I am concerned that it might be, in the worst case scenario, a ConvE-model training in the background (and not my model implementation).Now, I have tried to make sure that my worries are not legitimate by introducing exceptions in all parts of the ConvE code, if it is executed, and it seems to run through without a problem. In other words, it does not seem like ConvE is used, since no code from the ConvE-part of the code seems to be used. Also, loading the model will load an
RIE2-model.I would simply like to know; why is there a ConvE part being created in this
config.yaml? Even if I erase that part and re-run the experiment, it comes back into the newconfig.yaml-file for the new experiment. I do not believe there is an issue here, but I would just like to make sure with a 100 % certainty.Thank you in advance, and thanks again for a very nice package!
The config file is as seen below:
The new ConvE-file looks like this (so it should not be run, which it does not seem to be).