facebookresearch / pytext

A natural language modeling framework based on PyTorch
https://pytext.readthedocs.io/en/master/
Other
6.34k stars 801 forks source link

AssertionError: target_prob must be set to True for KD losses #992

Open snakecy opened 5 years ago

snakecy commented 5 years ago

Steps to reproduce

target_prob is set to true, and the error raised.




Observed Results

Relevant Code

// TODO(you): code here to reproduce the problem
snisarg commented 5 years ago

Can you paste the error and the config you're using please.

snakecy commented 5 years ago

@snisarg the config likes :

{
  "config": {
    "task": {
      "DocClassificationTask": {
        "features": {
          "word_feat": {
            "embed_dim": 128,
            "embedding_init_strategy": "zero",
            "pretrained_embeddings_path": "train/token_vec_128.bin"
          }
        },
        "featurizer": {
          "SimpleFeaturizer": {
            "sentence_markers": null,
            "lowercase_tokens": true,
            "split_regex": "\\s+",
            "convert_to_bytes": false
          }
        },
        "data_handler": {
          "columns_to_read": [
            "doc_label",
            "text"
          ],
          "shuffle": true,
          "sort_within_batch": true,
          "train_path": "train.tsv",
          "eval_path": "eval.tsv",
          "test_path": "test.tsv",
          "train_batch_size": 128,
          "eval_batch_size": 128,
          "test_batch_size": 128,
          "column_mapping": {},
          "max_seq_len": -1
        },
        "trainer": {
          "epochs": 2,
          "optimizer": {
            "Adam": {
              "lr": 0.001,
              "weight_decay": 1e-05
            }
          }
        },
        "exporter": null,
        "model": {
          "representation": {
            "BiLSTMDocAttention": {
              "lstm": {
                "dropout": 0.5,
                "lstm_dim": 64,
                "num_layers": 2,
                "bidirectional": true
              },
              "pooling": {
                "SelfAttention": {
                  "attn_dimension": 64,
                  "dropout": 0.4
                }
              },
              "mlp_decoder": null
            }
          },
          "decoder": {
            "load_path": null,
            "save_path": null,
            "freeze": false,
            "shared_module_key": null,
            "hidden_dims": [],
            "out_dim": null,
            "layer_norm": true
          },
          "output_layer": {
            "load_path": null,
            "save_path": null,
            "freeze": false,
            "shared_module_key": null,
            "loss": {
              "CrossEntropyLoss": {}
            },
            "label_weights": null
          }
        },
        "labels": {
          "export_output_names": [
            "doc_scores"
          ],
          "label_weights": {},
          "target_prob": true 
        }
      }
    },
    "save_snapshot_path": "m.pt",
    "export_caffe2_path": "m.c2",
    "export_onnx_path": "m.nx"
  }
}

the set is "target_prob": true

snisarg commented 5 years ago

And what's the error raised? How can I reproduce this?

snakecy commented 5 years ago

@snisarg when the set is "target_prob": true ,the error raised as the title mentioned.