시스템1 성능 실험

bart2001 commented 6 years ago

시스템1 (sejong_bio_filters)

학습 중

options.json

{
 "all_clip_norm_val": 10.0,
 "batch_size": 256,
 "bidirectional": true,
 "char_cnn": {
  "activation": "relu",
  "embedding": {
   "dim": 16
  },
  "filters": [
   [
    1,
    32
   ],
   [
    2,
    32
   ],
   [
    2,
    64
   ],
   [
    3,
    128
   ],
   [
    3,
    256
   ],
   [
    4,
    512
   ],
   [
    4,
    1024
   ]
  ],
  "max_characters_per_token": 4,
  "n_characters": 362,
  "n_highway": 2
 },
 "dropout": 0.1,
 "lstm": {
  "cell_clip": 3,
  "dim": 4096,
  "n_layers": 2,
  "proj_clip": 3,
  "projection_dim": 512,
  "use_skip_connections": true
 },
 "n_epochs": 10,
 "n_negative_samples_batch": 45,
 "n_tokens_vocab": 4488,
 "n_train_tokens": 32119740,
 "unroll_steps": 4
}

bart2001 commented 6 years ago

학습 최종 성능

Batch 156800, train_perplexity=17.776701
Total time: 67965.83387494087

bart2001 commented 6 years ago

n_character 설정이 잘못되어서 재학습 필요 (documentation에 따르면 261로 설정해야 함)

nlp-research / bilm-tf

시스템1 성능 실험 #3