asyml / texar-pytorch

Integrating the Best of TF into PyTorch, for Machine Learning, Natural Language Processing, and Text Generation. This is part of the CASL project: http://casl-project.ai/
https://asyml.io
Apache License 2.0
744 stars 118 forks source link

add the kl divergence loss #328

Closed haoyuLucas closed 3 years ago

haoyuLucas commented 3 years ago
  1. Implement the KL divergence loss for UDA(unsupervised data augmentation);
  2. Support distribution sharpening and confidence masking.
haoyuLucas commented 3 years ago

The Travis build failed because of an error in "tests/core/attention_mechanism_test.py". Any idea why this would happen?

gpengzhi commented 3 years ago

The Travis build failed because of an error in "tests/core/attention_mechanism_test.py". Any idea why this would happen?

The line that triggers the segmentation fault happens to be torch._C._log_api_usage_once("python.nn_module")

I suspect it is something related to sentencepiece==0.1.94 that prevents torch binaries from normally register its resources. Could you downgrade sentencepiece to 0.1.91? This might resolve such issue.

codecov[bot] commented 3 years ago

Codecov Report

Merging #328 (b5a8edb) into master (f016043) will increase coverage by 0.03%. The diff coverage is 95.45%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #328      +/-   ##
==========================================
+ Coverage   80.14%   80.17%   +0.03%     
==========================================
  Files         134      135       +1     
  Lines       11195    11217      +22     
==========================================
+ Hits         8972     8993      +21     
- Misses       2223     2224       +1     
Impacted Files Coverage Δ
texar/torch/losses/info_loss.py 95.45% <95.45%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update f016043...b5a8edb. Read the comment docs.

haoyuLucas commented 3 years ago

Downgrading the sentencepiece solved the problem.

haoyuLucas commented 3 years ago

Shall we merge this PR so that we can use this in Forte for Data Augmentation? Thanks!