search

MastafaF / multilingual_similarity_compare

Other
5 stars 0 forks source link

Shift everything to batch version with GPU support for faster results after testing #7

Open MastafaF opened 4 years ago

MastafaF commented 4 years ago

XLM-100 shifted to GPU with batch version tests:

Input: 

!sh similarity_XLM100_batch.sh 40 cls True

Output:

Confusion matrix:
langs   cs       de       en       es       fr       avg     
cs     0.00%   72.96%   70.46%   63.90%   60.44%   66.94%
de    62.70%    0.00%   51.18%   52.88%   58.41%   56.29%
en    73.93%   69.56%    0.00%   53.28%   65.47%   65.56%
es    63.34%   67.43%   37.46%    0.00%   51.95%   55.04%
fr    56.68%   52.85%   42.76%   39.09%    0.00%   47.84%
avg   64.16%   65.70%   50.47%   52.29%   59.07%   58.34%

Input: 

!sh similarity_XLM100_batch.sh 100 cls True

Output:

Confusion matrix:
langs   cs       de       en       es       fr       avg     
cs     0.00%   83.88%   87.08%   86.58%   84.75%   85.57%
de    79.59%    0.00%   72.76%   76.42%   74.89%   75.92%
en    92.31%   87.71%    0.00%   82.08%   84.62%   86.68%
es    86.25%   83.28%   62.70%    0.00%   72.83%   76.27%
fr    83.48%   75.89%   66.07%   66.77%    0.00%   73.05%
avg   85.41%   82.69%   72.15%   77.96%   79.27%   79.50%
MastafaF commented 4 years ago

With mean pooling:

Input: 

!sh similarity_XLM100_batch.sh 40 mean True

Output: 

Confusion matrix:
langs   cs       de       en       es       fr       avg     
cs     0.00%   69.56%   69.26%   65.33%   51.15%   63.83%
de    57.58%    0.00%   33.37%   44.36%   46.92%   45.55%
en    73.53%   60.14%    0.00%   42.72%   54.85%   57.81%
es    63.60%   61.84%   22.24%    0.00%   41.56%   47.31%
fr    53.08%   38.83%   34.20%   26.21%    0.00%   38.08%
avg   61.95%   57.59%   39.77%   44.66%   48.62%   50.52%

Input: 

!sh similarity_XLM100_batch.sh 100 mean True

Output:

Confusion matrix:
langs   cs       de       en       es       fr       avg     
cs     0.00%   80.05%   82.25%   83.38%   80.75%   81.61%
de    76.02%    0.00%   60.51%   65.67%   66.57%   67.19%
en    89.98%   79.12%    0.00%   69.66%   73.66%   78.11%
es    84.55%   76.32%   50.08%    0.00%   62.47%   68.36%
fr    80.82%   67.97%   52.11%   53.65%    0.00%   63.64%
avg   82.84%   75.87%   61.24%   68.09%   70.86%   71.78%