Final results

[daemon]

Thread containing final results for submission to conference. Mandatory checklist:

• Multiplies and parameters
• QRNN power measurements on 20/40/60/80/100
• AWD-LSTM power measurements on 100
• One example showing LD + LP/CN working poorly on AWD-LSTM for PTB
• LD + CN on QRNN (WT-2)
• LD + LP on QRNN (WT-2)
• LD + CN on QRNN (PTB)
• LD + LP on QRNN (PTB)
• No LD + CN on WT-2/PTB for QRNN + AWD-LSTM
• Final results should be just as good as previously reported in papers! (see AWD-LSTM, official GH repo) If not, we need to provide more evidence (namely train from scratch, ugh).
• Graphs showing performance of CN on non-LD and LD models

Optional checklist:

• WT-103 (~12 hours to train, probably ~18-24 hours with LD)
• Re-training after LD (use train from scratch and CN as baselines)
• Repackage code and make everything cohesive

[daemon]

Computing multiplies: (4650p, 800) fo (4650p, 1550p) fo (4650p, 1550p) fo (1200p, 1550p) fo (10000, 400p) softmax

import math
def mults(p):
    p = math.sqrt(p)
    return int(4650 * p * 800) + int(4650 * p * 1550 * p) + \
        int(4650 * p * 1550 * p) + int(1200 * p * 1550 * p) + int(10000 * 400 * p)

l = 0.536 # 0.6
l = 0.42 # 0.5
l = 0.32 # 0.4
l = 0.76 # 0.8
l = 0.12 # 0.2

# l = 0.8
print(mults(l), mults(1), mults(l) / mults(1))

# the 80% parameter model is roughly 69.15 (70)% of the multiplies