bclarkson-code
commented
3 months ago I ran a benchmark on the new code and the speedup is huge! ~8.5x faster
┏━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓ ┃ Benchmark ┃ Min ┃ Max ┃ Mean ┃ Min (+) ┃ Max (+) ┃ Mean (+) ┃ ┡━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩ │ original │ 0.677 │ 0.744 │ 0.692 │ 0.080 (8.5x) │ 0.081 (9.2x) │ 0.080 (8.6x) │ └───────────┴─────────┴─────────┴─────────┴─────────────────┴─────────────────┴─────────────────┘
Currently, the embedding layer uses more memory and is slower than I would expect. It should be optimised