Benchmark Gensim vs Amazon SageMaker

[menshikh-iv]

Description

We want to compare gensim and SageMaker:

• LSI (after #1896) vs SageMaker PCA
• LDA vs SageMaker NTM vs SageMaker LDA

Metrics

• hardware performance:
  • time
  • memory
• algorithm performance (corresponding to a particular algorithm. for example - perplexity for LDA, etc). Also possible solve some supervised task with end2end evaluation with simple linear classifier on top).

The expected result is a benchmark table + a blog post describing the motivation, results and analysis.

[souravsingh]

@menshikh-iv I am interested in pursuing the issue.

[menshikh-iv]

@souravsingh feel free to resolve it (but this not as simple as it seems, the results should be reproducible, this is important).

[menshikh-iv]

@souravsingh FYI pretty exiting benchmark from @manneshiva for inspiration: https://github.com/manneshiva/benchmark-word2vec-frameworks + https://rare-technologies.com/machine-learning-hardware-benchmarks/