Optimize typos tests

[irinakhismatullina]

In accordance with #667.

• Replaced test FastText model with a very small one.
• Loaded to drive a small raw dataset to speed up test for prepare_dataset() with loading file from drive.
• Added @long_test decorator to a couple of tests.

[irinakhismatullina]

I don't understand why it can't see changed bin file in out-of-docker builds. Any ideas @zurk @vmarkovtsev?

[vmarkovtsev]

Still 25 minutes, does not seem to help.

[irinakhismatullina]

When I run all typos tests locally, it takes 3 seconds (without installs and setting up env, just tests). Can it be Travis, or too much other builds?

[vmarkovtsev]

@irinakhismatullina Please refer to the Travis logs, it reports the time taken by each command.

[irinakhismatullina]

@vmarkovtsev Well, here total for all tests (format + typos) is 920s, 15 min. It doesn't show time for smaller chunks, or I couldn't find it. Also I still don't understand why it fails out of docker here. Logs tell that the file can't be found, i just replaced it. Everything runs alright locally out of docker and inside the docker in Travis.

[zurk]

Add your test_ft.bin to https://github.com/src-d/style-analyzer/blob/651ee1664fb074b0dded87c3e10c6e340f5dd5f7/setup.py#L79

Should help.

[irinakhismatullina]

Wow, that's strange, thank you.

[vmarkovtsev]

@irinakhismatullina Here is what I see

Python 3.5 is doing great, however, 3.6 and 3.7 are very, very slow. You need to go through the log timings and make a report.

[vmarkovtsev]

Wow, that's strange, thank you.

The reason is that we test the built package, and you tested the local files. Extra files must be included into the package, otherwise, they are not installed. We had precisely the same problem in the past and that's why we actually test the package now.

[zurk]

Something is strange. I think it can be Travis issues. Let's restart are see if we have the same result?

UPD: restarted.

[irinakhismatullina]

The reason is that we test the built package, and you tested the local files. Extra files must be included into the package, otherwise, they are not installed. We had precisely the same problem in the past and that's why we actually test the package now.

Yes, I understand, it's just that I definitely replaced the filename in setup.py, but looks like that change was somehow lost in the final commit. My bad.

[zurk]

Ok, @vmarkovtsev it is because of https://github.com/src-d/style-analyzer/blob/master/lookout/style/format/tests/test_quality_report_noisy.py

Since we always retrain the model for a noisy report it takes a huge amount of time in a test also. I am going to fix it ASAP. (In Python 3.5 we just skip this test because of difflib)

typos test are super fast.

[vmarkovtsev]

OK, this means that the original PR is fine. Merging.

[vmarkovtsev]

@irinakhismatullina Tests failed

[irinakhismatullina]

I wonder, usually pandas.read_csv("*.xz") works fine, but somehow after loading from drive or reading from tempdir it became a problem, that's unexpected.