versioneye / crawl_r

VersionEye crawlers implemented in Ruby.
https://www.versioneye.com
Other
10 stars 6 forks source link

Move LicenseMatcher into own project #59

Open timgluz opened 7 years ago

timgluz commented 7 years ago

It's one of reasons why specs are slow - it must rebuild the token index each time and it takes time;

Fastest solutions is to move everything into own repo; But it doesnt speed up creation of index;

Better approach: use proper tools, refactor it into 2 processes: training/match, where the training-process will build and save model; and match-process will re-use already existing model;

reiz commented 7 years ago

@timgluz That sounds good. We could store the trained index in DB or in a git repo.

timgluz commented 7 years ago

Currently doing research - going to checkout which tools are possible to use and which them are the best option;

timgluz commented 7 years ago

Done: all the relating packages can be found here: https://github.com/Fosslim

Steps for refactoring:

i wrote a simple ROR app fosslim/lmweb to demonstrate the usage