Sync with github/linguist

src-d / enry

A faster file programming language detector

https://blog.sourced.tech/post/enry/

Apache License 2.0

460 stars 51 forks source link

Sync with github/linguist #155

Open smola opened 6 years ago

smola commented 6 years ago

We synchronized with github/linguist in November 2017, an update is long overdue ;)

Latest enry v1.6.7 from Oct 24, 2018 is based on Linguist v5.2.0 commit 4cd558 from Sep 17, 2017.

This is an ☂️ issue with the goal to make enry use of at least at least Linguist v7.1.3 from Dec 12, 2018:

[x] support heuristics.yml instead of heuristics.rb github/linguist#4087. WIP in #189
[ ] support new XML strategy github/linguist#4222 - #192
[ ] support for new Tokenizer github/linguist#3846 -#193
[ ] fix Bayesian classifier accuracy drop (95% likely it's going to be fixed by a new Tokenizer above) - #194

smola commented 6 years ago

Well, we can wait until YAML heuristics are merged, so we don't need to fix the heuristics.rb parser: https://github.com/github/linguist/pull/4087

smola commented 6 years ago

YAML heuristics just landed in master.

bzz commented 6 years ago

Makes perfect sense and will be happy start looking into this in couple of weeks, if there is no rush.

bzz commented 5 years ago

It's WIP in #189

bzz commented 5 years ago

As scope has grown, this has been converted to ☂️ and description was updated, linking to subsequent issues.

bzz commented 5 years ago

Plan to cut a v1.7.0 release as soon as heuristics.yml update from #189 lands.

Other 3 items will be left for v1.8.0

bzz commented 5 years ago

The last item, a detection accuracy drop, seems to be the most important at the moment. I will get back to this, after quantifying the impact under the #246