Add Continuous Integration

src-d / awesome-machine-learning-on-source-code

Cool links & research papers related to Machine Learning applied to source code (MLonCode)

Creative Commons Attribution Share Alike 4.0 International

6.24k stars 840 forks source link

Add Continuous Integration #140

Closed bzz closed 5 years ago

bzz commented 5 years ago

Fixes first 2 items from #72:

check for link duplicates in README
check that links are accessible
fixes paper link duplication

CI is expected to fail, as there are some broken links in README right now.

If we agree on approach of dealing with broken links (remove/replace/etc) I can fix those here, but would prefer to deal with them in a separate PR.

dkhamsing/awesome_bot was chosen after a small research

https://github.com/raviqqe/liche
- ✅ fast, written in Go
- 🚫 fails with
  
  Error when reading response headers: small read buffer. Increase ReadBufferSize. Buffer size=4096
needs refactoring to set ReadBufferSize
- https://github.com/tcort/markdown-link-check
- 🚫 fails on redirects
- 🐌 slow

m09 commented 5 years ago

Good for me now.

m09 commented 5 years ago

Sorry I just re-read your initial PR description and you specify that you didn't want to fix the links in this one. I can remove my commits if you want, I didn't see it when I fixed the links. Still I think it might be good to have a non-failing CI on master.

bzz commented 5 years ago

Thank you for taking care @m09 - looks great to me!

Few more improvements:

added badge to README
scheduled a daily cron job \w CI of master, so we could notice \wo a PR if something gets broken

should be good to merge, as soon as @EgorBu or @vmarkovtsev have time to ✅.

BTW right now all https://dl.acm.org links are not accessible, can not open any of it in browser and CI says:

  1. [L141] 522 https://dl.acm.org/citation.cfm?id=3180167  
  2. [L148] 522 https://dl.acm.org/citation.cfm?id=2892661  
  3. [L183] 522 https://dl.acm.org/citation.cfm?id=3243130  
  4. [L213] 522 https://dl.acm.org/citation.cfm?id=3243132  
  5. [L230] 522 https://dl.acm.org/citation.cfm?id=3240471  
  6. [L238] 522 https://dl.acm.org/citation.cfm?id=3243131

May be good candidates for manually transfer to http://archive.is/ as soon as available.

m09 commented 5 years ago

Thanks for the cron, it'd be good indeed to fix stuff regularly to avoid having contributors fail CI for some reason independent from their contribution! Which leads me to ask: maybe we should keep the current job only for the cron and have a more elaborated one that fails only if the added links are problematic?

bzz commented 5 years ago

@m09 that is a nice improvement suggestion indeed!

But as with any improvement - I would prefer merging the baseline first and then having a separate issue for refactoring/adding the improvement, addressing it when we stumble up on it as an actual problem instead of keeping this open forever and keep adding hypothetical improvements.

WDYT?

\cc @EgorBu @vmarkovtsev for review