Porting to rust - Githubissues

anowell / are-we-learning-yet

How ready is Rust for Machine Learning?

http://arewelearningyet.com

Creative Commons Attribution 4.0 International

437 stars 61 forks source link

Porting to rust #96

Closed anowell closed 2 years ago

anowell commented 3 years ago

The scraping/scoring is now implemented in rust. It is still crawling, but better behaved (obeys documented rate limit, and includes an identifiable user-agent).

Few things to remain before merging:

[x] Output the crates_generated.yaml to the _data directory
[x] Update some of the html templates - the schema ouf crates_generated has changed slightly
[x] Document how to run this
[x] Incorporate it into the travis automation (double checking how travis is caching to reduce crates.io requests, but also ensure cache expiry happens at least daily)

The following can be done after merging:

[x] Switching to github actions (there is a placeholder file since I was looking into github action caching)
[x] Consider switching to cobalt.rs or some other simpler, ruby-free site generator

anowell commented 2 years ago

Everything here appears to be working. Publishing is probably still broke, but I need to merge something to dig into that since the publish should only happen on merge-to-master.

@quietlychris - with this change, we should be able to start rebasing other PRs - the checks should now pass to allow merging, even if the publishing might not be working quite yet.

elpiel commented 2 years ago

I will check this PR for inspiration when finishing up areweinspaceyet.org!

Re publishing: I was wondering if there is an easy way to enable releases of a commit hash in a subdirectory. This could be interesting for CI and testing the new changes

anowell commented 2 years ago

@elpiel - It should be possible to change or duplicate the publish step, but set destination_dir. Setting that to something like version/${{env.GITHUB_REF_NAME}} (or GITHUB_SHA might accomplish what you're trying to do? You may also need to take note of the keep_files option in that action too depending on how/where you run the action.

I'm not going to immediately do any of this, but I do like the idea for reviewing PRs. There is also a Comment Action which should make it possible to automatically comment on the PR with the URL.