Awesome-rust Renaissance (Rework of the list)

[luciusmagn]

The Premise

Awesome-rust is a very old list. So old, in fact, that it even predates Rust v1.0. Even better, awesome-rust existed before Cargo was released. Over the years, @kud1ing's tiny list grew into the massive collections it is now. It doesn't matter what you need to do, we probably have an entry for it.

Today, we have roughly 70 categories containing hundreds of projects. Sadly, we also have a lot of historical baggage and this list is in dire need of a review.

The Problem

There are projects in the list that are unmaintained or straight up broken. However, the fact that they are unmaintained isn't that much of a problem, so long as they weren't left in a major WIP state.

Another problem this list suffers from is inconsistency. Some projects have descriptions, some don't. Some categories order by author, other by crate name, another ones are a mess, where by we order by "what we have".

Last mildly annoying issue is that the Travis badges are slightly bigger than lines, so as they load the content gets continuously stretched. If one opens a link to some part of the list, they will probably end up in a whole different place of the list.

The Solution

I am a firm believer that it's about the time to do something about it. Of course, the task is bigger, but these should be the main points:

• Implementation of a new, uniform format. The goal is to improve readability, consistency and the outlook of the list
• Careful review of all entries, removing those that are either broken, have been superseded by another project* or left unmaintained in a heavy WIP state. It is paramount that we don't make a mistake here, so emphasis on the careful part.
• Filling in info for valid entries that lack either description, link to https://crates.io (if applicable), link to the repository (ditto) or a badge
• Review of the validity of categories (we had some issues with ambiguity quite some overlapping in the area of UI), turning some into subcategories of others, if needed
• On the topic of categories, we should also have a uniform ordering guideline, but whether we order by author or project name is up for discussion at the moment
• Doing something about the Travis badges. I am hoping that something like https://shields.io might help solve the problem with their custom badges, but more research needs to be done in the area. One solution is to remove them completely, but in doing so we lose some valuable easily accessible information. The best solution would be badges that wouldn't stretch the lines.
• Update the contribution guide to include all the relevant information, maybe add some policies to it. awesome-rust should definitely follow (and quietly does, I believe) the Rust Code of Conduct, so I am talking more about policies on what content we accept, what not.

*We are talking about cases such as where the author says that the original project Y is rewritten as X, which does the same, more and better

The Plan

First, I'd like to ask for some feedback on this, perhaps more suggestions:

1. Do you agree with my proposal to do a rework of the list?
2. What's your opinion on the Travis dilemma?
3. Would you prefer ordering by author (and their projects then alphabetically) or by project?
4. Is there a particular issue you would like to be addressed?

Next, I'd like to ask @memoryruins to produce his format proposals. We had a conversation about them a while back, so although some are posted in #419, there might have been some changes or new ideas (I don't quite remember what exactly came to fruition from said conversation).

Once @memoryruins does so, I'd also like to ask for input on the topic of design format.

If the operation is a go, we can start working out some of the details and then get to work. Of course, we wouldn't like the list to be dysfunctional for the duration of the renewal, so a new branch rework would be created and work on awesome-rust v2 would be done there.

[luciusmagn]

@memoryruins @svenstaro

[kud1ing]

Back then i saw 2 different approaches:

• generate HTML/Markdown from a single source of data (e.g. JSON file(s)):
  • Pros:
  • entries can be defined once but used in different categories (e.g,. Hyper is for both server and client)
  • output format (badges, sorting, whitespaces) can be changed central and independent of the data
  • data formats like JSON would be easier to parse and process for others than Markdown
  • Cons:
  • a generator needs to be run somewhere regularily
  • contributing to JSON files is not as nice as to a Markdown file
• keep the data in Markdown but improve the cleanup process
  • Pros:
  • people can still contribute to the Markdown file
  • Cons
  • the cleanup script can get complex
  • one Markdown file which is growing and growing

I've always liked one single file for viewers because you can see everything at once, and you can search on that page. I hope that stays.

[luciusmagn]

I am a proponent of the single-file design as well and I'd like to preserve the spirit of the list as it is.

I never thought about using a generator, but perhaps it is not a bad idea. I feel that TOML might be a nice middle ground as it is pretty readable and quite popular in the Rust community.

[kud1ing]

@nasa42 has used TOML for https://github.com/nasa42/libs.rs/tree/master/categories

[brycx]

Hi

I've been trying to do look through some of the listed projects, but am having a hard time defining when a project should be removed.

For example I know libOctavo has had no activity, based on the actual code files, for 2 years. It also states itself as a heavy WIP. suruga lists itself as inactive, so that seems to be a no-brainer.

But for some projects, like a gameboy simulator that has been done as a learning project a couple of years ago. That may still be useful to keep because others can still learn from it. How are such projects to be handled?

[kud1ing]

@brycx I think one needs to distinguish "usefulness" as "i can use the code" and "the code is interesting/insightful etc.".

To me "i can use the code" was the most important goal for awesome-rust. There is no reasons we couldn't keep pointers to interesting code but if they can not be used it should be in a different category.

[brycx]

@kud1ing Thanks. I'll have another swing at it.

[kud1ing]

That being said, i don't think there is an easy solution. It will always be a struggle.

[palfrey]

Is there still interest in this? Now that we've got a rustified linter (#792), I was considering whether more could be done beyond just checking for broken links.

Thoughts for changes:

• Enforce a format for project lines (e.g. [name](link to project) - description [crate badge if relevant][build badge] is my current thinking, plausibly with a check that if the build badge is Travis and the link is Github then make sure the names line up)
• Check the ToC i.e. make sure it corresponds to the current list of headers further down
• Alphabetical sorting of projects within a section?
• Possibly replacing all the <img src blah with proper Markdown syntax

Some of this would be easier by having a generator, but I think the idea of having a easy to edit format for users is also good, while having a decent tool that makes it easy for people to do the Right Thing when adding new entries. I'm willing to write such extensions to the linter tool if wanted? Possibly fold the cleanup.rs into it and document differences between current and desired state in line with rustfmt in builds?

[joshka]

Something I recently contributed to a different awesome repo was just converting everything to a table, and use the flat styled shields.io badges for each repo showing the number of stars. Number of stars (a proxy for popularity) seems more useful than whether or not the latest build is failing.

You do sacrifice a bit of vertical space (~20%) to show the list as a table, but the upside seems by making the items in the list easier to scan (as a top to bottom scan is not ragged and is not interrupted by non-text information).

Before: https://github.com/tensorchord/Awesome-LLMOps/tree/863e9b841d26064bcceb050748c00eb561d2d130 After: https://github.com/tensorchord/Awesome-LLMOps

---

On the caching side of things, github automatically caches the svgs behind the camo.githubusercontent proxy, loading them from there instead of from the root site, so things are relatively fast. On the Awesom LLM Ops repo, 281 projects (some without links to their badge) takes around 20s total to load whe throttled with whatever Firefox's DSL setting does or 3.4s for the wifi setting.

Having the items in a table means the row size doesn't seem to jankily adjust when the badges load (this problem was mentioned above)