What gaps exist in the Rust ecosystem?

[ErichDonGubler]

Problem Statement

It seems very valuable to have a place to recognize gaps in the ecosystem. This would allow Rust enthusiasts to know where their efforts may be appreciated, especially in tandem with other suites of information like possible mentors and a way to represent demand in the ecosystem.

Proposed Solution

Perhaps I can help with jumpstarting that effort by offering something I've been making for a while now! Currently, I've been making efforts to compile a list of recognized gaps in the Rust ecosystem using this repo call "Not-Yet-Awesome Rust", as a play on the traditional "awesome" lists you see around here on Github:

https://github.com/not-yet-awesome-rust/not-yet-awesome-rust

There's been a significant number of responses to the few Reddit posts I've made (~90 total comments on each of the two of three successful threads so far), which I've tried my best to keep up with until now (ordered by decreasing recency):

• 1
• 2 -- not so many responses here...
• 3

I've tried to make the content as actionable as I can by myself by taking the time to elicit use cases where they're not obvious. Most of the work until now has been lots of organizing -- there's not a lot of code involved in gathering this information as opposed to networking with the Rust community, asking around often, and being patient when wrangling the responses in whatever forms come back.

Notes

One enthusiastic user (@nabijaczleweli) has already made use of this list to implement parsers for CP437 and Registry.pol files from the Active Directory policy system in Windows, which felt amazing as an organizer! I consider the greatest successes of an effort like this to be the stimulation of the ecosystem, and seeing the gaps getting closed that way should, in my mind, be the top priority.

I mentioned earlier that this particular community might want to also provide some other aids for closing ecosystem gaps (emphasis not present in source):

This would allow Rust enthusiasts to know where their efforts may be appreciated, especially in tandem with other suites of information like possible mentors and a way to represent demand in the ecosystem.

I've tentatively created places to organize these two auxiliary ideas:

• The discussion repo is intended to be primary composed of issues where people can make initial contact with ecosystem gaps and announce their efforts or desire to close the gap. This can let the community have some form of:
  • gauging demand associated with certain gaps in the ecosystem, and
  • encourage social Rust users by providing a community that will assist them in their efforts. This is especially important for new Rust users, who will likely have some fire from recently trying out Rust!
• The mentors repo is a place to collect contact information (at a minimum, a Github username) for individuals willing to mentor FOSS projects in certain domains. For example, I work in data forensics, and am more than happy to provide myself as a resource when a crate developer wishes to consult someone with experience in that field.

[vitiral]

I think this kind of effort is super needed! Looking at not-so-awesome-rust however I have the concern that while is is great to list areas that rust is lacking to a competitor like python or C, I'm not sure simply listing gaps is the way to do it. Rather, looking at specific use cases and how much demand there is for those use cases is more interesting. How much demand is there for HD5 or iCalandar in rust?

Regardless, I think this is an important exercise, and simply listing these is useful because at the very least someone can use it as their first project while learning rust -- which is great!

[Enet4]

Allow me to chime in on the subject of gaps. I have encountered a fair variety of missing libraries and frameworks in the Rust eco-system in my "line of work", ranging from data science & machine learning, to medical imaging informatics. The latter is really a niche with hardly any demand at the moment, but the former is much more concerning: as the number of professional data scientists increase, the creation of high-level machine intelligence tools and frameworks, as well as the integration of machine learning models in production is a place where Rust can shine, but little work has been done on this end. I wrote about this in my Rust 2018 blog post.

The environment is getting better indeed, but some obstacles are still stopping me from using Rust more often, and the same issues may occur to other people. Just to name a few:

• Building HDF5 files is relevant, since I have often felt the need to process (and massage?) multiple data files, saving them in a single HDF5 data set. The owner of hdf5-rs seems to be experiencing some difficulties in the design and development process (it's true that this is not an easy task).
• Are we vision yet? One can count at least 3 image processing crates, with no clear go-to choice, unlike what happens in Python (what you need is most likely in SciPy already). There are also no complete and up-to-date bindings to OpenCV that I know of.

I pinpointed these two examples because they might involve different forms of intervention. The first one seems to me like a case where project maintenance is lacking. In order for the wider community to contribute, concrete issues need to be laid out and managed with possible mentoring in mind. This in turn requires an initial effort to make the codebase more accessible to new developers, by keeping everything documented and potentially polishing its public and internal APIs (the Rust API guidelines come to mind here). In the second example, I can only propose promoting stabilization of those which are more mature and actively maintained.

To serve as an ending note, I wonder what kind of metrics we can consider towards Rust projects in need of intervention. If a project is popular on GitHub (e.g. many stars/followers/forks), but without a major release yet, would this be a good sign of demand? Should there be a greater focus on projects with less contributors/maintainers, since those are less likely to become stable? Should we be completely proactive in choosing where to intervene, only seek out projects explicitly looking for help, or something inbetween? The mentors repo is also an interesting idea, and will influence how we tackle these interventions.

[ErichDonGubler]

@vitiral: Eliciting use cases has been my focus since NYAR's inception. It sounds like this concern might still exist even after seeing its content, though?

Demand is something I would hope to measure at least informally using the discussions repo after it has been developed a bit and I've had time to create discussions for content that already exists on NYAR. I was thinking that a site aggregating those informal metrics (like reactions to an issue, which in turn corresponds to content in the main NYAR list) could be used to loosely measure demand, at least as a rough first steps. What so you think about this approach?