scicloj / scicloj.github.io

The Scicloj website
http://scicloj.org
MIT License
4 stars 4 forks source link

Idea: Drive `en/docs/resources/libs.md` with data #57

Closed teodorlu closed 2 years ago

teodorlu commented 2 years ago

Idea: by generating the collection of libraries from data, we can:

  1. Generate backwards compatible markdown
  2. Easily share an EDN file with all the libraries --- for example in order to produce nice visualizations

Status:

  1. model.edn represents a possible data model for tags and libraries
  2. gen.clj is a small CLI working on model.edn. It's able to produce lib.md as it is --- except for tag ordering, which I didn't think was too important.

Remaining work:

Right now, running $ gen.clj lib.md generates the tag list, and the heading ## Diverse toolsets. Each section in the markdown file requires a bit of manual work, mapping the headline over to a category tag (eg Diverse toolsets -> :div-tools).

To see the current status, just check the PR diff for libs.md. libs.md in this pr is generated by $ gen.clj lib.md.

What now?

  1. First, I'd like to know whether we want to merge gen.clj and model.edn once they reach a more complete state. The downside of autogenerating files is implicit process -- instead of just editing markdown files, one has to know that the source data file or the script has to be edited.
  2. If yes, I can finish up the remaining sections
  3. After merging, anyone can download https://github.com/scicloj/scicloj.github.io/blob/master/content/en/docs/resources/model.edn and experiment with visualization.

In other words, I want to scope this PR to just provide the EDN file with backwards compatible markdown file generation.

teodorlu commented 2 years ago

@daslu - update.

model.edn is looking quite good. The data about each library fitted quite nicely into the data model.

I'm able to generate a libs.md file from model.edn, but it's not a 100 % match. See diff on this PR. libs.md on this PR is generated 100 % from model.edn.


I think it's okay to merge model.edn as it is. But the libs.md I generated has some defects. We could consider simplifying the libs.md format a bit to make it easier to auto generate. Spacing, use of - and :, and parentheses are causing some difficulties as of now.

If we get model.edn on master, It would be interesting to share it with the wider Clojure community, see if we can surface any ideas about presenting the list.

teodorlu commented 2 years ago

We could consider having a :readiness property rather than :act and :exp-tags.

Here's an :act example:

{:lib/name "fastmath",
 :lib/url "https://github.com/generateme/fastmath",
 :lib/category :div-tools,
 :tags #{:stat :rand :act :ml :math},
 :star :star,
 :description
 "a collection of functions for mathematical and statistical computing, machine learning, etc., wrapping several JVM libraries"}

Here's an :exp example:

{:lib/name "Notespace",
 :lib/url "https://github.com/scicloj/notespace",
 :lib/category :visual-tools,
 :tags #{:exp :lit :act},
 :star :star,
 :description
 "notebook experience with Clojure namespaces edited at any editor"}

I'm also not 100 % sure what :star: means in combination with :exp and :act.

Perhaps we can orient it closer to "call to actions" / "jobs to be done"? What do we want the reader to do when reading this document?

Is this a library the user is encouraged to use heavily? Is this a library the user is welcome to contribute to?

daslu commented 2 years ago

Thanks, @teodorlu, I will look more closely soon.

Very much open to changing the way things are presented, as long as we keep the links working (e.g., https://scicloj.github.io/docs/resources/libs/#geospatial-processing).

At some point, we may lose those links to subsections, but that requires more thought.

A :star: means that a library is actually worth using nowadays -- "known to be actively used and useful". We want to draw attention to those in a very clear way. Good idea to consider how this relates to other tags.

teodorlu commented 2 years ago

Very much open to changing the way things are presented, as long as we keep the links working (e.g., https://scicloj.github.io/docs/resources/libs/#geospatial-processing).

Sounds good!

Then I'll look into a more straightforward markdown presentation when I have time. I think we should be able to keep all the links working. I'd be happy to get some help testing this - to make sure I don't break anything.

I think when we have a nice markdown representation that we can generate from the data is a good point in time to merge.

teodorlu commented 2 years ago

At some point, we may lose those links to subsections, but that requires more thought.

Right now, I think our documentation system auto generates the links from the heading title. We might be able to control those links manually ourself too with plain html.

teodorlu commented 2 years ago

We can also consider <abbr> elements for tags, to explain what each tag means.

https://developer.mozilla.org/en-US/docs/Web/HTML/Element/abbr

daslu commented 2 years ago

Merged this, looks fantastic @teodorlu. :pray: