Automatically generate packages page from manifest

[karthik]

Jeroen said he can figure this out

[jeroen]

Implemented in https://github.com/ropensci/roweb2/commit/811f4a7e92ce9d307f9223ca55f4dd3deeaf250d.

We still need to improve this because currently roregistery only contains plain text descriptions. We probably want to be able to insert hyperlinks or badges here.

[jeroen]

Added some more tweaks and added support for markdown syntax inside the description field.

[cboettig]

@jeroen Wow, you're fast @jeroen . Meant to give you a heads up that we (@sckott & I, though may enlist some interns to improve the metadata) are hoping to transition the registry itself to something that would be easier to update; currently that json file is curated by hand.

The plan is to automatically generate codemeta descriptions from the package repo using codemetar (primarily DESCRIPTION file info, but also scrapes stuff like badges) for each package, which creates a registry that looks like this: https://github.com/codemeta/codemetar/blob/master/inst/notebook/ropensci.json (which is just generated automatically from ropensci github repos)

One thing we still need to add to this is the categories and keywords; ideally these will be selected and maintained by package maintainers in their DESCRIPTIONs long term.

Thoughts / critique / feedback?

[maelle]

A few months later 😉 The currently produced big codemeta.json of all packages has keywords, derived from GitHub repo topics (and from keywords in DESCRIPTION for the only package that has them :grin: ).

• I need to add more GitHub repo topics over both organizations, and also more status badges.

• I'll soon write code to actually process the big codemeta.json into a table. That table will have status badges.

• The codemeta creating and processing should run automatically, currently in the future.

• The packages page should have search across package names, description and keywords. I'll try to think about this too.

[maelle]

Also noting https://github.com/ropensci/roweb2/issues/56

[jeroen]

We should discuss in the meeting what the plan is exactly. Should this replace @sckott https://github.com/ropensci/roregistry?

[sckott]

Yes we should discuss. Yes, the plan is to generate packages page from this metadata, but it shouldn't be too different from how you have it setup now in that we'll run a cron job to generate the single json file and you can pull that in as you do know

[jeroen]

Meeting update: once maelle has implemented the code that adds the keywords to the registery, we want to show those in the front-end table.

[maelle]

Another note, I'd like to auto-generate the different filters https://github.com/ropensci/roweb2/blob/master/content/packages.html#L31 based on the categories present in the registry csv (this way if we update categories, no need to update the website table code).

[maelle]

There's also some "manual" category grouping at the moment https://github.com/ropensci/roweb2/blob/3d3c0f0c598459f0e5ca5b1bafbd1c3edcb58bde/themes/ropensci/static/js/tableinit.js#L35 (not criticism, just a note I'm making)

[maelle]

I wonder why the BIOC button (e.g. treeio) is blue when the package is available on BioConductor (as opposed to green for the CRAN button)

[maelle]

In the current table on the site the maintainers names look like hyperlinks but they aren't. I wonder whether there should be... a mailto:? A view filtered by maintainer?

[karthik]

Ideally it should go to their GitHub profile page.

[karthik]

or actually i prefer a filtered view by maintainer

[maelle]

good idea by @noamross, the table should also show which packages have been reviewed.

[maelle]

or actually i prefer a filtered view by maintainer

I had missed that it's already implemented actually!

[maelle]

Current state, script to run on the server with the clones https://github.com/ropenscilabs/makeregistry#example Output is both a raw Codemeta file, and the JSON registry used as data source for the packages page.

Not run automatically yet, @sckott to explore that.

@jeroen codemetar will extract all URLs from DESCRIPTION. One of them is recognized as CodeRepository, the others (e.g. pkgdown website link) get stored as relatedLink cf this example

[sckott]

cron job is running now

I still need to sort out adding new repos as they are added to ropensci/ropenscilabs

[maelle]

Yay!

So there is no API call to GitHub before cloning at the moment?

[sckott]

not yet

[sckott]

i think this is all set now @maelle

[maelle]

or at least we now track remaining issues elsewhere 😁 https://github.com/ropenscilabs/makeregistry