Rewrite listings generator

[lassik]

• Fix #45.
• Add extras field to manually list SRFIs we can't scrape.
• Change the generated shell scripts to use { ... }

[lassik]

@jpellegrini I looked at #45 and ended up rewriting the shell script template used by the generator in listings.scm.

That means all of the auto-generated shell scripts are changed in this PR, though Chibi is the only one that was materially affected.

This PR includes the output of running ./listings.sh, which caused changes to the Chibi, Chicken, Racket, and Unsyntax listings. Can you tell whether the changes are reasonable. In particular, a lot of Chicken SRFIs were added.

[jpellegrini]

@jpellegrini I looked at #45 and ended up rewriting the shell script template used by the generator in listings.scm.

That means all of the auto-generated shell scripts are changed in this PR, though Chibi is the only one that was materially affected.

This PR includes the output of running ./listings.sh, which caused changes to the Chibi, Chicken, Racket, and Unsyntax listings. Can you tell whether the changes are reasonable. In particular, a lot of Chicken SRFIs were added.

I'll take a look!

[lassik]

OK, now it should be fixed!

Racket is now at version 8.2 so I upgraded the scraper to use it.

Hi @lassik ! I think it's ok, but it seems that there are some SRFIs that are just not advertised by the implementations, although present. Like SRFI 7 on Unsyntax and Racket, adn SRFI 46 on Unsyntax. Maybe we need a place to list the SRFIs that are supported but not detectable by each implementation?

Right. Added extra for these in listings.scm.

PS: the terseness of the identifiers and their being used in sequence here reminds me of FORTH... (reversed of course)

(disp tab tab "gunzip |")

😄

Good point! It does look like Forth.

I've used (define (disp . xs) (for-each display xs) (newline)) in lots of scripts for about a year, it's trivial and experimentally it works well for lots of problems. The name disp reminds that is uses display internally.

[lassik]

New table now up at https://doc.scheme.org/srfi/support/

[jpellegrini]

New table now up at https://doc.scheme.org/srfi/support/

Great! :)

I have also included STklos' SRFI 15 entry in extras, and it generates it properly on my local machine, but it isn't showing up... As I remember there was no manual step needed here. Did anything go wrong?

[jpellegrini]

Oh, I see - there is a difference between http://htmlpreview.github.io/?https://github.com/schemedoc/srfi-metadata/blob/master/table.html and https://doc.scheme.org/srfi/support/

[lassik]

The table isn't pushed automatically to scheme.org at the moment. I have a manual rsync command to do it. Let me know if you want ssh access to the server.

[APIPLM]

Where is the 7 and 46 from in unsyntax.scm file? There is not files7.scm 46.scmin \unsyntax-v0.0.3.tar.gz\unsyntax-v0.0.3.tar\unsyntax-v0.0.3\src\ folder when downloading the unsyntax-v0.0.3.tar.gz file to check it.

[jpellegrini]

Where is the 7 and 46 from in unsyntax.scm file?

We have an "extra" field for these cases in the structure used to represent implementations in listings.scm at top level:

 (make-scheme "unsyntax" "gitlab"
              "nieper" "unsyntax"
              "master" "v0.0.3"
              "src/srfi/[0-9]+.s.?.?"
              #f
              '(0 7 46))      ;; <== here!

[jpellegrini]

at top level

I meant "top level" as the root of this (srfi-metadata) project:

https://github.com/schemedoc/srfi-metadata/blob/0c7a600f278bbe69cc8bc25330f0e08e80e1730b/listings.scm#L101-L106

[APIPLM]

Thanks. it is the commit a5a558d5bbe2c4a7bbc2e0aff80c09caef4bdc1f, Add extras field to manually list SRFIs. Can I ask that, what is the reason for that we can not scrape them? I thought the single data source for the srfi implementations is from the repo of the implementation. Of course, the first thing is that the table should have correct SRFI implement number of each of the implementations.

[lassik]

Some SRFIs are not normally implemented in their own library. Instead, the features from those SRFIs are integrated into the core of the Scheme implementation. For example, SRFIs 0 and 46 are included as part of R7RS, so all R7RS Schemes implicitly have them, even if they don't say it.

[APIPLM]

Okay. I mean that we push that further if we can. Such like that they can have a category of R7RS for them. And then the implementator can realize early those SRFIs are part of R7RS. Then collaborate to the owner of the R7RS to figure out if the extra information they need to follow the R7RS. That table we have to make that collaborations to happen, not hold somewhere. If we can, that we try to persuade the implementators to put somewhere in their data source. Or there is the process to run through, then this category information is available.

[lassik]

The list of SRFIs included in R7RS-small can be found by searching for srfi in the PDF. R7RS-large is being built entirely out of SRFIs.

Yes, we should probably include an R7RS flag in listings.scm, which means that certain SRFIs are implemented.

The master database for a lot of SRFI information is srfi-data.scm. Useful new information could be added there.

[jpellegrini]

It's not only the R7RS thing.

Sometimes SRFIs (not even related to RnRS) are implemented as part of the core/compiler/interpreter - either because it's more convenient for that implementor, or because it's the only way - some SRFIs require internal changes to the compiler. Then scraping doesn't work, unless the developer(s) make a list of supported SRFIs explicitly available.

I agree that it would be nice if implementations would advertise all the supported SRFIs, even withdrawn. I made this PR, for example: https://github.com/egallesio/STklos/pull/253 Also nice would be if more Schemes supported SRFI-176 (I also helped add it to STklos), with the list of supported SRFIs in the output.

But I also understand that we have to take into account the available time and priority of developers.

[lassik]

I propose writing SRFI lists in package.scm files. A package is meant to be a collection of Scheme libraries, but a Scheme implementation is quite similar (after all, it is a software package), and could use much of the same metadata.

So each Scheme implementation would have a package.scm file at the top of its source tree, stored in git and shipped in release tarballs. That file would list the SRFIs supported. I don't yet know if implementers want this, but Chicken, Gambit, and Chibi are on board with the package.scm file for libraries; Chibi already uses such files via Snow-Fort.org.

[lassik]

The implementation-metadata repo is basically a collection of community-maintained package.scm files for implementations that don't have them yet. In principle, most of srfi-metadata could be merged into implementation-metadata.

[APIPLM]

Yes. it is not only the R7RS. The way we break down the R7RS into the pieces of SRFIs. It is somehow a single point for the implementors to see that what abstract are needed for the features the platform supports. And also that is how the standard evolve. The master database will help to the process more smoothly with precise meta information.

[lassik]

Most of the stuff in R7RS-small comes from R5RS. Only a few things come from SRFIs.

R7RS-large is very different: all of the new stuff comes from SRFIs.

Some parts of R6RS were first drafted as SRFIs (try "Filter by keywords: R6RS Process" on the SRFI homepage) but most of R6RS was also designed separately. Building RnRS from SRFIs is a new thing.