Provide org-mode files as optional for the users

[abarocio80]

For example "rational-ui.org" for "rational-ui.el".

The process of generating the org-file from the emacs-lisp sources could be automated with a simlpe script. May be something like:

#!/bin/sh

title="$(basename $1 .el)"

printf "#+title: %s\n" "$title"
printf "#+property: header-args:emacs-lisp: :tangle (expand-file-name \"%s\" rational-config-path) :mkdirp yes :comments none :results output silent :noweb yes :shebang \";;; %s -*- lexical-bindings:t -*-\"" "$1" "$1"
printf "\n\n#+begin_src emacs-lisp\n;;; %s -*- lexical-binding:t -*-\n"
sed -e "s/^;;;\s*\(.*\)\s*$/\n;; * \1/" "$1"| sed -e "/^\s*$/d" | sed "s/^\s*;; \(.*\)\s*$/#+end_src\n\n\1\n\n#+begin_src emacs-lisp/"
printf "#+end_src\n"

This could be triggered on push to the gh repo (like with the site generation).

[daviwil]

It would definitely be nice to have Org files (or a site) generated from the config files to be used as a reference. We could potentially generate those in a GitHub Action and push it to a GitHub Pages site. What do you think?

[abarocio80]

I'm still thinking on how to manage the comments. With the script I have done, all lines starting in comments (;;) will be exported as paragraphs surrounded with code blocks (even with empty blocks), including those comments not intended to be exported.

But, YES!, I want that functionality.

The problems:

• How to parse the source file, specially the parts we don't want in an org-file (or a site -- e.g. the license).
• Manage multiline comments correctly.
• How to manage comments inside the code.
• How to implement this: shell script or emacs-lisp? (Is ruby/python/etc forbiden?)
• Where the org-files will be stored (project root or other directory?

[ajxn]

Why not just import the file in a org-mode document as they are? And look at those problems later.
Shellscript should work, I think. Emacs I don't know, but it would have been nice to have HTML documents from org files.

But I am just guessing right now. :-)

(I want to have those document referenced from README.org :-) )

[abarocio80]

OK. Maybe, use empty lines as markers for code blocks? And everything else is code.

That will leave us yet with the problem to document the code in the org file. Reading a prity formated code is not my ideal of HTML documentation, even less, of a site page.

[ajxn]

I thought about a suggestion that we should have something like README-modulename.org for every module, or something like that. Then one might want a place-holder in the file where the source should be. Then one can write prose around the source code, and even make references into the code. Or even just import the elisp file into the Org-mode file when generating documentation (like HTML and ODT).

Just some thoughts.

43 and #52 has some relevant discussion.

[abarocio80]

OK. Importing the code (fragment by fragment) into the org-file is a good idea, as long as we mantain the org-file along the source code. The idea is to document the code, not to generate it (I don't think literate-programming is a good idea for this project). And to document it from the source code, so we don't have to keep track on the differences between documentation and implementation. What I mean: is easier to document the source code than to mantain documentation on separated files. For the latter, I would prefer to have a wiki (which I don't like).

[abarocio80]

OK. I have this script to convert the files. It works well as far as I can tell:

#!/bin/awk

BEGIN {
    code=0;
    FS=";;+ "
    getline
    while ((start = index($0, ".el")) !=0) {
        out = substr($0, 1, start -1)
        $0 = out
    }
    printf("#+title: %s\n\n", $2)
    while($0 !~ /^;;;? +Commentary/) { getline }
    printf("* %s\n", $2)
}

{
    if( $0 ~ /^;; /) {
        if (code==1) {
            printf("#+end_src\n\n")
        }
        printf("%s\n", $2)
        code=0;
    } else {
        if ($0 ~ /^;;; / ) {
            if (code==1) {
                printf("#+end_src\n\n")
            }
            printf("* %s", $2)
            code=0
        } else {
            if ($0 ~ /^;;;; / ) {
                if (code==1) {
                    printf("#+end_src\n\n")
                }
                printf("** %s", $2)
                code=0
            } else {
                if ($0 ~ /^;;;;; / ) {
                    if (code==1) {
                        printf("#+end_src\n\n")
                    }
                    printf("*** %s", $2)
                    code=0
                } else {
                    if ( $0 ~ /^\s*$/ && code==0) {
                        printf("\n")
                    } else {
                        if (code==0) {
                            printf("\n#+begin_src emacs-lisp\n\n")
                        }
                        printf("%s\n", $0)
                        code=1
                    }
                }
            }
        }
    }
}

END {
    # Cerramos el último bloque de código
    if (comment==1) {
        printf("#+end_src\n\n")
    }
}

Is this a good idea? Where should this script live?

[bkaestner]

Is this a good idea? Where should this script live?

Given that this is only usable on Linux with awk installed, I wonder whether it should be written within Emacs Lisp instead.

[bkaestner]

I prepared a first draft for a detangle-file function. While I'd like to publish it soon, I'm not sure whether I have time in the coming days to polish it enough, so feel free to play around with the gist :)

[abarocio80]

Is this a good idea? Where should this script live?

Given that this is only usable on Linux with awk installed, I wonder whether it should be written within Emacs Lisp instead.

Well, the idea is that this will become a Github Action, where we can control the environment and ensure there is a linux system (even an emacs one). If we translate this to emacs-lisp, we will need a tiny extra step (installing emacs) on each push. If we then convert the org-files to generate a web site, then emacs is in deed needed.

But again, the idea is not necesarily to have this functionality as part of the day to day rational-emacs "normal" user.

[abarocio80]

I prepared a first draft for a detangle-file function. While I'd like to publish it soon, I'm not sure whether I have time in the coming days to polish it enough, so feel free to play around with the gist :)

The script looks nice (good job with your emacs-lisp-fu), but I'm not sure about breaking lexical units (e.g. defuns) in multiple codeblocks. What do you think abot this?

[bkaestner]

but I'm not sure about breaking lexical units (e.g. defuns) in multiple codeblocks.

The script could use (forward-sexp) instead of (forward-line) on a non-comment line. However, that will also skip (use-package ...) forms.

[abarocio80]

About update suggestion

Mh, I see. Is there a similar command that works on macros too? (Edit: forward-sexp works also with use-package, and all balanced parentesis).

Maybe forward-thing? (forward-thing 'list) works with all lists, including code. (Edit: I think forward-sexp works good enough, and is more compact, prone to less errors.)

About the functionallity in general

I'm liking this option even more than my aw[ful/k] script. I'm still thinking on implement this only as a Github Action, but it would be nice to have this available to the end user.

About implementation

So, do we implement this as a module, within its own package, or as part of the proposed (#59) rational-core?

[ajxn]

Isn't generating HTML etc up to the user, if they want to have that?

If anything, a Info-book in Info format that is easy accessible from within Emacs would be nice (like with C-h i mRational Emacs). Probably built on README.org.
But to have some documentation of modules like this I guess that a hack/prototype to get example would be more useful then a final solution so far. But I have been wrong before. :-)

[abarocio80]

Well, the idea is not to have that documentation as part of the repo, but to generate a site for documenting the project. Otherwise, the users could use this very hack to generate their own config documentation if they want that.

[ajxn]

Ok, so what are we waiting for? :-) Create a first shoot at this and then make changes when there are something to look at.

Also remember that we have the possibility to run script in Babel when we export the Org-mode file to HTML, so this script can run and generate the contents in the Org-mode file. Possible generating HTML or Org-mode code that is shown when the page is rendered/exported from Emacs. And yes, Emacs can run and execute some Elisp code from a file to run this.

[telenieko]

Newbie here, what are the benefits of this proposal?

I mean, the .el files are not that long; simple and well commented, there also appears to be some in progress documentation in a docs folder (texinfo). So, what would be the benefit of shipping .org versions of the .el files?

Though I can think of many issues, for one: discrepancies between .el and .org at the commit level; unless handled by a pre-commit hook (client dependant) the .org conversion would live on a separate commit (could be fixed on merge with squash).

Maybe what could be more interesting is to script extracting the variables and customisations defined in the .el files and put them into the docs?

[ajxn]

Should we close this, or is it any more work on this issue?