Feature request -> org to emacs-lisp

[jkitchin]

I have several emacs-lisp libraries written in org-mode, which I tangle out. I would like to try using lentic instead, and keep them in .el form, but have the org-form available for the documentation. Is there an easy way to convert the org-file to a lentic-friendly .el file?

[phillord]

There are two transformations between org and emacs-lisp. The first is relatively simple -- it will take an org file which has already got emacs-lisp sections inside and convert it into an valid emacs-lisp file (just by commenting out everything which is not lisp). The second converts what I call "orgel" (not convinced this is a good name, but there you have it). This is an emacs-lisp file with a few minor adjustments to the conventions (so a "Header:" tag, and begin_src tags) and all the documentation in org mode. This was designed exactly for making a literate el file (with org mode).

The lentic source is an example of this.

Is this what you are after?

[jkitchin]

That sounds like what I want, but I was looking for the specific M-x lentic-transform-org-to-elisp command. Is there something that simple to do what you described? I hacked something that got me a lot of the way there, but there are lots of corner cases I did not consider, e.g. empty lines, and I thought there must be a better way.

[phillord]

Well, in theory you should just be able to set the buffer-local lentic-init to "lentic-org-el-init", then create the lentic buffer and away you go.

In practice, it's not that simple yet. I gave it a go with org-show.org and it fails for two reasons. First, my "begin_src" regexp balks on the :tangle marking. That's easy enough to fix, but having hacked that, it still fails because you've got multiple languages in there. Currently I identify the end of emacs-lisp code blocks by looking for "end_src" but that's not true in your case because you have other languages in there too.

It's fixable (my asciidoc support has a similar problem -- regexpable openers, but not closers), but not trivial. Can you send me some sample files, so I can see if there are any other problems I haven't thought of?

[jkitchin]

The other problem I can see is a file like: https://github.com/jkitchin/org-ref/blob/master/doi-utils.org, or the org-ref.org code that is also there. In these files org can selectively tangle some blocks (all emacs-lisp this time) and not others. That allows me to integrate tests and example usage in the file. I can live without that, and also without multiple different types of code blocks in org-show. That is a pretty special application of literate programming! I really want to try moving org-ref into a lentic style because it would make it easier for others, but I REALLY love the org organization of this file. I will give your suggestion a try.

[tumashu]

org-mode + tangle is too slow, my 2000 lines emace lisp need 4min to tangle ....

[phillord]

@jkitchin So, selective untangling can be supported I think. Lentic just needs to take account of the :tangle argument and only uncomment the relevant src tags. I've just started to add support for multiple lenticular views, so it should even be possible to have one view of the org file, one of the org-ref source and one of the examples or tests.

I'll have a go with org-ref.org and see whether I can get a nice transformation. But, I need to change my block matcher which currently looks for pairs of begin_src/end_src tags.

[phillord]

@tumashu I haven't used org-mode untangle on a file of that size myself. Lentic though does work on 2000 line files with acceptable performance.

[jkitchin]

That is an interesting idea. would blocks that are not tangled out be commented in the elisp source? One advantage of that is the testing/examples can still be tested in org-mode then, even with other languages, but not impact the elisp side. This is presently an issue with tangling for me, which is why I am looking to move towards a lentic view. I find if I don't set the tangle args correctly on each block, i.e. to explicitly tangle or not, then the output is not reliable.

@tumashu I have tangled files that are thousands of line of org-mode with no issue. 4 min sounds like something else is going on to me.

[phillord]

@jkitchin So I can see a number of arrangements that might work. You might, for instance have three views like so:

org-ref.org ---> org-ref.el
            ----> org-ref-test.el

Now these would all have the same data. The org view would look like this:

* Section

Here I discuss my interesting function

#+begin_src emacs-lisp
(defun my-interesting-function())
#+end_src

#+begin_src emacs-lisp-test
(ert-deftest my-interesting-function-test())
#+end_src

The el view would look like this:

;; * Section

;; Here I discuss my interesting function

;; #+begin_src emacs-lisp
(defun my-interesting-function())
;; #+end_src

;; #+begin_src emacs-lisp-test
;; (ert-deftest my-interesting-function-test())
;; #+end_src

and the test-el view like this:

;; * Section

;; Here I discuss my interesting function

;; #+begin_src emacs-lisp
;; (defun my-interesting-function())
;; #+end_src

;; #+begin_src emacs-lisp-test
(ert-deftest my-interesting-function-test())
;; #+end_src

As well as views, Emacs would associate these with files, and you could generate them in batch, so testing would work on a CI server if you choose. I've done this some of this with https://github.com/phillord/take-wing, but I really would like the multiple views so I can hold examples (rather than tests) in the same place. The tangle functionality of org would still be useful if you wanted to produce a "minimal source" without the documentation, although to be honest why bother? It's not like the documentation is going to fill your hard drive.

Now, the multiple view support is, as I say, very new (i.e. the org-mode support went into day) and so is a bit hairy at the moment. I think it should be performant enough, but you can never tell for sure till you try. And as I have said, the current implementation of lentic-org does not allow this and would require some recoding (the latex implementation would work with just configuration because latex has explicit start and end markers).

Lentic and the general idea of lenticular views is relatively new, so whether any of this would be a good thing to do or not, I just don't know. But, it's close enough to working, so we can try.

BTW, I've been showing your "python, org and reproducible science" video to my students for the last few years. Great fun.

[jkitchin]

That sounds really interesting! I am going to need a bigger monitor ;)

I am glad you like the video. I have a youtube channel now of similar things. https://www.youtube.com/channel/UCQp2VLAOlvq142YN3JO3y8w. I especially like the org is awesome and the org-ref videos.

Phil Lord writes:

@jkitchin So I can see a number of arrangements that might work. You might, for instance have three views like so:

org-ref.org ---> org-ref.el
            ----> org-ref-test.el

Now these would all have the same data. The org view would look like this:

* Section

Here I discuss my interesting function

#+begin_src emacs-lisp
(defun my-interesting-function())
#+end_src

#+begin_src emacs-lisp-test
(ert-deftest my-interesting-function-test())
#+end_src

The el view would look like this:

;; * Section

;; Here I discuss my interesting function

;; #+begin_src emacs-lisp
(defun my-interesting-function())
;; #+end_src

;; #+begin_src emacs-lisp-test
;; (ert-deftest my-interesting-function-test())
;; #+end_src

and the test-el view like this:

;; * Section

;; Here I discuss my interesting function

;; #+begin_src emacs-lisp
;; (defun my-interesting-function())
;; #+end_src

;; #+begin_src emacs-lisp-test
(ert-deftest my-interesting-function-test())
;; #+end_src

As well as views, Emacs would associate these with files, and you could generate them in batch, so testing would work on a CI server if you choose. I've done this some of this with https://github.com/phillord/take-wing, but I really would like the multiple views so I can hold examples (rather than tests) in the same place. The tangle functionality of org would still be useful if you wanted to produce a "minimal source" without the documentation, although to be honest why bother? It's not like the documentation is going to fill your hard drive.

Now, the multiple view support is, as I say, very new (i.e. the org-mode support went into day) and so is a bit hairy at the moment. I think it should be performant enough, but you can never tell for sure till you try. And as I have said, the current implementation of lentic-org does not allow this and would require some recoding (the latex implementation would work with just configuration because latex has explicit start and end markers).

Lentic and the general idea of lenticular views is relatively new, so whether any of this would be a good thing to do or not, I just don't know. But, it's close enough to working, so we can try.

BTW, I've been showing your "python, org and reproducible science" video to my students for the last few years. Great fun.

---

Reply to this email directly or view it on GitHub: https://github.com/phillord/lentic/issues/11#issuecomment-74567697

Professor John Kitchin Doherty Hall A207F Department of Chemical Engineering Carnegie Mellon University Pittsburgh, PA 15213 412-268-7803 @johnkitchin http://kitchingroup.cheme.cmu.edu

[phillord]

I started off mostly using multiple windows with each lentic buffer open. That was what I thought would be the best view. But to be honest, I have found in practice, a single view is enough. I just cycle between them with lentic-mode-move-lentic-window which I bind to F1. When the mulitple views are fully working this should cycle through each view in turn. As you say, you need a big monitor for multiple windows, which is fine on my desktop but fails on a laptop. I'm going to work on the windowing features for the next couple of weeks, I think, then release a new version.

Nice videos -- haven't done a class in org mode yet. But I might try.

[phillord]

lentic supports multiple kinds of source blocks now, so most of what we discussed in here should be possible now!

[jkitchin]

Thanks for the update! I look forward to trying it out.

John

---

Professor John Kitchin Doherty Hall A207F Department of Chemical Engineering Carnegie Mellon University Pittsburgh, PA 15213 412-268-7803 @johnkitchin http://kitchingroup.cheme.cmu.edu

On Mon, May 11, 2015 at 12:34 PM, Phil Lord notifications@github.com wrote:

lentic supports multiple kinds of source blocks now, so most of what we discussed in here should be possible now!

— Reply to this email directly or view it on GitHub https://github.com/phillord/lentic/issues/11#issuecomment-100973030.

[phillord]

Just tidying up!