Add contributions guideline

[inaimathi]

There's been enough interest in making contributions to this project that I think I finally need to bite the bullet and write up a formal process for how to do so. I think this can start as a section in README.md.

[silent-entertainment]

Perhaps I jumped the gun when i duplicated and rebranded cl-notebook.

I fetched cl-notebook for the bug fix and looked at it again tonight. First slime went into the debugger when I tried to pull up charts from the link, because the path is absolute (/home/inaimathi/.cl-notebook/books/charts). I opened charts directly to look at it.

I was trying to see how you pulled the chart into _notebook Duh! I see it now. It was just too late to mess with it.

After getting lost in other ideas and other code for a few weeks... I think I might fork cl-notebook and try to add some features regarding the UI and then make it a dependency of claxiom, in which I'm going to play with some ideas.

[silent-entertainment]

Regarding the UI, I have a few ideas. Though it might just be personal preferences, I've been using computers for about 40 years, and I've noticed a few things about user interfaces. Certain things are convenient, and other things can be very distracting. There are just certain things that I would change for myself

For example, I do like your context menus at the current cell. Having it adjacent to where you're working is convenient. However, I only want it there when I need it, because when I'm writing code it doesn't need to be between the last thought in the previous cell and the current thought at the current cell. In fact, it's nice to know that it's there if I need it, but after I learn the keyboard shortcuts from the menus-hint, hint-I might only need to refer to it occasionally for something that I haven't used for several days. I want to keep my hands on the keyboard, and the keyboard shortcuts need to be somewhere other than the source code or the home page.

How often am I going to Open a notebook or Export? It's nice when a UI is obvious, but those three buttons take up a lot of screen real estate just below where I'm focused, and menus (with hints for keyboard shortcuts) and toolbar icons are standards.

I don't mean to nitpick. These are just things that I would adjust to make it easier for me to focus.

Otherwise, you did a fabulous job on it. Your code is very clean. Even though I hadn't even looked at CL code in a decade, most of your code in cl-notebook is easy for me to understand.

Of course, I'm not sure it would make sense to hack the DOM from a notebook, because that sounds like too much work, compared to editing the page layout at the source in core.lisp? Though I see that the DOM elements are named.

Edit: Looking more closely at core.lisp, I'm seeing what looks like a hook to manipulate the DOM fairly easily, though I might need a refresher to understand how or if it would be feasible to inject a script or a hander in the right places.

[inaimathi]

Nit picks are welcome (though these particular ones should perhaps be in a separate "UI Improvements" issue). In no particular order:

• One of the high-level goals of cl-notebook is to be a credible replacement (or at least precursor) to Emacs as a Common Lisp development platform. Which means a high degree of usability in the humane sense. This includes being primarily keyboard-oriented (at the moment, IIRC, every button you see on screen can be triggered by clicking on it, or by a keybinding. There are also default keybindings for navigation inside of/between cells, and for cell transposition)
• I intend to make the end result highly customizable. I haven't given much through to how I might make it easier to customize the layout components (I vaguely assumed that if I gave things CSS classes at a granular enough level, custom style sheets might solve this problem, but that might not be enough). I have given thought to how customization of keybindings might happen, but haven't gotten around to implementing it. This is going to involve some refactoring of a bunch of things in src/http-front-end, because I want there to be at least two layers of customization; firstly, a set of keybindings at the server level (stored on disk/in DB/in a notebook somewhere) and secondly, a set of bindings at the client level (stored in LocalStorage). The idea would be to let the client bindings take precedence over the server bindings

[silent-entertainment]

Oops! You already revealed how. It was late last night when I looked at it!

[silent-entertainment]

Thanks for the link. I downloaded it and intend to read it immediately.

I see all of the key bindings in core.lisp, but I missed it or forgot about it after the first couple of times through it. I don't recall seeing anything in the UI that provides clues. Looking again at core.lisp this morning, I realized for the first time that there is a history slider. Whereas cl-notebook is an amazing piece of work, I just feel lost in the UI.

Emacs is a PITA. I haven't been able to figure out how to evaluate a form in the buffer from my chromebook which is booted into Ubuntu, even though the key binding is clearly in the menu. I'm ready to throw out the chromebook after inadverently and repeatedly pressing the power button and shutting it down because the power button is adjacent to the backspace key! What a terrible design for a UI! Now I'll have to dig to find out if I can disable it or buy another laptop.

Recently reading a lot of Norvig, Seibel, et al, I have repeatedly read that one of the greatest features of CL is how much information you can compress into a single screen, when you use the language appropriately, and I keep seeing the contrast when looking at Jupyter's python, Terrence Parr's ANTLR, etc. You may have seen my rants about Jupyter's python code. It looks like they unrolled all of the loops, possibly because python is so friggin' slow. (See Norvig's Python for Lispers.) Of course, Java is so verbose, and its syntax is so contorted, that it only complicates the fact that to grok ANTLR you need to be familiar with four different languages (Java, StringTemplates, ANTLR AND the target language) to do anything useful with it, even though it can produce Lisp-like tress (which are broken with how it formats single-character tokens in the concrete syntax tree, making reading it in Lisp more difficult because you'd have to alter the reader). I can understand possible reasons that Parr might have had for choosing Java to develop ANTLR, but IMO they were the wrong reasons, because it's so academic (in the sense of impractical) that it's a wonder that it's so widely used.

I have long had an interest in language parsers, so I looked at esrap, which is in CL. Even esrap and its offshoots have missed what Norvig, Graham, Hoyte, et al, revealed about CL decades ago in PAIP, Let Over Lambda, etc.

The point of my rant here is that to be truly useful the UI has to allow you to put as much information on a screen as possible without any barriers between the blocks of information.

[silent-entertainment]

It has been a very long time since I have looked at CSS in depth, but from what I recall and from recent tinkering in javascript, now that I understand more about the facility for customizing cl-notebook through a notebook, laying out elements through DOM manipulation should be simple enough with an adequate API.

[silent-entertainment]

I realize that I went off topic.

By the way, one copy of Humane Interfaces that I found has the entire text of the book center-justified. I think it had to be a joke, because I can't see how anyone doesn't realize how difficult it is to read!

Here's a simple idea for improving the UI, while still leaving the entire screen clear for what is really important. Long before Windows came onto the scene, the standard key binding for Help was F1. In some of the old text-based applications, there often was a list of key bindings in the bottom two rows of the screen, but that takes up too much screen real estate unless you can disappear it, which I seem to recall being a feature of one of those older programs.

When graphical windowing interfaces and book-length help pages came around, the F1 key was still available, and in the corner of the application window is the mouse-clickable question mark icon. Usually when you hover the mouse over the question mark, you see a contextual hint, "Help - F1". The question mark draws attention to available help without cluttering the screen with menus and toolbars, and the contextual hint reminds you that you don't have to move your hands from the keyboard to get there. Likewise, in the other corner there was the icon for drop-down menu for the window commands, maximize, minimize and close and sometimes including help, in place of a seperate button, and all of the menu items listed key bindings.

In the same vein, menu systems are often available with alt-key combinations, but for some reason the menu is always there cluttering up the screen and encouraging you to point and click, which is much less efficient.

Nowadays, in the browser web pages often have the menu icon in the title bar at the top of the web page, but for the most part we've lost the contextual hints to any key bindings for navigating actions. We've all been trained to point and click, to be less efficient, to be good little button pushers.

Forget on-screen menus and icon toolbars. They have little to do with focusing on writing code or prose. The menus don't need to be visible unless an action is required. All that is really needed is the corner visual and the contextual hint that it's available with a key binding, and the visual should be near the current focus.

If a cell is in focus, the question mark (or preferably something less distracting, maybe similar to the minimize icon or the currently popular menu icon) in the upper-right corner of the cell provides the hint for where to find the contextual key-bindings. Press F1 or some other meaningful key and up pops the contextual commands and key bindings near the current focus. With use (practice) the key bindings become habitual, removing a step from the process.

If no cell is in focus, move the visual to the notebook scope.

There could also be the option to turn off the icon, which leaves the entire screen uncluttered and available for what really matters.

[silent-entertainment]

Hey Leo,

I haven't forgotten about the project.

Last week, I bought a new PC with a 23" monitor and a real keyboard, so that I can see what's on the screen more easily with my 50yo eyes, so that I can work on this thing. I just got it booted into Linux this morning, because I hadn't had a chance to play with it previously.

And then the day before yesterday, a tree jumped out in front of me and picked a fight with my truck. The tree won.

I'm still familiarizing myself with your code, and I'll see about what I'd like to accomplish first. I'll probably start with customizing the UI to suit my preferences better, and then perhaps I'll tackle some of your issues.

Several weeks ago, I read your blog and know that you have other priorities. I have a lot of free time, so maybe I can help you with some things on this very cool application.

Jack

[silent-entertainment]

@inaimathi

I'm still working on it.

I was having so much difficulty with Lubuntu that I decided to switch to Gentoo, with which I have a lot of prior experience from several years ago. Installation has been a nightmare, because I'm trying to do a non-standard installation and because it has been several years since I've configured and compiled a Linux kernel.

The installation is almost done. I have Gentoo booting from a full installation on a USB flash drive, with persistence, and now I'm working on installing the desktop environment.

I wanted to run a few ideas by you and get your thoughts.

What I'd like to do is to hijack JupyterLab's interface and maybe use cl-notepad as a back end. JupyterLab is very versatile, and the interface is very functional. However, I can't appreciate certain aspects of the design.

The cellular structure of a notebook seems to serve only to define document structure and to allow tracking of changes to individual sections of the notebook document, but the manner in which it has been structured makes the interface clumsier than it needs to be.

Think of a notebook as a single structured document. For example, in markdown a document can be structured by inserting elements. HTML5 gives us new HTML tags to better structure documents. Let's explore it.

\<article class="weaver" name="carpal-chain"> ## Carpal Chain

A carpal chain (carpal-chain) is a kinematic chain (kinematic-chain) with a basal revolute joint (basal-joint), a basal link (basal-link), a ball joint (mid-joint) a distal link (distal-link) and a distal revolute joint (distal-joint). \</article>

\@aside

Annotations can be used to make it easier to read and write.

\@aside end

\@weaver carpal-chain

A carpal chain (carpal-chain) is a kinematic chain (kinematic-chain) with a basal revolute joint (basal-joint), a basal link (basal-link), a ball joint (mid-joint) a distal link (distal-link) and a distal revolute joint (distal-joint).

\@weaver end

\@cleaver carpal-chain

\@aside

The @cleaver translates the named object into code, such as follows:

(defclass carpal-chain (kinematic-chain)
  ((basal-joint :initarg :basal-joint)
   ...))

I'm seeing cells embedded in the document. Let the application take care of it. Now, adding a code cell looks like the following:

\@aside end

\@result carpal-chain

And the @result takes care of rendering the output from running the code.

Now, documentation and code are one, so they'll always be synchronized, and the documentation can be enfolded into the application for your online help and widgets that provide functionality to the user interface. Furthermore, it could be implemented to be bidirectional, so that you can write in either code or prose and get the other. Of course, we'll have to abstract certain parts of cl-notepad, which you've considered in some of your issue notes.

What are your thoughts on adapting cl-notepad in such a way?

[silent-entertainment]

Another thought...

I've been looking at jscl and the possibility of writing the interface in plain old common lisp.

jscl provides an adaptation of closette, which means that clos-style classes and methods are available. parenscript is great for what it does, but jscl elevates it to a new level.

Frankly, I don't see why someone hasn't figured out how to plug ECL into Mozilla Firefox or other browser- at least i haven't found such a project, but I only did a quick search. It was probably a lot simpler and versatile to compile to JavaScript.

[silent-entertainment]

With the right abstractions, the application could be plugged into any suitable front end, back end or framework.

[silent-entertainment]

Oh! Take a look at tui.editor (ToastUI). It gives you a full markdown editor, raw or wysiwyg, and it uses CodeMirror for editing code blocks. You can synchronize multiple views of the same text, such as markdown, wysiwyg and viewer. You'd never have to look at raw markdown, but it's available when you want or need it.

[silent-entertainment]

Use JupyterLab to bootstrap the entire application from a Jupyter notebook. Hijack the JupyterLab user interface to do as we please with it in common lisp (jscl). The notebook becomes a markdown document with embedded cells, for which the JupyterLab API gives us all sorts of functionality that can be adapted to our user interface, Jupyter NoteBook and cl-notebook give us a choice of the back ends for our cell and notebook histories. And Jupyter Notebook can export a notebook to markdown.

The user is relieved from structuring the document through the UI by using simple annotations, from dealing with the mouse or from learning a bunch of keystrokes for creating and manipulating cells, and from manipulating cell types through the UI, and the annotations are visible, simple documentation of what is intended that can be viewed in any convenient form by giving it to a widget. The annotations are whatever we need them to be. Annotations provide a simple notation for defining contexts, which could be nested, to serve as structured directives to the interface or the server.

I realize that much of it is possible to build into cl-notebook. We can do that over time and divorce JupyterLab, but imagine the work of turning cl-notebook into something more like JupyterLab, with its nice MDI interface, plugins for different document types, its transformation of the browser into an integrated development environment workspace, which you can hide by taking the browser fullscreen. That's f'ing cool.

[silent-entertainment]

I imagine a lot of this might sound like opinionated crazy talk, but ever since I first looked at a computer about 40 years ago, I've been quite dismayed that no matter how far along the learning curve you are or how good your code is you're effectively working for the computer when the computer should be working for you.

Of course, people have different preferences and opinions about how a workspace should be organized and how a tool should work. Some think that natural language is too ambiguous to be practical for instructing a computer. However, the domain is limited, and the problem isn't intractable. A big priority is to make the logic accessible without imposing a burden on the programmer. There is absolutely no reason that documenting code to reveal the function of part of an application in isolation without looking at other parts.

Currently, the documentation of many libraries only enumerate the signatures of API components, without providing any clue about the role or context of the operators other than the concepts that the chosen labels convey. There just isn't enough information in the signature to convey the correct usage of the operator without referring to the implementation of every connected symbol.

Of course, skill in any art requires discipline, learning and practice, but modern computers are so computationally powerful that it's about time that we stop being slaves to the machine and reduce the effort that is required to implement a desired task, to tell the machine what is required of it to accomplish a desired result.

How many hours do we spend learning to understand and communicate our native natural language? K thru 12 we're corralled into classrooms. If we choose to pursue a degree, that's another two, four, six, eight... years of formal practice in college or university. We practice communicating in the language. Then to get a computer to do anything novel we decide that it's necessary to invent a different and strange language to communicate intent? It's absurd! It's inhumane. It's impractical.

There are enough available open-sourced libraries available that can be utilized to bridge the gap, to transform ideas into procedures that a computer can execute.

Let's stop being slaves to the machines, because we are their masters.

[silent-entertainment]

@iaimathi,

My Gentoo box is ready to go!

I was looking at various client-server browser applications for ideas, and I came across next-notebook, which wraps the Jupyter Notebook API, so that it can be used to interact with Jupyter Notebook from Next (Nyxt?) Browser. By the way, Nyxt looks neat, and I'll have experiment with it.

I was looking at Nyxt and ceramic/remote-js, when it clicked. I just hadn't pictured it in such a way yet, but it should have been obvious to me, because I knew that such a thing was available-there's a name for it, but I can't think of it. If the application can control the browser by sending arbitrary code (RPC) to the browser, the only JavaScript that needs to be written or compiled (parenscript, jscl, etc.) is for the procedure calls, which can be wrapped in abstractions, and the entire application can be written in one language, plain common lisp.

This solves a major hurdle. I haven't found an ideal Javascript compiler. In one of his papers about JACL , the author points out how the available systems stray from the ideal, and he was very thorough. parenscript hasn't kept up with the standard, and code ends up littered with parenscript macros, which I don't even completely grasp. There's a mismatch due to the differences between common lisp and javascript, so you're still forced to write javascript. Projects that have tried to put common lisp in the browser also stray from the ideal. As the author of JACL points out, jscl has a design flaw that makes it unnecessarily slow, and the JACL design addresses the issue. However, JACL's console is built on Chrome DevTools, and I don't know what's involved in making it more portable. Furthermore, the JACL code is over 3,000 lines of JavaScript, and I don't even want to look at it. I understand JavaScript, but I don't want to read it.

Jupyter already has the framework. The common-lisp-jupyter kernel appears to be functional. JupyterLab already has the API for plugging in adapters for additional document types, so it's ready to go. All that is necessary is the journal (history), for which there are ipynb and cl-notebook (fact-base). I prefer fact-base, because ipynb's use of JSON for the implementation just doesn't sit right with me. Other parts of cl-notebook might fit the big picture, too.

Anyway, I think this is the plan.