silent-entertainment / claxiom

A notebook-style in-browser editor for Common Lisp, the resurrection of cl-notebook with a new name and a new face.
GNU Affero General Public License v3.0
6 stars 0 forks source link

Migrate system and package from cl-notebook to claxiom #5

Closed silent-entertainment closed 3 years ago

silent-entertainment commented 3 years ago

As I mention in the installation note re ql:quickload, in the project README the system and package will be migrated from cl-notebook to claxium, to eliminate conflict with cl-notebook, in case anyone has or wants both systems, since claxiom is distinct from cl-notebook.

I'm placing this task at the top of the list of priorities and will get on it ASAP.

silent-entertainment commented 3 years ago

Last night, I worked on migrating from the cl-notebook to the claxiom namespace. I used grep and sed with surgical precision, to rename the system name, package names, URIs, etc. However, there was an issue with the default notebook at ~/.claxiom/books/_notebook. The default _notebook is now broken for some reason that I haven't been able to determine. Starting (claxiom:main) has the usual buttons on the bottom, but the rest of the page is blank. Opening a new notebook by pointing to a non-existing file seems to work, but then the New Cell button does nothing.

I re-cloned the pre-migration claxiom repo for further exploration. I'm finding that the pre-existing interface is quite mysterious, as well as buggy, and I'm going to have to look more closely at the code to try to grok what is undocumented.

I've also been exploring darkmatter, which is a similar application. I had looked at darkmatter before I forked cl-notebook to resurrect it as claxiom. darkmatter seems much more polished than cl-notebook was before it was seemingly abandoned. The interface has a menu bar, in contrast to the context widget that is adjacent to the current cell in cl-notebook, and the menu items reveal corresponding keyboard shortcuts. It is very convenient to have a context menu that is local to the cell, to have the tool available where you're working, but the cl-notebook implementation of it has a few issues that require attention. Darkmatter also has a symbol navigator. Darkmatter just seems to work better than cl-notebook, or maybe it's just more user-friendly without documentation. However,, darkmatter has only two cell types (markdown and code), so it doesn't have the lispy html markup cell or the parenscript cell that cl-notebook gives us. The interface of darkmatter has a couple of quirks that might be related to undocumented features, which dissuaded me from using it as a base for claxiom because the interface of 'cl-notebookseemed to smooth it out, though now I'm discovering that the implementation of that smoothness is rather buggy or just unpredictable. Also,darkmatteruses theacejavascript editor and 'paredit, whereas cl-notebook uses CodeMirror and pareditesque for that functionality.

The other issue that complicates either using darkmatter as a base or merging its code into claxiom for its features is that it mixes so many languages, instead of leveraging available common lisp libraries that provide the same functionality in common lisp syntax. Instead of writing the html interface in cl-who (for example) and the the project-specific client code in parenscript, the html is in html [template] files, and the javascript is in javascript files. The thought of dealing with the multiple syntaxes for even a relatively small code base is very unappealing to me, as I've elucidated in README., because as the code base grows the potential for other issue grow, too.

Im not sure what the best strategy might be here. I'm going to have to study the code bases.

silent-entertainment commented 3 years ago

Progress Update:

I've been investigating the glitches in the migration to the claxiom namespace. As I mentioned above, migrating to the new namespace has caused a problem, in that opening a new notebook shows a blank workspace in the browser and the front end is unresponsive to trying to add a new cell. However, looking at the REPL output shows that the requests to handle new cells are being processed and that the file for the new notebook is being populated with new cells. Therefore, the problem appears to be in the front end, and the problem is probably due to the mangling of names in the syntax of parenscript. In other words, the technique that I used for the migration most likely missed something in src/ui/http-front-end, due to how identifiers are marked up for translation to identifiers in javascript.

At least I now have a hint at where to look for the problem, or at least I think that I do.

silent-entertainment commented 3 years ago

I guessed wrong!

I looked in hppt-front-end. I used grep. I even looked at the javascript output. I'm not seeing the problem.

silent-entertainment commented 3 years ago

I'm wondering if I can rule out there being a problem in the code base by defining cl-notebook as a nickname for package claxiom and reverting to the original static/books/_notebook. This might indicate malpractice in the surgery of _notebook.

silent-entertainment commented 3 years ago

Invoking (cl-notebook::main-dev) runs the server in a thread and gives us diagnostic output in the REPL. For the migrated code base it is (claxium::main-dev), reflecting the new package name. I have been looking at both the output from the current claxium github repo, which is working, and the output from the migrated branch, which isn't working and which I haven't published at the github repo, and the outputs are qualitatively similar. In fact, prior to loading the page at localhost:4242 both versions show aborting compilation with 2 FATAL errors (giving no further information), but only the first version works.

Of course, the output isn't very informative. It is just the status messages from usocket (almost 1000 lines each, after the page is loaded in the browser).

Most of the javascript in static/js looks like third-party libraries, and nothing in static/js had a signature that indicated the need for migrating any identifiers. Therefore, I'm probably back to looking for a problem in ui/http-api or ui/http-front-end.

silent-entertainment commented 3 years ago

TADA! It's working now.

Now why didn't I think of that? I knew that Firefox had Web Development tools. For several weeks recently, I had been working on Android, without a console, until I found a workaround, because I didn't have a proper system. Due to a serious change in lifestyle, it had been almost a decade since I had touched any programming other than the occasional javascript. I'm just now familiarizing myself with the tools. Please, be patient.

Firefox's Web Console gave me some more information, though it was difficult to work with as the buffer quickly filled up and the console became unresponsive. Most of the errors complained that canvas was null. It brought my attention to line 19 in src/ui/http-front-end/core.lisp. I had noticed the var CLNOTEBOOK = ~a before, but I glossed over it and didn't look at it closely enough, due to the signature.

Changing CLNOTEBOOK to CLAXIOM solved the migration problem. Before I upload the branch to the claxiom repo, I'm going to do some more testing, but I'm hoping this issue is closed.

silent-entertainment commented 3 years ago

claxiom has been migrated from cl-notepad to the claxiom namespace.