hh commented 1 year ago

https://ii.nz/tmux.org generates https://ii.nz/tmux.html

Adding asciinema :prologue and :epilogue to tmux blocks like this:

#+name: emacs
#+begin_src tmux :session ii:emacs :prologue "asciinema rec emacs.cast --stdin --overwrite -c '" :epilogue "'"
emacs -nw .
#+end_src

Will produce a emacs.cast file, this can be displayed by putting behind html. Use https://ii.nz/emacs.cast, but follow the instructions here to help me figure out the right way to display the characters we type:

#+name: emacs.cast
#+begin_export html
  <head>
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <link rel="stylesheet" type="text/css" href="asciinema-player.css" />
    <!-- Place favicon.ico in the root directory -->
  </head>
  <div id="emacs"></div>
  <script src="asciinema-player.min.js"></script>
  <script>
    AsciinemaPlayer.create('emacs.cast', document.getElementById('emacs'),
      {
        autoPlay: true,
        preload: true,
        loop: true,
        theme: 'solarized-dark',
        poster: "data:text/plain,I'm regular \x1b[1;32mI'm bold green\x1b[3BI'm 3 lines down"

      }
      );
  </script>
#+end_export

From https://github.com/asciinema/asciinema-player#addeventlistenereventname-handler

input event is dispatched for every keyboard input that was recorded.

Callback's 1st argument is an object with data field, which contains registered input value. Usually this is ASCII character representing a key, but may be a control character, like "\r" (enter), "\u0001" (ctrl-a), "\u0003" (ctrl-c), etc. See input events in asciicast file format for more information.

This event can be used to play keyboard typing sound or display key presses on the screen amongst other use cases.

player.addEventListener('input', ({data}) => {
  console.log('input!', JSON.stringify(data));
})

iiamabby commented 1 year ago

Display everything including the escaped unicode

HTML

<div id="center-items">

    <div style ="width:50%; height:50%;"id="emacs">
        <p id="keystroke">#</p>
    </div>
</div>

JavaScript:


<script src="asciinema-player.min.js"></script>
<script>
var p = document.getElementById('keystroke')
AsciinemaPlayer.create({
    url: 'emacs.cast',
    inputOffset: -2.0,
    autoPlay: true,
    preload: true,
    loop: true,
    theme: 'solarized-dark',
    poster: "data:text/plain,I'm regular \x1b[1;32mI'm bold green\x1b[3BI'm 3 lines down"
}, document.getElementById('emacs')).addEventListener('input', ({data}) => {p.innerText=data});


These two line are the what makes the feature work:  ```document.getElementById('emacs')).addEventListener('input', ({data}) => {p.innerText=data}); ```

iiamabby commented 1 year ago

Added key mapping and decoding

Git repo

Decodes escaped Unicode sequence and converts into the keycode number, then finds the matching description for the keycode

hh commented 1 year ago

@zachmandeville can you take a look at this and add it to an org file? I think we are getting close!

zachmandeville commented 1 year ago

I took a look at the ticket description and current work. @abdabTheCreator did great work in that asciinema-keydisplay repo to show the inputs above an embedded cinema cast. I am not certain if that is what the ticket is asking for, though, beecause I am not quite sure what the ticket is asking for in general hahaha.

I wrote up an org file to try to capture the current situation, what we're trying to do, and my questions. I pushed it to our org repo, as I don't want discussion org files in this slim config repo. The contents are included in this comment below, too:

Introduction
Using asciicinema
Capturing keyboard input.
How does this fit into the doom config?
Capturing key presses in an emacs org file
Current thoughts

Introduction

There is an open ticket in ii/doom-config titled “display recorded input in an overlay”. It is about using asciicinema with org and tmux to create screencasts.

This org file is meant to track current thoughts around the ticket, its purpose, and where it fits into the doom-config.

Using asciicinema

asciinema(homepage) is a tool for recording your terminal sessions, with a mechanism for playing back or sharing these sessions when done. It is useful for recording demos or recording help videos displaying the steps to enter into a terminal.

The output of running asciicenma is a .cast file, which can then be uploaded to asciinema’s cast repository at asciicinema.org or saved and played locally using the asciinema-player.

asciinema.org

you can upload the cast you made with the command

asciinema upload file.cast

this returns a private archive.org url with your playback. It ends up looking a bit like youtube, but only with terminal casts.

asciinema player

Alternately, you can embed the cast into your own website using asciinema-player. This is a combo of css and javascript that you add to your own page and then display the cast file without having to upload it anywhere else.

the css and js can be downloaded from their repo, or you could host them somewhere and then link to them externally.

A simple version of the html page would look like so

<html>
  <head>
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <link rel="stylesheet" type="text/css" href="https://link.to/asciinema-player.css" />
  </head>
  <body>
  <div id="cast"></div>
  </body>
  <script src="https://link.to/asciinema-player.min.js"></script>
  <script>
    AsciinemaPlayer.create('/example.cast', document.getElementById('cast'));
  </script>
</html>

In the above, you would have example.cast in the same root folder as this index.html, and it combined with the css and js would create an embedded screencast in the #cast div. You coudl then style it further as you see fit.

asciinema to svg

There is also a cast-to-svg tool you can use so that you can embed the recording without needing additional CSS and javascript.

https://github.com/marionebl/svg-term-cli

this tool takes a cast file as input and outputs a named svg. Then you’d end up with html like so:

<html>
  <head>
    <meta name="viewport" content="width=device-width, initial-scale=1">
  </head>
  <body>
    <img src='/example-cast.svg' />
  </body>
</html>

the longer the session, the bigger the file, and so if we are meaning to do incredibly long sessions, or having multiple casts on a page, this may not be the best option, but it could be useful for including asciinema output into an org file’s exported html.

For example, you could have an org file with a tmux block of

docker run busybox/helloworld

and then below it as “results” you’d see the terminal session of just this command, showing docker pulling the file and printing out hello.

Capturing keyboard input.

You can capture your key input during an asciicinema session by passing in the –stdin flag at the start of the session. The resulting file will include a data map with the unicode characters pressed. You can then add some additional javascript to translate that unicode to their correct symbols and then paste it somewhere on that embedded webpage. Abby does a great breakdown of that, with an demo site, in the repo abdabthecreator/asciinema-keydisplay .

In the demo site, the keys display above the embedded cast. Overlaying the keys onto the cast, in semi-transparent font, is just a matter of some CSS. Essentially, you’d put both in a wrapper div, and then give the <p> tag showing the inputs an absolute positioning.

How does this fit into the doom config?

To be honest, I am not sure yet! From the start of the ticket, it seemed like we were wanting to incorporate asciinema into tmux blocks. When we run the blocks, we do it inside an asciinema session. This would be handy for saving the resulting work as a playback video of sorts in the end.

However, there aren’t really keypresses we need to catpure here. All the work is being piped in from tmux. Some of the examples seemed to imply we’d open up emacs in the tmux session, that is connected to another emacs client, and then use that parent client to pipe in keypresses via tmux to the second client. This seems unecessarily convoluted, and I’m not sure the use case.

If we are trying to show a playback of the tmux command being run in some final html, then it feels like we’d want to wrap each tmux command in an asciicinema invocation, with the resulting file name and path being passed in from the src block. We could then just put an img tag beneath each block that has that name, using some elisp triggers and such. Is that what we are trying to achieve here?

As it relates to the doom-config, I imagine we’d likely figure out the exact elisp we’d want to run and then add that to our existing ob-tmux library as an additional option to pass in. This seems the cleanest way.

Capturing key presses in an emacs org file

The other intention I can see here is to capture our keypresses in the org file itself. We want to show how you can navigate from code block to code block and hit ctrl-c ctrl-c to trigger that codeblock. The idea being that we can create cooperation templates for learning labs, that come with our emacs and some file you are meant to read through and excute the scripts as you approach them. In this case, the person isn’t trying to use emacs, emacs is more of a executable document viewer.

If that is the case, I think it’d be better to record a screencast separate from the org file, since it is intended to show the keypresses for any org file. We could then link to that screencst in our documentation. Alternately, it is possible to present elisp functions as a link. So we could hae a link above each code block that, when clicked, runs the command “excute this code block”. The person reading through the document only needs to know how to scroll and click.

At that point, though, it might be more useful to think about a rendered website that has these links as buttons that pass the info to the emacs server which then runs the command. Then, when someone is starting up their lab, they just navigate to a given page, read the instructions, click where they’re meant to, and see the output in a tmux window open on another tab. this is approaching katacode and other online labs connected to VM’s…which is maybe what we want?

In either case, this seems like work that would happen outside the doom-config, and wouldn’t require recording keyboard presses in tmux blocks.

Current thoughts

At the moment, I would like to discuss what our hoped-for goal is, and where that work is best done. The ticket is open in doom-config, but this may not be where the work is needed?

hh commented 1 year ago

What I would like to add to this repo is examples/tmux.org that when rendered as a reveal js presentation shows the keystrokes.

It may be that we use a hosted asciinima, or place the files within a certain place in the current folder (probably want both options long term).

I think eventually this work will find it's way into the doom-config, as we figure out what changes need to be made as defaults for tmux code blocks.

Another feature later, it to asciinima cat the tmux output results into a plain text results block. So that tmux interactive work can be rendered similar to shell block output.

hh commented 1 year ago

However, there aren’t really keypresses we need to catpure here.

Much of this is to create slides or content that others will be able to understand the keystrokes being sent as folks use emacs, shell, and other terminal applications that may be a completely new concept to them.

All the work is being piped in from tmux.

Not necessarily, some of this is launching sessions of emacs to record the keystrokes so others can scrobble in the player along to learn what keys are pressed.

Some of the examples seemed to imply we’d open up emacs in the tmux session, that is connected to another emacs client, and then use that parent client to pipe in keypresses via tmux to the second client.

I suspect the 'left eye' is most cases will not be in a tmux session, but in the broadway gui. Folks will click on the gui version, driving the 'right eye' which is the tmux session (and recorded keystrokes).

This seems unecessarily convoluted, and I’m not sure the use case.

Primary use was is left eye / gui emacs, driving right eye / tmux (sometimes emacs, sometimes vi, and others), to help create slides and presentations that others can use to share what they did (including keystrokes).

hh commented 1 year ago

asciinema to svg

Might be useful in some scenarios, but as an additional approach.

hh commented 1 year ago

I've gotten this pretty close, I'd like @zachmandeville to look at using https://coder.ii.nz/templates/iipod/workspace and invite me to show what's missing or alternatives.

zachmandeville commented 1 year ago

I checked out the template (in the currently running workspace zz-test-ascii) and the basic flow works!

i ran several tmux blocks, which created a cast file upon completion. I then exported it as a revealjs presentation.

I then opened up the exported html file in the browser and could see on each slide the tmux output.

For command 4, it opens up emacs in tmux. so I then went to that emacs session aand tried opening a file, writing a line, and deleting the line. I wanted to see if keypresses came through.

I found, when looking at the resulting .cast, the coloring was all types of distorted that it made it difficult to tell what was on the screen let alone if the keypresses shown. There are several layers of terminal coloration happening though. We have the terminal itself, then the emacs theme, then I opened an org file with src blocks so there was an additional gray color for each block. This was then converted into an html inside the reveal.js. Somewhere in that block there is a colour disagreement.

I think it is really slick to have a tmux session run that then saves a recording of that output. It's something I can see being really helpful too. The recording of an emacs session in the slide had less of an immediate draw, it is meta-recursive in a way that is more confusing to my brain than exciting.

Personally, I think recording emac session to teach how to navigate org files and emacs commands, showing the keys pressed as part of a rehearsed/clear session, is useful--but as a separate exercise from this.

What I can imagine immediately with this tool is something like: I am writing an org file about setting up a tool (creating a docker contaioner let's say) and i want to show what happens when you invoke docker build -t foo ., with the layers being pulled down and such. So I can execute that command as part of the writing, the logged output gets save as an asciicinema, and then it all gets saved as a presentation someone can move through and compare their output to the one we are showing. That is nice.

hh commented 1 year ago

To reduce the scope here, let's focus on just modifying the command4 block of the https://ii.nz/ii.org file. Feel free to put it as a gist, or update the file directly via the hh/ii-nz workspace on https://coder.ii.nz

Just update the html block so that keystrokes are displayed on the command4 slide:

* record command4.cast :noexport:
#+name: command4
#+begin_src tmux :session ":command4" :prologue "asciinema rec command4.cast --stdin --overwrite -c '" :epilogue "'" :exports none
emacs -nw .
#+end_src
* Recording 4
#+name: command4.cast
#+begin_export html
  <head>
    <meta name="viewport" content="width=device-width, initial-scale=4">
    <link rel="stylesheet" type="text/css" href="https://ii.nz/asciinema-player.css" />
    <!-- Place favicon.ico in the root directory -->
  </head>
  <div id="command4"></div>
  <script src="https://ii.nz/asciinema-player.min.js"></script>
  <script>
    AsciinemaPlayer.create('command4.cast', document.getElementById('command4'),
      {
        autoPlay: true,
        preload: true,
        loop: true,
        theme: 'solarized-dark',
        poster: "data:text/plain,I'm regular \x4b[4;32mI'm bold green\x4b[3BI'm 3 lines down"

      }
      );
  </script>
#+end_export

ii / doom-config

Add Support for asciinima input display #1

Display everything including the escaped unicode

Added key mapping and decoding

Introduction

Using asciicinema

asciinema.org

asciinema player

asciinema to svg

Capturing keyboard input.

How does this fit into the doom config?

Capturing key presses in an emacs org file

Current thoughts