Swift LispKit

Overview

LispKit is a framework for building Lisp-based extension and scripting languages for macOS and iOS applications. LispKit is fully written in the programming language Swift. LispKit implements a core language based on the R7RS (small) Scheme standard. It is extensible, allowing the inclusion of new native libraries written in Swift, of new libraries written in Scheme, as well as custom modifications of the core environment consisting of a compiler, a virtual machine as well as the core libraries.

The NumericalScheme demo showcases how to create a derived LispKit interpreter that inherits everything from LispKit (without code duplication) and defines a new native as well as Scheme-based library.

The iOS version of the LispKit framework supports all libraries except for (lispkit system call). Library (lispkit draw) works consistently across iOS and macOS but does not support color lists on iOS. In general, the differences between the macOS and iOS version of the framework are minor. Interestingly, the iPhone 12 Pro (2020) seems to consistently outperform the MacBook Pro 16" (2019) for simple LispKit benchmarks.

LispPad implements a simple, lightweight, integrated development environment for LispKit on macOS with a Cocoa-based UI. The LispPad Library Reference documents the core LispPad and LispKit libraries in PDF form. On iOS, application LispPad Go provides a simple Scheme IDE based on LispKit. The source code of LispPad Go is available on GitHub. A much simpler command-line tool for iOS is included in the LispKit framework itself (see below).

Features

LispKit provides support for the following core features, many of which are based on R7RS:

• Modules based on R7RS libraries
• Hygienic macros based on syntax-rules
• First-class environments
• call/cc, dynamic-wind and exceptions
• Dynamically-scoped parameters
• Multiple return values
• Delayed execution via promises and streams
• Support for the full numerical tower consisting of arbitrary size integers, rationals, real numbers, and inexact complex numbers.
• Unicode strings and characters
• Vectors and bytevectors
• Text and binary ports
• R7RS-compliant records
• R6RS-compliant hashtables
• R6RS-compliant enumerations
• SRFI 18-compliant multi-threading
• All R7RS (small) libraries: (scheme base), (scheme case-lambda), (scheme char), (scheme complex), (scheme cxr), (scheme eval), (scheme file), (scheme inexact), (scheme lazy), (scheme load), (scheme process-context), (scheme read), (scheme repl), (scheme time), (scheme write), (scheme r5rs)
• Some R7RS (large) libraries from Scheme Red and Scheme Tangerine editions: (scheme bitwise), (scheme box), (scheme charset), (scheme comparator), (scheme division), (scheme fixnum), (scheme flonum), (scheme generator), (scheme hash-table), (scheme ideque), (scheme list), (scheme mapping), (scheme red), (scheme rlist), (scheme set), (scheme sort), (scheme stream), (scheme text), (scheme vector)
• LispKit-specific libraries: (lispkit base), (lispkit core), (lispkit control), (lispkit system), (lispkit system call), (lispkit system keychain), (lispkit system pasteboard), (lispkit box), (lispkit math), (lispkit math matrix), (lispkit math util), (lispkit math stats), (lispkit list), (lispkit list set), (lispkit hashtable), (lispkit dynamic), (lispkit type), (lispkit vector), (lispkit gvector), (lispkit bitset), (lispkit record), (lispkit bytevector), (lispkit char), (lispkit char-set), (lispkit string), (lispkit format), (lispkit port), (lispkit date-time), (lispkit draw), (lispkit draw turtle), (lispkit draw barcode), (lispkit draw chart bar), (lispkit styled-text), (lispkit datatype), (lispkit object), (lispkit enum), (lispkit enum r6rs), (lispkit regexp), (lispkit stream), (lispkit thread), (lispkit thread channel), (lispkit thread future), (lispkit thread shared-queue), (lispkit url), (lispkit graph), (lispkit match), (lispkit iterate), (lispkit log), (lispkit debug), (lispkit set), (lispkit stack), (lispkit queue), (lispkit heap), (lispkit disjoint-set), (lispkit wt-tree), (lispkit comparator), (lispkit combinator), (lispkit logic), (lispkit prolog), (lispkit clos), (lispkit test), (lispkit prettify), (lispkit text-table), (lispkit csv), (lispkit markdown), (lispkit sqlite), (lispkit archive tar), (lispkit archive zip), (lispkit crypto), (lispkit json), (lispkit json schema), (lispkit http), (lispkit http server), (lispkit http oauth), (lispkit serialize), (lispkit sxml), (lispkit sxml xml), (lispkit sxml html), and (lispkit pdf).

LispKit is incompatible or incomplete with respect to the following R7RS features:

• Lists are immutable. Mutable cons-cells are supported in a way similar to Racket
• Literals in syntax-rules are not matched based on their definition but their symbol identity
• Datum comments introduced via #; do not always work as in other Scheme dialects.

The following SRFI libraries have been ported to LispKit and are included in the framework:

• SRFI 1: List Library
• SRFI 2: AND-LET - an AND with local bindings, a guarded LET special form
• SRFI 6: Basic String Ports
• SRFI 8: receive - Binding to multiple values
• SRFI 9: Defining Record Types
• SRFI 11: Syntax for receiving multiple values
• SRFI 14: Character-set library
• SRFI 16: Syntax for procedures of variable arity
• SRFI 17: Generalized set!
• SRFI 18: Multithreading support
• SRFI 19: Time Data Types and Procedures
• SRFI 23: Error reporting mechanism
• SRFI 26: Notation for Specializing Parameters without Currying
• SRFI 27: Sources of Random Bits
• SRFI 28: Basic Format Strings
• SRFI 31: A special form rec for recursive evaluation
• SRFI 33: Integer Bitwise-operation Library
• SRFI 34: Exception Handling for Programs
• SRFI 35: Conditions
• SRFI 39: Parameter objects
• SRFI 41: Streams
• SRFI 46: Basic Syntax-rules Extensions
• SRFI 48: Intermediate Format Strings
• SRFI 51: Handling rest list
• SRFI 54: Formatting
• SRFI 55: require-extension
• SRFI 63: Homogeneous and Heterogeneous Arrays
• SRFI 64: A Scheme API for test suites
• SRFI 69: Basic hash tables
• SRFI 87: => in case clauses
• SRFI 95: Sorting and Merging
• SRFI 98: An interface to access environment variables
• SRFI 101: Purely Functional Random-Access Pairs and Lists
• SRFI 102: Procedure Arity Inspection
• SRFI 111: Boxes
• SRFI 112: Environment inquiry
• SRFI 113: Sets and bags
• SRFI 118: Simple adjustable-size strings
• SRFI 121: Generators
• SRFI 125: Intermediate hash tables
• SRFI 128: Comparators
• SRFI 129: Titlecase procedures
• SRFI 132: Sort Libraries
• SRFI 133: Vector Library
• SRFI 134: Immutable Deques
• SRFI 135: Immutable Texts
• SRFI 137: Minimal Unique Types
• SRFI 141: Integer division
• SRFI 142: Bitwise Operations
• SRFI 143: Fixnums
• SRFI 144: Flonums
• SRFI 145: Assumptions
• SRFI 146: Mappings
• SRFI 149: Basic syntax-rules Template Extensions
• SRFI 151: Bitwise Operations
• SRFI 152: String Library
• SRFI 154: First-class dynamic extents
• SRFI 155: Promises
• SRFI 158: Generators and Accumulators
• SRFI 161: Unifiable Boxes
• SRFI 162: Comparators sublibrary
• SRFI 165: The Environment Monad
• SRFI 166: Monadic Formatting
• SRFI 167: Ordered Key Value Store
• SRFI 173: Hooks
• SRFI 174: POSIX Timespecs
• SRFI 175: ASCII Character Library
• SRFI 177: Portable keyword arguments
• SRFI 180: JSON
• SRFI 189: Maybe and Either: optional container types
• SRFI 194: Random data generators
• SRFI 195: Multiple-value boxes
• SRFI 196: Range Objects
• SRFI 204: Wright-Cartwright-Shinn pattern matcher
• SRFI 208: NaN procedures
• SRFI 209: Enums and Enum Sets
• SRFI 210: Procedures and Syntax for Multiple Values
• SRFI 214: Flexvectors
• SRFI 215: Central log exchange
• SRFI 216: SICP Prerequisites
• SRFI 217: Integer sets
• SRFI 219: Define higher-order lambda
• SRFI 221: Generator/accumulator sub-library
• SRFI 222: Compound objects
• SRFI 223: Generalized binary search procedures
• SRFI 224: Integer mappings
• SRFI 227: Optional Arguments
• SRFI 228: Composing Comparators
• SRFI 229: Tagged procedures
• SRFI 230: Atomic Operations
• SRFI 232: Flexible curried procedures
• SRFI 233: INI files
• SRFI 235: Combinators
• SRFI 236: Evaluating expressions in an unspecified order
• SRFI 239: Destructuring Lists

Project

The project defines four different targets:

• LispKit: the core interpreter framework, including all support files, for macOS
• LispKit iOS: the core interpreter framework, including all support files, for iOS
• LispKitTools: a framework for tools supporting LispKit; e.g. a read-eval-print framework for macOS
• LispKitRepl: a command-line tool implementing a read-eval-print loop for macOS

Architecture

LispKit consists of:

1. a compiler translating LispKit expressions into bytecode,
2. a virtual machine for interpreting the generated bytecode. The virtual machine is stack-based, handles tail calls and continuations, and provides a garbage collector,
3. a large range of libraries, all packaged together with the framework, and
4. a simple read-eval-print loop for macOS and iOS.

Details can be found in the LispKit Wiki.

Command-line tool

Overview

This project includes a command-line tool, called the LispKit Shell, for executing LispKit applications in the terminal. It can be used to try out and experiment with the LispKit framework. The command-line tool can also be used interactively as a read-eval-print loop. The read-eval-print loop parses the entered LispKit expression, compiles it to bytecode, executes it, and displays the result.

Downloading the source code

First, clone the LispKit repository via git. The following command will create a directory swift-lispkit.

> git clone https://github.com/objecthub/swift-lispkit.git
Cloning into 'swift-lispkit'...
remote: Enumerating objects: 7020, done.
remote: Counting objects: 100% (365/365), done.
remote: Compressing objects: 100% (349/349), done.
remote: Total 7020 (delta 174), reused 201 (delta 10), pack-reused 6655
Receiving objects: 100% (7020/7020), 11.29 MiB | 5.20 MiB/s, done.
Resolving deltas: 100% (4853/4853), done.

Next, switch to Xcode and build the LispKit command-line tool via scheme LispKitRepl:

> open LispKit.xcodeproj

Compiling the command-line tool with the Swift Package Manager

A debug binary can be built in the following way:

> cd swift-lispkit
> swift build -Xswiftc "-D" -Xswiftc "SPM"
Fetching https://github.com/objecthub/swift-markdownkit.git from cache
Fetching https://github.com/objecthub/swift-numberkit.git from cache
Fetching https://github.com/objecthub/swift-sqliteexpress.git from cache
Fetching https://github.com/objecthub/swift-commandlinekit.git from cache
Fetching https://github.com/weichsel/ZIPFoundation.git from cache
Cloning https://github.com/objecthub/swift-sqliteexpress.git
Resolving https://github.com/objecthub/swift-sqliteexpress.git at 1.0.3
Cloning https://github.com/weichsel/ZIPFoundation.git
Resolving https://github.com/weichsel/ZIPFoundation.git at 0.9.12
Cloning https://github.com/objecthub/swift-markdownkit.git
Resolving https://github.com/objecthub/swift-markdownkit.git at 1.0.5
Cloning https://github.com/objecthub/swift-numberkit.git
Resolving https://github.com/objecthub/swift-numberkit.git at 2.3.9
Cloning https://github.com/objecthub/swift-commandlinekit.git
Resolving https://github.com/objecthub/swift-commandlinekit.git at 0.3.3
[180/180] Linking LispKitRepl

The debug binary can now be run like this:

.build/debug/LispKitRepl -r Sources/LispKit/Resources -d LispKit

It is possible to execute a Scheme program at Sources/LispKit/Resources/Examples/Channels.scm like this:

.build/debug/LispKitRepl -r Sources/LispKit/Resources -d LispKit Sources/LispKit/Resources/Examples/Channels.scm

For experimentation with a new resources directory (e.g. containing new or modified Scheme libraries), the binary can also be run via .build/debug/LispKitRepl -d LispKit assuming that directory ~/Documents/LispKit contains a copy of the resources directory.

There is also a Makefile which provides convenience build rules. Building and running the REPL can be achieved, e.g. by invoking make run (for the debug REPL) or make repl (for the release REPL). If a Scheme program should be executed instead of the REPL being run, the program variable needs to be set, e.g. as in make run program=path/to/program.scm. Details for building binaries or executing tests can be looked up in the Makefile.

Building and running the iOS tool

Within Xcode, switch the scheme to the target LispKitRepl iOS and build and run the project. By default, this will invoke the iOS simulator and run an application "LispKitRepl iOS". The application has a chat-inspired UI in which Scheme commands can be entered at the bottom and sent to the interpreter via the "arrow-up" button. The interpreter will then execute the command and print the result in the console area above. There is a single button with a trash can in the toolbar for resetting the interpreter. This button turns into a button for cancelling running programs (e.g. if there is an infinite loop or other deadlock).

There is also a more advanced open-source iOS application called LispPad Go which implements a complete IDE for LispKit. LispPad Go is available from the iOS app store.

Requirements

The following technologies are needed to build the components of the LispKit framework. For building the command-line tool, all that is needed is the Swift Package Manager. For compiling the framework and trying the command-line tool directly in Xcode, the Swift Package Manager is not needed.

• Swift 5.9
• Xcode 15
• Swift Package Manager
• NumberKit
• MarkdownKit
• CommandLineKit
• SQLiteExpress
• ZIPFoundation
• Swift Atomics
• CLFormat
• DynamicJSON
• NanoHTTP
• OAuth2
• CBORCoding
• KeychainAccess