Thou shalt not make unto thee any graven image, or any likeness of any thing that is in heaven above, or that is in the earth beneath, or that is in the water under the earth.
Graven Image is a Common Lisp portability library (a less fancier name might've been =trivial-debugging=) for better interaction and debugging of a running Lisp image. One can inspect and debug all the things under heaven and [[https://www.corecursive.com/lisp-in-space-with-ron-garret/][above it]]—all in their own REPL-resident Lisp image!
Graven Image reuses compiler internals to improve/redefine the existing standard functions. This "improvement" often comes at a cost of changing the API of a function (for better customizability) or making it slightly less reliable (due to unstable, compiler-internal, or otherwise hacky implementation.)
The library is purposefully limited in scope:
Improving the standard functions has a priority over introducing new ones.
The interaction paradigm of the improved functions should stay standard (i.e. use =query-io= where standard requires =query-io=.)
Function arglist can be modified for convenience, but it should preferably stay as close to standard/implementation-specific arglist as possible.
Graven Image strives to not modify the REPL/image/shell in any way, relying on implementation defaults instead.
Portably reusing implementation-specific functionality is better than re-implementing it. But if some functionality is e.g. unique to SBCL, it might be dropped as non-portable.
Getting started
Clone the Git repository:
git clone --recursive https://github.com/aartaka/graven-image ~/common-lisp/
And load =:graven-image= in the REPL:
(asdf:load-system :graven-image) ;; or, if you use Quicklisp (ql:quickload :graven-image)
You can also install Graven Image via Guix, using the bundled =guix.scm= file:
guix package -f guix.scm
Someday (after at least minor version bump) I'll send a patch to Guix so that you can also install it with =guix install=. Until then—stay tuned (or send the patch yourself—I don't mind.)
If you want your REPL to start with Graven Image constructs accessible without package prefix, then simply use the package directly.
(asdf:load-system :graven-image) ;; Imports external symbols of Graven Image into the current package ;; (CL-USER?) Safe, because Graven Image shadows no CL symbols. (use-package :graven-image)
Or, if you're more orderly and disciplined about packages than I am, you can always use =trivial-package-local-nicknames= and define a shorter Graven Image nickname:
(trivial-package-local-nicknames:add-package-local-nickname :g :graven-image :cl-user)
If you want to replace the standard utilities with Graven Image ones, you can safely do so with the familiar:
;; For functions (fmakunbound 'apropos) (setf (fdefinition 'apropos) (fdefinition 'gimage:apropos)) ;; For macros (setf (macro-function 'time) (macro-function 'gimage:time))
The functions that Graven Image exposes are safely =:use=-able star-appended functions/macros (i.e. =describe*= instead of =describe=). Currently improved ones are:
All of the functions exposed by Graven Image are generics, so one can easily define =:around= and other qualified methods for them.
* =y-or-n-p=, =yes-or-no-p*= (generic functions)
Signature:
y-or-n-p &optional control &rest arguments => generalized-boolean yes-or-no-p &optional control &rest arguments => generalized-boolean
Improvements are:
* =apropos-list=, =apropos*= (generic functions)
Signature:
apropos-list string &optional (package nil) exported-only docs-too => list of symbols apropos string &optional (package nil) exported-only docs-too => no values
=apropos-list= now allows listing exported symbols only (with =exported-only=), which was a non-portable privilege of SBCL/Allegro until now. Search over docs (more intuitive for =apropos(-list)= than mere name search) is possible with =docs-too=.
Based on this foundation, =apropos*= lists symbols with their types, values, and documentation, so that implementation-specific formats are gone for a better and more unified listing:
(apropos* :max) ;; MAX [FUNCTION (NUMBER &REST ;; MORE-NUMBERS) : Return the greatest of its arguments; among EQUALP greatest, return...] ;; :MAX [SELF-EVALUATING] ;; CFFI::MAX-ALIGN ;; SB-ASSEM::MAX-ALIGNMENT [CONSTANT = 5] ;; ... ;; SB-C::MAXES ;; ALEXANDRIA:MAXF [MACRO (#:PLACE &REST NUMBERS) : Modify-macro for MAX. Sets place designated by the first argument to the...] ;; SB-KERNEL::MAXIMAL-BITMAP ;; ... ;; SB-LOOP::LOOP-ACCUMULATE-MINIMAX-VALUE [MACRO (LM OPERATION FORM)] ;; SB-LOOP::LOOP-MAXMIN-COLLECTION [FUNCTION (SPECIFICALLY)] ;; SB-LOOP::LOOP-MINIMAX [CLASS (STRUCTURE-OBJECT)] ;; ...
* =function-lambda-expression= (generic function)
Signature:
function-lambda-expression function/macro/method/symbol &optional force => list, list, symbol, list ;; Alias: lambda-expression function/macro/method/symbol &optional force => list, list, symbol, list
This function tries to read source files, process the definitions of functions, and build at least a barebones lambda from the arglist and documentation of the function. So that CL =function-lambda-expression= returns:
(function-lambda-expression #'identity) ;; => NIL, T, IDENTITY (function-lambda-expression #'print-object) ;; => NIL, T, PRINT-OBJECT
While the new Graven Image =function-lambda-expression= now returns:
(function-lambda-expression #'idenitity) ;; => (LAMBDA (THING) "This function simply returns what was passed to it." THING), ;; NIL, IDENTITY, (FUNCTION (T) (VALUES T &OPTIONAL)) (function-lambda-expression #'print-object t) ; Notice the T for FORCE, to build a dummy lambda. ;; => (LAMBDA (SB-PCL::OBJECT STREAM)), NIL, PRINT-OBJECT, (FUNCTION (T T) *)
Which means:
*** Return values
Things that =function-lambda-expression*= now returns are:
*** Helpers
Based on these new features of =function-lambda-expression*=, here are some Graven Image-specific helpers:
function-lambda-list function => list function-arglist function => list lambda-list function => list arglist function => list function-name function => symbol function-type function => list
* =time= (macro)
Signature:
time* &rest forms => return-values
The improved =time*= from Graven Image reuses as much implementation-specific APIs as possible, with the predictable output format.
And it also allows providing several forms, yay!
** =benchmark= (macro)
Signature:
benchmark* (&optional (repeat 1000)) &body forms => return-values
While =time= is the standard benchmarking/profiling solution, it's almost always too simple for proper benchmarking. Most systems getting complex enough end up with some form of custom benchmarking. Shinmera's [[https://github.com/Shinmera/trivial-benchmark/][trivial-benchmark]] is one such example. Graven Image =benchmark= is heavily inspired by =trivial-benchmark=, but has a more portable foundation in the form of =with-time*=.
As many other benchmarking macros, =benchmark*= repeats its body a certain number of times, collecting timing stats for every run, and then prints aggregate statistics for the total runs.
(gimage::benchmark* (20) ;; Repeat count. (loop for i below 1000 collect (make-list i) finally (return 1))) ;; Benchmark for 20 runs of ;; (LOOP FOR I BELOW 1000 ;; COLLECT (MAKE-LIST I) ;; FINALLY (RETURN 1)) ;; - MINIMUM AVERAGE MAXIMUM TOTAL ;; REAL-TIME 0.0 0.00175 0.019 0.035 ;; USER-RUN-TIME 0.000668 0.0016634 0.016315 0.033268 ;; SYSTEM-RUN-TIME 0.0 0.00021195 0.003794 0.004239 ;; GC-RUN-TIME 0.0 0.00085 0.017 0.017 ;; BYTES-ALLOCATED 7997952.0 8008154.5 8030464.0 160163090.0
** =with-time= (macro)
Signature:
with-time* (&rest time-keywords) (&rest multiple-value-args) form &body body
As the implementation detail of =time= and =benchmark=, =with-time= allows to get the timing data for interactive querying. =time-keywords= allow =&key=-matching the timing data (like =:gc= time or bytes =:allocated=) for processing in the body. While =multiple-value-args= allow matching against the return values of the =form=. So we get best of the both worlds: timing data and return values. This flexibility enables =time=, with its requirements of printing the data and returning the original values at the same time.
For example, here's how one would track the allocated bytes and garbage collection times when running a cons-heavy code:
(gimage:with-time* (&key aborted gc-count gc allocated) (lists lists-p) (loop for i below 1000 collect (make-list i :initial-element :hello) into lists finally (return (values lists t))) (unless aborted (format t "Bytes allocated: ~a, GC ran ~d times for ~a seconds" allocated gc-count gc))) ;; Bytes allocated: 7997952, GC ran NIL times for 0 seconds
* =describe= (generic function)
Signature:
describe* object &optional (stream t) respect-methods
Describes the =object= to the stream, but this time with portable format of description (determined by =graven-image:description= and specified for many standard classes) and with predictable set of properties (=graven-image:fields=). In Graven Image, both =describe= and =inspect= have the same format and the same set of fields.
As a note of respect to the original =describe=, Graven Image one allows to reuse the =describe-object= methods defined for user classes. To enable this, pass T to =respect-methods=.
** =graven-image:fields= (generic function)
Signature:
fields* object &key strip-null &allow-other-keys
Returns an undotted alist of properties for the =object=. Custom fields provided by Graven Image are named with keywords, while the implementation-specific ones use whatever the implementation uses. Arrays and hash-tables are inlined into fields to allow indexing these right from the inspector.
See =fields*= documentation for more details.
** =graven-image:description= (generic function)
Signature:
description* object &optional stream
Concise and informative description of =object= to the =stream=. Useful information from most of the implementations tested—united into one description header.
* =inspect= (generic function)
Signature:
inspect* object &optional strip-null
New'n'shiny =inspect*= has:
And here's a help menu of the new =inspect= (in this case, inspecting =readtable*=), just to get you teased:
This is an interactive interface for 5 Available commands are: :? Show the instructions for using this interface. :HELP Show the instructions for using this interface. :QUIT Exit the interface. :EXIT Exit the interface. (:LENGTH NEW) Change the page size. (:WIDTH NEW) Change the page size. (:WIDEN NEW) Change the page size. :NEXT Show the next page of fields (if any). :PREVIOUS Show the previous page of fields (if any). :PRINT Print the current page of fields. :PAGE Print the current page of fields. :HOME Scroll back to the first page of fields. :RESET Scroll back to the first page of fields. :TOP Scroll back to the first page of fields. :THIS Show the currently inspected object. :SELF Show the currently inspected object. :REDISPLAY Show the currently inspected object. :SHOW Show the currently inspected object. :CURRENT Show the currently inspected object. :AGAIN Show the currently inspected object. (:EVAL EXPRESSION) Evaluate the EXPRESSION. :UP Go up to the previous level of the interface. :POP Go up to the previous level of the interface. :BACK Go up to the previous level of the interface. (:SET KEY VALUE) Set the KEY-ed field to VALUE. (:MODIFY KEY VALUE) Set the KEY-ed field to VALUE. (:ISTEP KEY) Inspect the object under KEY. (:INSPECT KEY) Inspect the object under KEY. :STANDARD Print the inspected object readably. :AESTHETIC Print the inspected object aesthetically.
Possible inputs are:
* =dribble= (generic function)
Signature:
dribble* &optional pathname (if-exists :append)
Dribble the REPL session to =pathname=. Unlike the implementation-specific =dribble=, this one formats all of the session as =load=-able Lisp file fully reproducing the session. So all the input forms are printed verbatim, and all the outputs are commented out.
Beware: using any interactive function (like =inspect= etc.) breaks the dribble REPL. But then, it's unlikely one'd want to record interactive session into a dribble file.
* =documentation= (generic function)
Signature:
documentation object &optional (doc-type t) doc object &optional (doc-type t)
Improved version of =documentation=. Two main improvements are: =doc-type= is now optional, and =doc*= alias is available for convenience.
documentation.lisp also defines more =documentation= methods (and respective =setf= method) to simplify documentation fetching and setting. In particular, method on =(symbol (eql t))= to simplify symbol documentation search; and =(t (eql 'package))= with a new doc-type for package documentation convenience.
* =break= (macro)
Signature:
break* &rest arguments
A more useful wrapper for =break=, listing the function it's called from and the provided symbol values. See examples in the docstring.
Graven Image is made to be extensible. That's why most of the improved functions are generic: one can define special methods for their data and patch the behavior with =:before=, =:after=, and =:around= methods. Most of Graven Image functions mention the variables/things influencing them in the docstring. Here's a set of useful customizations:
* Beeping before =yes-or-no-p=
Restoring the standard-ish (beeping with bell (ASCII 7) character) behavior:
(defmethod gimage:yes-or-no-p :before (&optional control &rest arguments) (declare (ignore control arguments)) (write-char (code-char 7) query-io) (finish-output query-io*))
* Changing the accepted yes/no options for =yes-or-no-p= and =y-or-n-p*=
;; Make it strict yes/no as per standard. (defmethod gimage:yes-or-no-p :around (&optional control &rest arguments) (declare (ignore control arguments)) (let ((gimage:yes-or-no-options* '(("yes" . t) ("no" . nil)))) (call-next-method)))
;; Add more yes/no options (Russian, for example). (defmethod gimage:y-or-n-p :around (&optional control &rest arguments) (declare (ignore control arguments)) (let ((gimage:yes-or-no-options (append gimage:yes-or-no-options* '(("да" . t) ("ага" . t) ("нет" . nil) ("не" . nil) ("неа" . nil))))) (call-next-method)))
* Sorting =apropos-list= lists
Implementations are not good at sorting things, and their results are not often useful. Sorting things the way one needs is a useful extension. Here's a simple yet effective =:around= method that sorts things by =string= occurence:
(defmethod gimage:apropos-list* :around (string &optional packages external-only docs-too) "Sort symbols by the relation of subSTRING count to the length of symbol." (declare (ignorable packages external-only docs-too)) (let ((result (call-next-method))) (sort (remove-duplicates result) ;; For more comprehensive matching, see ;; a1b4ebd649e0268b1566e80709e7cea41363d006 and other commits ;; before c090d6dc14e05c561cf5c39cf5f6cc02e8cd04c5.
(let ((match-count 0))
(uiop:frob-substrings
(string sym) (list (string string))
(lambda (sub frob)
(incf match-count)
(funcall frob sub)))
(/ match-count (length (string sym))))))))** Changing printer settings for Graven Image output
Graven Image =inspect= function uses =interface-lines*= for the number of properties to list. If your screen is more than 20 lines high, you might want to add more lines:
(defmethod gimage:inspect :around (object) (declare (ignore object)) (let ((gimage:interface-lines* 45)) (call-next-method)))
Most of Graven Image functions also rely on implementation/REPL-specific printer variables, which might be un-intuitive, overly verbose, or too short. Binding printer variables around Graven Image functions helps that too:
(defmethod gimage:apropos :around (string &optional package external-only docs-too) (declare (ignore string package external-only docs-too)) ;; Note that you can also use ;; `sb-ext:compiler-print-variable-alist' and ;; `sb-ext:debug-print-variable-alist' on SBCL. (let ((print-case :downcase) (print-level 2) (print-lines 2) (print-length* 10)) (call-next-method)))
A noisy apropos function listing like
X86::X86-OPERAND-TYPE-NAMES [VARIABLE = ((:REG8 . 1) (:REG16 . 2) (:REG32 . 4) (:REG64 . 8) (:IMM8 . 16) (:IMM8S . 32) (:IMM16 . 64) (:IMM32 . 128) (:IMM32S . 256) (:IMM64 . 512) (:IMM1 . 1024) (:BASEINDEX . 2048) (:DISP8 . 4096) (:DISP16 . 8192) (:DISP32 . 16384) (:DISP32S . 32768) (:DISP64 . 65536) (:INOUTPORTREG . 131072) (:SHIFTCOUNT . 262144) (:CONTROL . 524288) (:DEBUG . 1048576) (:TEST . 2097152) (:FLOATREG . 4194304) (:FLOATACC . 8388608) (:SREG2 . 16777216) (:SREG3 . 33554432) (:ACC . 67108864) (:JUMPABSOLUTE . 134217728) (:REGMMX . 268435456) (:REGXMM . 536870912) (:ESSEG . 1073741824) (:INVMEM . 2147483648) (:REG . 15) (:WORDREG . 14) (:IMPLICITREGISTER . 75890688) (:IMM . 1008) (:ENCIMM . 464) (:DISP . 126976) (:ANYMEM . 2147547136) (:LLONGMEM . 2147547136) (:LONGMEM . 2147547136) (:SHORTMEM . 2147547136) (:WORDMEM . 2147547136) (:BYTEMEM . 2147547136) (:LABEL . 4294967296) (:SELF . 8589934592))]
turns into a much more readable
x86::x86-operand-type-names [variable = ((:reg8 . 1) (:reg16 . 2) (:reg32 . 4) (:reg64 . 8) (:imm8 . 16) (:imm8s . 32) (:imm16 . 64) (:imm32 . 128) (:imm32s . 256) (:imm64 . 512) ...)]
* Suppressing documentation errors in =documentation=
Several implementations throw errors when trying to get documentation for non-existent method combinations, classes, etc. It's convenient to suppress these:
(defmethod gimage:documentation* :around (object &optional doc-type) (ignore-errors (call-next-method)))
Actually, one can try to write an =:around= method for regular =documentation=, but this modification is not guaranteed to work on all implementations.
You can help with any of the [[https://github.com/aartaka/graven-image/issues?q=is%3Aopen+is%3Aissue][open issues]] most are well-described and split into bite-sized tasks. See .github/CONTIBUTING.md for the contributing guidelines.