alex-gutev / generic-cl

Generic function interface to standard Common Lisp functions
https://alex-gutev.github.io/generic-cl/
MIT License
143 stars 8 forks source link

Specialize elt for arrays and list indices #4

Closed anlsh closed 4 years ago

anlsh commented 4 years ago

This functionality probably exceeds generic-cl's mission, but I thought I'd submit a PR nonetheless.

This PR allows generic-cl:elt to access contiguous slices (ie displaced arrays) of multidimensional arrays. It's easiest to explain this by example

(defvar a #2A((0 1) (2 3)))
(elt a '(0))
;; => #(0 1)
(elt a '(1))
;; => #(2 3)
(setf (elt a '(1)) #(200 300))
;; => #(200 300)
a
;; => #2A((0 1) (200 300))
(defvar b (elt a '(0)))
(setf (elt b '(0)) -100)
a
=> #2A((-100 1) (200 300))

The current elt for arrays only accepts integers (implicitly anyways, since row-major-aref will fail on anything else). I've changed the old elt type specs from (array t) to (array integer), and the newfangled behavior is specialized on (array list). Therefore no existing code should be affected

Notes

This implementation is probably slow (though as fast as it can be, I think). There are two reasons

  1. There's no place to cache per-dimension word sizes for the arrays (would love to be proven wrong here), so both setf and elt have recompute on those every call.

  2. The setf operation copies thevalue array index-by-index rather than as a block. I don't think CL has any efficient functions for that though

anlsh commented 4 years ago

As to point (2), I think efficient copying is probably possible via using ´cffi´ to call ´memcpy´ or something similar. If we want to consider the PR I can update it after looking into this

alex-gutev commented 4 years ago

I've considered the pull request and it seems like a good addition to generic-cl.

I've started resolving the merge conflicts however can wait prior to merging it in the master branch, if you'll be making further changes.

alex-gutev commented 4 years ago

Regarding point 1, the only way, I can think of, for caching the word sizes is to create a new structure type which is a wrapper around a multi-dimensional array with an extra field which stores the word size. In code making use of generic-cl, by implementing the appropriate generic interfaces, this type can fully replace a CL multi-dimensional array with few modifications to the surrounding code.

anlsh commented 4 years ago

Hold off on merging for now. I just remembered that the indices need to be checked using ´array-in-bounds-p´: the current implementation will happily calculate the word position using out-of-bounds indices, creating a really nasty and possibly silent failure case.

I'll update tomorrow with the fix and conclusions on possible ´memcpy´ing and subclassing ´array´

anlsh commented 4 years ago

I've updated the branch to add the bounds checking I mentioned, factor out some common logic between the elt and setf elt methods, add documentation, and resolve the merge conflicts. Sorry about the force pushing- force of habit there :|

As to the two other points

  1. Like you mentioned, we could definitely speed up the word-size calculations by creating a subclass of array with an extra attribute to cache the word sizes, and further specializing elt on this subclass so that it could simply access the cache. We could also shadow make-array with a similar function which returns instances of this subclass instead, which shouldn't(?) affect any existing code.

  2. Efficient copying of definitely can't be accomplished portably. I suspected this last night, but was hoping that certain implementation-specific functionality might have made it possible. After asking around on the Lisp discord it seems might be possible through SBCL, but even if is the relevant functionality isn't documented anywhere in the SBCL manual. Anyways, I don't want to force this issue

Regardless, the existing code would need to be included anyways: so I'd say we're good to merge