atlas-engineer / nfiles

User configuration and data file management
BSD 3-Clause "New" or "Revised" License
18 stars 5 forks source link

Searialize lisp-file contents to `read`-able data? #6

Open aartaka opened 2 years ago

aartaka commented 2 years ago

How about this: define a reader macro for classes and (de)serialize objects with this macro in *readtable*? Something like sharpsign-s reader macro for structures, but for classes. Maybe even rebind the structure one? Then we can rebind readtables to the new syntax for the time or reading:

(defmethod files:serialize ((profile profile) (file lisp-file) stream &key)
  (let ((*readtable* (copy-readtable)))
    (set-dispatch-macro-character #\# #\S #'read-class-or-structure)
    ;; A hypothetical function writing the object with the #S syntax.
    (write-nested-maybe-objects (content file) :stream stream)))

(defmethod files:deserialize ((profile profile) (file lisp-file) stream &key)
  (declare (ignore stream))
  (let ((*readtable* (copy-readtable)))
    (set-dispatch-macro-character #\# #\S #'read-class-or-structure)
    ;; A hypothetical function writing the object with the #S syntax.
    (read (call-next-method))))

Benefits:

Downsides:

Ambrevar commented 2 years ago

If I get it right, you want to skip the deserialize call so that read-ing would be enough, is this correct?

In any case, we would not replace cl-prevalence, since write-nested-maybe-objects is essentially what cl-prevalence does.

Note: can we leverage make-load-form?

aartaka commented 2 years ago

Looks interesting. You never know how much foresight you can discover in the spec ( ͡° ͜ʖ ͡°)

EDIT: an -> can

Ambrevar commented 2 years ago

Actually make-load-form would only work if we load the file instead of merely read-ing it. Not sure that would fit the bill here...

aartaka commented 2 years ago

Actually make-load-form would only work if we load the file instead of merely read-ing it. Not sure that would fit the bill here...

That's would work perfectly in conjunction with #5! Imagine: we serialize the data to a huge make-load-form output, write it to a file, complile to fasl, and then load it, getting the whole set of objects restored! \(@ ̄∇ ̄@)/

aartaka commented 2 years ago

A small demo that seems to work on simple/standard types:

(defvar *data* nil)

(defun object->file (object file)
  (with-open-file (f file :direction :output
                          :if-does-not-exist :create
                          :if-exists :supersede)
    (prog1
        (write `(setf *data* ,(make-load-form object)) :stream f)
      ;; Optional, to speed things up.
      (compile-file file)
      file)))

(defun file->object (file)
  (load file)
  *data*)
Ambrevar commented 2 years ago

OK, but what abou the readtable trick from the original post?

Ambrevar commented 2 years ago

Also before working on this, let's run some benchmarks (e.g. on Nyxt's GHT).