nikodemus / esrap

OLD REPOSITORY: Please go to:
https://github.com/scymtym/esrap
81 stars 25 forks source link

More safe rules #4

Closed archimag closed 11 years ago

archimag commented 13 years ago

Hi,

I believe that the use of the global rules\ unsafe. So I made ​​a small change that allows my to use own safe rules. With this change, I can write like this:

(defvar *my-rules* (make-hash-table))

(defvar *my-compiled-grammar* nil)

(defmacro with-my-rules (&body body)
  `(let ((*rules* *my-rules*))
     ,@body))

(defun compile-my-grammar ()
  (setf *my-compiled-grammar*
        (with-my-rules
          (compile-grammar 'my-toplevel-symbol))))

(defmacro define-my-rule (symbol expression &body options)
  `(with-my-rules
     (defrule ,symbol ,expression ,@options)))

(defun parse-my-doc (text)
  (unless *my-compiled-grammar*
    (compile-my-grammar))
  (parse *my-compiled-grammar* text))
nikodemus commented 13 years ago

I'm not sure what you mean by unsafe. Can you expand on that?

archimag commented 13 years ago

For example, the code

(defrule if ....)

Has a high probability of conflict with other library which use esrap.

vsedach commented 13 years ago

To give a concrete example, archimag is using this right now to extend the esrap-based 3bmd markdown parser to understand CLiki-style wiki links. Compare the current situation (https://github.com/archimag/cliki2/blob/master/src/markup.lisp) with the proposed extensions (https://github.com/archimag/cliki2/blob/new-design/src/markup.lisp) to see how this looks in practice.

nikodemus commented 13 years ago

Thanks.

I see the issue.

I don't want to expose RULES, though, because it's implementation as a hash-table is not part of the public interface.

Right now doing what you do, and reaching behind the scenes to bind RULES\ is one way, and another is to only use symbols in your own packages to name rules.

I think TRT is to expose a separate GRAMMAR object which encapsulates the rule table, and make it part of the API, so that you can do things like:

(in-grammar mypackage:cliki)

(defrule ...)

(parse ... :grammar mypackage:cliki)

A GRAMMAR object also seems like the right granularity for locking. I'll see what I can cook up.

nikodemus commented 13 years ago

Didn't mean to close this...

archimag commented 13 years ago

I like the solution:

(in-package :esrap)

(defmacro define-grammar (name &rest options)
  (let ((package-options (remove-if #'(lambda (opt) (eql (car opt) :inherit))
                                    options))
        (inherit (second (assoc :inherit options))))
    `(eval-when (:compile-toplevel :load-toplevel :execute)
       (let* ((grammar (defpackage ,name ,@options))
              (rules-symbol (intern "*RULES*" grammar)))
         (proclaim `(special ,rules-symbol))
         (setf (symbol-value rules-symbol)
               (if ,inherit
                   (alexandria:copy-hash-table (grammar-rules ,inherit))
                   (make-hash-table)))
         grammar))))

(defun grammar-rules (&optional (grammar *package*))
  (symbol-value (find-symbol "*RULES*" grammar)))

(defun find-rule-cell (symbol &optional (grammar *package*))
  (check-type symbol nonterminal)
  (gethash symbol (grammar-rules grammar)))

And etc.       

Now it can be used, for example, so:

(define-grammar #:cliki2.grammar
  (:use #:cl)
  (:inherit #:3bmd-grammar))

(in-package #:cliki2.grammar)

(defrule article-link ...)
(defrule person-link ...)
(defrule hyperspec-link ..)
(defrule category-link ..)
(defrule code-block)
(defrule category-list ..)

(defrule 3bmd-grammar:inline-extensions
    (or article-link
        person-link
        hyperspec-link
        category-link
        code-block
        category-list))
nikodemus commented 13 years ago

Uh, ok. Not quite what I had in mind.

Just be ready to change your code when RULES goes away. :)

nikodemus commented 13 years ago

Apropos: to clarify "not quite what I had in mind" -- I'm not going to pun grammar's to packages. Too many things can go wrong there, and the package namespace is already contested enough.

Other than that you have the basic shape of what it will look like in user code.

(defgrammar my-grammar (another-grammar-to-include) ...options?...)

(define-my-grammar-rule foo ...)

(define-my-grammar-rule bar ...)

For backwards compatibility there will be an ESRAP:GLOBAL-GRAMMAR, and DEFRULE will be an alias for DEFINE-GLOBAL-GRAMMAR-RULE.

archimag commented 13 years ago

Now it seems to me that the most right will be something like

(defparameter *my-grammar* (make-grammar))

(defrule my-fist-rule (...)
  (:lambda (list)
     ...)
  (:grammar *my-grammar*))

But it requires a lot of refactoring. If you like this variant, then I can try to do this job.

nikodemus commented 11 years ago

scymtum's grammar object implementation looks like the way forward.