Implement passing of external variables

[hanshuebner]

XQuery allows the declaration of query parameters by the way of variables declared as "extern". We need to be able to set such variables when evaluating an XQuery buffer. My proposal would be:

• When an XQuery buffer is evaluated, cider-any-uruk sends the query for analysis to a Clojure handler
• The Clojure handler analyzes the query and returns the names of the query parameters
• cider-any-uruk asks the user to input the values for each parameter. Previously entered parameter values should be remembered and be used by default. If the user wants to re-set the parameters, they need to use a prefix argument to C-c C-c (or something similar). Point being: By default, the XQuery should be evaluated with the previously entered parameters
• cider-any-uruk evaluates the query, passing the parameters as function arguments to the handler.

[proofit404]

This issue is relatively tricky to do.

I understand how important it is to you.

Most complex part of it is probably XQuery document parsing. We need to get list of external variable names. Currently I don't know how to do this correctly and this question require additional research.

Before that you can use cider-any-uruk-variables. This is buffer local variables which must hold plist of necessary external variable name to value mapping.

For example, XQuery document written below with file local commends will be successfully render to <foo>Hello</foo>.

declare namespace bk = "http://ww.marklogic.com/ns/gs-books";
declare variable $term as xs:string+ external;

<foo>{ $term }</foo>

(: Local Variables: :)
(: cider-any-uruk-variables: (:term "Hello") :)
(: End: :)

Without this variable evaluation will fail.

I hope that this is useful until I implement interactive variables input.

[hanshuebner]

You don't need to deal with the parsing of XQuery, we will take care of that. We just need a clojure hook that is called before an XQuery is evaluated and that returns a list of variables which the user would then asked to set.

[hanshuebner]

I'm trying your intermediate solution using this buffer:

xquery version '1.0-ml';

declare variable $term as xs:string+ external;

1;2;3;doc(concat('/provider/11.xml'));$term

(: Local Variables: :)
(: cider-any-uruk-variables: (:term "Hello") :)
(: End: :)

The local variables are not recognized, though:

1. Unhandled java.lang.Exception
   com.marklogic.xcc.exceptions.XQueryException: XDMP-UNDVAR:
   (err:XPST0008) Undefined variable $term [Session: user=restuser,
   cb=mpr [ContentSource: user={none}, cb={none} [provider:
   address=localhost/127.0.0.1:8000, pool=1/64]]] [Client: XCC/8.0-5,
   Server: XDBC/8.0-4.2] in /eval, on line 5 expr:

Also, the fact that XQuery comments nest does not play well with using plists for variables in file-local variables, as you've already discovered. Using strings instead of keywords ("term" "Hello") should work, however.

[hanshuebner]

Don't worry too much about this, I'll debug and also factor out the Clojure code so that it is easier to work with. It would be great if you could implement the emacs side of setting variables in the cider-any-ukuk-variables plist.

[proofit404]

Hmmm. I'm not certain but it seems like local buffers variables are recognized on the Emacs side but not work properly with uruk. Example which you provide constructs following Clojure form:

(let [db {:uri "xdbc://localhost:8889/" :user "proofit404" :password "<secret>" :content-base nil}]
  (with-open [session (uruk/create-session db)]
    (doall (map str (uruk/execute-xquery
                     session
                     ;; NOTE: uruk doesn't work with nested comment literals
                     (clojure.string/replace
                      "xquery version '1.0-ml';

declare variable $term as xs:string+ external;

1;2;3;doc(concat('/provider/11.xml'));$term

(: Local Variables: :)
(: cider-any-uruk-variables: (:term \"Hello\") :)
(: End: :)"
                      #"\(: cider-any-uruk-variables: .* :\)"
                      "")
                     {:variables {:term "Hello"}})))))

This form evaluation ends up in following exception:

XQueryException com.marklogic.xcc.exceptions.XQueryException: XDMP-UNDVAR: (err:XPST0008) Undefined variable $term
 [Session: user=proofit404, cb={default} [ContentSource: user={none}, cb={none} [provider: address=localhost/127.0.0.1:8889, pool=1/64]]]
 [Client: XCC/8.0-5, Server: XDBC/8.0-5.5]
on line 5
expr:    com.marklogic.xcc.impl.handlers.ServerExceptionHandler.handleResponse (ServerExceptionHandler.java:34)

But if we keep $term variable in this string then it will works as expected:

(let [db {:uri "xdbc://localhost:8889/" :user "proofit404" :password "<secret>" :content-base nil}]
  (with-open [session (uruk/create-session db)]
    (doall (map str (uruk/execute-xquery
                     session
                     ;; NOTE: uruk doesn't work with nested comment literals
                     (clojure.string/replace
                      "xquery version '1.0-ml';

declare variable $term as xs:string+ external;

$term

(: Local Variables: :)
(: cider-any-uruk-variables: (:term \"Hello\") :)
(: End: :)"
                      #"\(: cider-any-uruk-variables: .* :\)"
                      "")
                     {:variables {:term "Hello"}})))))

It is possible to use ("term" "Hello") for variables map right now, but I think it is necessary to keep replace string mechanism. You may want to pass nested maps. For example as variable type specification:

(: cider-any-uruk-variables: ("term" (:value 1 :type :xs-integer)) :)

[proofit404]

I try to implement desired feature in it whole use case.

The first thing I add to cider-any is callback chains. If backend function return cons of the form ("(clojure form)" . backend-command) then cider-any will schedule another form evaluation and apply backend to it result with different command. I'm not sure that this technique has its final form so I keep it in separate branch until we agree on it API. Please check callback-chain branch.

This approach was used in variables extraction mechanism. When you heat C-c C-c it evaluate clojure form which tries to extract variables list. Then it asks user for missed variables values and execute xquery with this variables substitution.

For example we have following xquery document:

declare namespace bk = "http://www.marklogic.com/ns/gs-books";
declare variable $term as xs:string+ external;
declare variable $my-variable as xs:integer external;

<abc>
  <foo>{ $term }</foo>
  <bar>{ $my-variable }</bar>
</abc>

When you heat C-c C-c it ask you to enter term variable value. It must be meaningful S-expression. Type "Hello" including double quotes around string. If you type Hello or hello it will be interpreted as a symbol or a class name. Then cider-any will ask you for my-variable value. Type (:value 1 :type :xs-integer). This will make variable map according to uruk documentation for passing non string external variables.

Then you will see rendered document with values you enter. On second C-c C-c heat you will see document render without asking for values. If you add new variable to the document it will ask you only for new one. If you heat C-u C-c C-c it will ask you for all values again.

Note that strings are used in variable mapping as keys. So if you store all or part of variables in file local vars use strings as a key.

(: Local Variables: :)
(: cider-any-uruk-variables: ("term" "Hello") :)
(: End: :)

Variable extraction is done by cider-any-uruk-parse-variables-form function. Edit form it return to extract variable list with parser instead of basic regex.

Also I'm waiting for your comments on the problem described in comment above.

[hanshuebner]

I think we can close this issue for now. I have found that it is possible to set a default for external variables by appending an assignment (:= "foo") to its declaration, which is sufficient for interactive development.

On Sun, Aug 7, 2016 at 6:42 PM, Malyshev Artem notifications@github.com wrote:

I try to implement desired feature in it whole use case.

The first thing I add to cider-any is callback chains. If backend function return cons of the form ("(clojure form)" . backend-command) then cider-any will schedule another form evaluation and apply backend to it result with different command. I'm not sure that this technique has its final form so I keep it in separate branch until we agree on it API. Please check callback-chain branch.

This approach was used in variables extraction mechanism. When you heat C-c C-c it evaluate clojure form which tries to extract variables list. Then it asks user for missed variables values and execute xquery with this variables substitution.

For example we have following xquery document:

declare namespace bk = "http://www.marklogic.com/ns/gs-books";declare variable $term as xs:string+ external;declare variable $my-variable as xs:integer external;

{ $term } { $my-variable }

When you heat C-c C-c it ask you to enter term variable value. It must be meaningful S-expression. Type "Hello" including double quotes around string. If you type Hello or hello it will be interpreted as a symbol or a class name. Then cider-any will ask you for my-variable value. Type (:value 1 :type :xs-integer). This will make variable map according to uruk documentation for passing non string external variables.

Then you will see rendered document with values you enter. On second C-c C-c heat you will see document render without asking for values. If you add new variable to the document it will ask you only for new one. If you heat C-u C-c C-c it will ask you for all values again.

Note that strings are used in variable mapping as keys. So if you store all or part of variables in file local vars use strings as a key.

(: Local Variables: :)(: cider-any-uruk-variables: ("term" "Hello") :)(: End: :)

Variable extraction is done by cider-any-uruk-parse-variables-form function. Edit form it return to extract variable list with parser instead of basic regex.

Also I'm waiting for your comments on the problem described in comment above.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/xquery-mode/cider-any/issues/4#issuecomment-238092965, or mute the thread https://github.com/notifications/unsubscribe-auth/AAGoz38oFwrjAV1mua8nPy-2ZDk6bpmRks5qdgsLgaJpZM4JahuV .

LambdaWerk GmbH Oranienburger Straße 87/89 10178 Berlin Phone: +49 30 555 7335 0 Fax: +49 30 555 7335 99

HRB 169991 B Amtsgericht Charlottenburg USt-ID: DE301399951 Geschäftsführer: Hans Hübner

http://lambdawerk.com/

[proofit404]

No problem. But I prefer to keep callback chain branch. Just for the case if we decide to give it another try.

[proofit404]

Is it necessary to remove local variables comment support?

[hanshuebner]

I think local "variables from comments" can go.

[proofit404]

Local variables support was removed in the master branch.