XSL Transformation

[BalduinLandolt]

As you know, I'm very fond of this extension. One thing I'm still missing with it, though, is XSL transformations.

I know there are other extensions that can do XSL transformations for you. But I think it would be very neat to have it all in one place. And since this extension comes with Lemminx, which again comes with Xerces that can do XSL transformation, this shouldn't be too hard to do.
(I have been playing around with this extension, my fork can be found here. That pretty much does the trick for my personal use. But for a published extension it has the major downside of not only requiring java, but also a Saxon .jar to do the actual transformation. Again, your extension already ensures java, and it comes with Xerces.)

How do you guys feel about this?
Personally I'd love to see this feature. And once I have a bit more time (not anytime soon, I'm afraid), I'd gladly contribute to that.

(In fact I find it a bit strange that the LSP API does not have the concept of some sort of transformation... I could think of several use cases for that. And if that were the case, it would be extremely easy: The extension could request transformation from Lemminx, which could hand that over to Xerces.)

[angelozerr]

For XSL I think Saxon is better than Xerces. There are several licenes for Saxon so if we need to consume Saxon in LemMinX we must take care of Saxon license with EPL 2.0 (LemMinX license).

How do you guys feel about this?

If we see that there are a lot of people who are interested with this feature, let's go to work on it.

Personally I'd love to see this feature. And once I have a bit more time (not anytime soon, I'm afraid), I'd gladly contribute to that.

(In fact I find it a bit strange that the LSP API does not have the concept of some sort of transformation.

If we would like to integrate with LSP

• we could provide a CodeAction which generate a file with the result of apply of XML and XSL declared with stylesheet processing instruction.

• create a custom LSP request which returns the result of apply XML and XSL declared with stylesheet processing instruction.

[BalduinLandolt]

For XSL I think Saxon is better than Xerces. There are several licenes for Saxon so if we need to consume Saxon in LemMinX we must take care of Saxon license with EPL 2.0 (LemMinX license).

I'm curious, why is Saxon better for XSL?

I somehow was under the impression you were using Xerces and not Saxon because of the license.
In any case, Saxon-HE is certainly the only Saxon version that can be used, it is the only open license. But I don't know the two licenses go together...
In any case it could be interesting to have a setting to point to an external Saxon .jar, so people who need features of other Saxon version could still do that.

Using Saxon in Lemminx would of course make Lemminx a bit heavier, but if that's not a problem, why not.
Maybe it could even be used for XSD 1.1 (see #222 and #253 ) until Xerces has a version that supports XSD 1.1 on Maven.

If we see that there are a lot of people who are interested with this feature, let's go to work on it.

I'm probably not representative. But I do think that XSL is a usefull feature for a language support package. So... yes! :)

If we would like to integrate with LSP

* we could provide a CodeAction which generate a file with the result of apply of XML and XSL declared with stylesheet processing instruction.

* create a custom LSP request which returns the result of apply XML and XSL declared with stylesheet processing instruction.

It's of course up to you guys... But I think it would be better to go for the latter option. If it works elegantly, it could even be suggested as a new feature to the official LSP, once you have a working implementation.
I would consider the following options/parameters for the request:

• It wil of course need a reference to the XML file
• it should probably specify what kind of transformation it should be (i.e. XSL in our case)
• optionally it should be able to specify the output (i.e. XML to XML or XML to HTML or XML to CSV or XML to TXT, etc.). (maybe even the URI to the file where the output is stored?)
• optionally it should be able to specify the ruleset (i.e. URI to the stylesheet)

If the last two parameters are left undefined, it should check the processing instructions if there is a stylesheet declared.
Like this, it would allow for an option to associate XML files with transformation scenarios in the editor, whithout having to change the XML itself. (Oxygen XML has this feature, and it's very useful.)
Another thing that needs to be considered is if it should be able to handle multiple stylesheets; in that case it needs to be able to return multiple output documents.

[angelozerr]

For XSL I think Saxon is better than Xerces.

Sorry with my wrong comment. I would to say that Xerces doesn't support XSL transformation if I remember. It's Xalan which support that.

Saxon XSL processor seems having better performance than Xalan And see https://stackoverflow.com/questions/16455276/saxon-9-xslt-transformer-vs-xalan-2-7

I somehow was under the impression you were using Xerces and not Saxon because of the license.

Exactly. Saxon provides several product https://www.saxonica.com/products/products.xml, the only product that we can use in open source land is Saxon HE which supports only XST transformation. It

We need for instance support for XML Schema 1.1 but it requires Saxon EE which is a paid product.

Using Saxon in Lemminx would of course make Lemminx a bit heavier, but if that's not a problem, why not.

I think but it will require SAxon EE which is a paid product.

But I think it would be better to go for the latter option.

Yes it was my idea too because you can manage complex usecases (ex: fill parameters, with a vscode command palette, etc).

The question is how to show the result of XSL transformation?

[angelozerr]

I'm probably not representative. But I do think that XSL is a usefull feature for a language support package.

We will see if other people could be really interested with XSL. For the moment we have few user issues for XSL.

[BalduinLandolt]

I see, in that case, Saxon is certainly the better choice.

the only product that we can use in open source land is Saxon HE which supports only XST transformation

I wasn't aware that XSD is not in Saxon HE, but you're right. Then Saxon would only be for XSLT, but that's something already.
(By the way: I love "open source land"! ^^)

But a combination of Saxon (for XSLT) and Xerces (for all other things) should work, right? And XSD 1.1 would just have to wait, I guess...

Yes it was my idea too because you can manage complex usecases (ex: fill parameters, with a vscode command palette, etc).

Sounds reasonable.

The question is how to show the result of XSL transformation?

I'm not yet familiar enough with LSP, to know what the normal way would be: Would it return the result as a string, or just a URI to a file in the file system?
If it's just a string, one could go the same way as eg. this or this extension: They simply open a new window in the editor and copy the result in this window. Then they leave saving it to the user. It's not the nicest way, but it works and is simple to implement.
If it returns a URI, i'd suggest the Request already contains the URI where to save the result (then vs code can do fancy stuff with settings, where to save the output), Lemminx would then save the result to the specified file, and if transformation and writing the output to the file were successfull, return the URI of the result file. Then it would be up to vs code (as a setting), to open the result either in vs code, in the system explorer, or in another application (e.g. Browser, if result is HTML).

Does that make sense?

[BalduinLandolt]

I'm probably not representative. But I do think that XSL is a usefull feature for a language support package.

We will see if other people could be really interested with XSL. For the moment we have few user issues for XSL.

Sounds good. :)

[angelozerr]

I'm not yet familiar enough with LSP, to know what the normal way would be: Would it return the result as a string, or just a URI to a file in the file system?

I mean what is the UI to manage XSL transformation result. The basic idea is to generate a file, but preview could be very interesting. I have 3 editors, one for XML input, one for XSL and one for preview. When I type something (without saving) in the XML editor or XSL editor, the preview editor is refreshed. I don't know if vscode could provide this kind of virtual editor.

[BalduinLandolt]

That sounds like something that could be resource intensive, to do a transformation on every key stroke. But I guess it could be an option, on which event (key stroke, save, etc.) this should happen.

Also you might get problems with errors, because when typing, most of the time, the xml is not well-formed. But I guess the language server already handles this problem, so that should work too.

But in general this sounds like a really cool idea.

An entirely other question though:

We will see if other people could be really interested with XSL.

How do you determine if people find that a useful feature?

[bmix]

I'm curious, why is Saxon better for XSL?

Saxon implements the latest (3.0) XSL and XPath (3.1) specification, while Xalan supports only 1.0 of both, which is 20 years old technology.

Version 2 of the specs introduced, among other things:

• A serious change to the data-model, for example the concept of node-set has been replaced with sequences (everything is now a sequence, btw)
• A lot of new functions have been added, which count now approx. 200 (also counting the XPath 3.x functions). See here for a list, that also let's you set the XPath version for comparison.
• Grouping instructions (<xsl:for-each-group>)
• Regular expressions with special character classes for XML names (NCName, etc.)
• Date-and-time arithmetic and formatting
• A wider range of data types
• User-written functions in XSLT

Version 3 added:

• Support for JSON, including maps and arrays
• Higher-order functions
• Support for HTML5
• Inline text templates: <p>Written by {@author} on {@date}</p>
• Separately-compiled XSLT packages

In any case, Saxon-HE is certainly the only Saxon version that can be used, it is the only open license. But I don't know the two licenses go together...

There is Saxon-JS2 now. It runs either in the browser or on NodeJS. Not sure, if the NodeJS variant would be of any use here, but it may well be, on the side of VSCode, outside of Lemminx, till SaxonHE gets supported.

However, SaxonHE has become much less restrictive with release 10. Now it also supports Higher Order Functions, both user definable and the new functions from the W3C XPath spec (map/reduce, etc.).

In any case it could be interesting to have a setting to point to an external Saxon .jar, so people who need features of other Saxon version could still do that.

Yes, that would be nice for those, who licensed SaxonPE or SaxonEE.

Maybe it could even be used for XSD 1.1 (see #222 and #253 ) until Xerces has a version that supports XSD 1.1 on Maven.

...but only the (expensive) SaxonEE does XML Schema...

• It wil of course need a reference to the XML file

Not in all cases! XSL-T supports the notion of a named template with the name xsl:initial-template which allows us to run XSL-T without an input XML file (for example, if you want to create a new file from within the stylesheet)

• it should probably specify what kind of transformation it should be (i.e. XSL in our case)

What would be the alternatives?

• optionally it should be able to specify the output (i.e. XML to XML or XML to HTML or XML to CSV or XML to TXT, etc.). (maybe even the URI to the file where the output is stored?)

With XSL-T >2 that is done internally via the xsl:output directive.

• optionally it should be able to specify the ruleset (i.e. URI to the stylesheet)

You mean automatically associate stylesheets with XML files? That seems to be a job for XML Catalogs.

If the last two parameters are left undefined, it should check the processing instructions if there is a stylesheet declared.

Isn't this something an XSL processor does automatically?

Like this, it would allow for an option to associate XML files with transformation scenarios in the editor, whithout having to change the XML itself. (Oxygen XML has this feature, and it's very useful.)

I agree. This is especially useful if one wants to add an XSL-FO processing step. Ideally, something like this would be done by XProc and that could be represented in a scenario dialog.

Another thing that needs to be considered is if it should be able to handle multiple stylesheets; in that case it needs to be able to return multiple output documents.

You mean in a pipeline? XProc comes to mind... (see above) ;-)

[bmix]

Saxon HE which supports only XST transformation. It

And XQuery! (see https://www.saxonica.com/products/feature-matrix-10.xml)

The question is how to show the result of XSL transformation?

With the HTML live view if it is transformed to HTML, otherwise in a read only view for XML or by loading the result into a new editor tab, maybe. It would be interesting, to be able to apply transformations on the currently shown document live. This would enhance the editor with refactoring and XML aware search and replace possibilities for XML documents.

I would advise against a "live update" feature for XSL transforms. As @BalduinLandolt mentioned, that'd be a resource hog and complicate things because of well formedness. In general, this is not needed for XSL development.

[bmix]

We will see if other people could be really interested with XSL. For the moment we have few user issues for XSL.

Maybe, because there is no XSL capability as of yet? ;-) Also, XML people tend to be introverts ;-)) I am a heavy XML user, so I am very interested in this project, but I can see, that it is a huge task and I think your progress is very nice. No need to rush.

For the future, of course, I would like to see XSL-T and XQuery integration, but currently I can do this with build scripts or, what I typically do for XQuery and XSLT development, with BaseX IDE and oXygenXML. Stability is more important!

If the Lemminx server could be extended via Javascript, that would be nice, however... Though, Nashorn has been abandoned and everybody prepares for GraalVM, as it seems.

[angelozerr]

@bmix thanks a lot for your great feedback.

If the Lemminx server could be extended via Javascript, that would be nice, however... Though, Nashorn has been abandoned and everybody prepares for GraalVM, as it seems.

We would like to avoid doing that. SaxonJS is not a solution for us, Saxon-HE is the only distribution that we can use it.

If you want to consume SaxonJS, it should be done directly inside vscode extension (without consuming a language server like we did with LemMinx that we can consume in a lot of IDE like Eclipse IDE).

It seems that https://marketplace.visualstudio.com/items?itemName=deltaxml.xslt-xpath consumes SaxonJS. What do you think about this project?

XSL support should be nice, but I would like to avoid working on features that other extension support very well. For instance I think we should work on RelaxNG support whoch doesn't exist.

[bmix]

@bmix thanks a lot for your great feedback.

If the Lemminx server could be extended via Javascript, that would be nice, however... Though, Nashorn has been abandoned and everybody prepares for GraalVM, as it seems.

We would like to avoid doing that. SaxonJS is not a solution for us, Saxon-HE is the only distribution that we can use it.

I couldn't agree more! Since your code is Java, it just makes sense to use the RealThing™. My remark about extending Lemminx with Javascript was addressing another issue: Writing Lemminx Extensions with Javascript, instead of Java.

It seems that marketplace.visualstudio.com/items?itemName=deltaxml.xslt-xpath consumes SaxonJS. What do you think about this project?

I didn't know about this project. I will test it immediately. Looks pretty good, thanks for pointing that out!

XSL support should be nice, but I would like to avoid working on features that other extension support very well. For instance I think we should work on RelaxNG support whoch doesn't exist.

I have no issues with using separate packages, except for incompatibilities, that may arise or missing XML features, that I have here, but not there...

Getting the XML editing experience right also means, to implement a lot of surrounding specs need to be implemented (XML Catalogs, different schema languages, XML editing features). Should any major features be added later these would be needed anyway, so I understand your focus.

Thanks for the reply, @angelozerr.

[angelozerr]

@bmix is Saxon-HE is not enough to support correctly XSLT? Please note LemMinx is extensable so you can add your own extension and consume it in vscode-xml.

If you are interested, please read https://github.com/eclipse/lemminx/blob/master/docs/LemMinX-Extensions.md

An idea is that (if you are interested) to develop your XSLT LemMinx extension based on SAXON-HE and we could integrate it in LemMinx extensions https://github.com/eclipse/lemminx/tree/master/org.eclipse.lemminx/src/main/java/org/eclipse/lemminx/extensions

What do you think about that?

I didn't know about this project. I will test it immediately. Looks pretty good, thanks for pointing that out!

Please give me feedback about this extension, thanks!

Getting the XML editing experience right also means, to implement a lot of surrounding specs need to be implemented (XML Catalogs, different schema languages, XML editing features).

LemMinx supports that and we provided advanced features like CodeLens in XSD, DTD to help developping XSD, DTD.

Should any major features be added later these would be needed anyway, so I understand your focus.

I'm aware to support any XML features like XSL and perhaps XQuery in the future but the goal today is to provides all important features for XML editing. I think we start to cover it, but now the goal is to fixing bugs and help newbie user to bind XSD, DTD to xml (we have a lot of issues about that). For instance see my current work to display referenced grammar in the outline https://github.com/eclipse/lemminx/pull/895#issuecomment-700089829

[GitMensch]

As the question came up about "is someone interested in XSL transformation" - I definitely am, to be able to replace external tools currently used for it. I do also like the point of the preview window, but I think it should not update on each keystroke, a command to open/update the preview window and in there a refresh button to update it would be fine.

@bmix What was the result of your testing of deltaxml.xslt-xpath and its working together with this extension?

Note: XML->XLS auto-preview in a browser does not work because of security policy issues see https://github.com/antfu/vscode-browse-lite/issues/10 - if this is solved then the browser preview case would be solved (not useful for other transformations, but at least for this common case).

[angelozerr]

As the question came up about "is someone interested in XSL transformation" - I definitely am, to be able to replace external tools currently used for it.

Thanks for your feedback.

do also like the point of the preview window, but I think it should not update on each keystroke, a command to open/update the preview window and in there a refresh button to update it would be fine.

Note: XML->XLS auto-preview in a browser does not work because of security policy issues see antfu/vscode-browse-lite#10 - if this is solved then the browser preview case would be solved (not useful for other transformations, but at least for this common case).

I think the better mean is to develop a webview to preview xsl like https://marketplace.visualstudio.com/items?itemName=bierner.markdown-mermaid does

If anybody could be interested to write a lemMinx extension for XSL, don't hesitate to ping me, I could give you some directives. Thanks! We could start like this and if we will see if it works good we could integrate it inside LemMinx to benefit with binary support (no need to install Java).