Make bundle:// URIs work with px:copy

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Try to copy a file with a bundle:// URI using px:copy

What is the expected output? What do you see instead?
It doesn't resolve properly

This is useful if you want to bundle static auxilliary files with a module.

Mailing list discussion: 
https://groups.google.com/forum/?fromgroups=#!topic/daisy-pipeline-dev/f6_Y4W2WI
Nk

Original issue reported on code.google.com by josteinaj@gmail.com on 21 Nov 2012 at 4:28

[GoogleCodeExporter]

We shouldn't rely on bundle:// uris, it's unsafe for our module model. I'd 
propose to add a maven plugin, ant step or sth of the like to create a 
xmlcatalog of a directory with static resources. 
New issue on http://code.google.com/p/daisy-pipeline/issues/detail?id=284

Original comment by capitan....@gmail.com on 26 Mar 2013 at 6:34

• Changed state: WontFix

[GoogleCodeExporter]

In the case of binary files, would having the static resources in a xmlcatalog 
help? If you try to load a file with:

<p:data href="http://some/file/in/catalog" content-type="binary/octet-stream"/>

it is not resolved correctly. Instead you get an HTML file back (it tries to 
get the file from the web and returns some 'requested page not found' page).

<p:load> (and <p:document>?) work with http:// URIs, but only for XML files.

PS: For copying text files with bundle: URIs, I'm currently using a hack that 
involves p:xinclude (see 
https://github.com/daisy-consortium/pipeline-mod-braille/blob/master/liblouis/li
blouis-formatter/src/main/resources/xml/xproc/utils/copy-text-file.xpl). 
Unfortunately this solutions doesn't apply for binary data.

Original comment by bertfrees on 17 Apr 2013 at 12:08

[GoogleCodeExporter]

I tried to make a little overview of what currently works and what not:

https://code.google.com/p/daisy-pipeline/wiki/XProcFileOperations 

Original comment by bertfrees on 17 Apr 2013 at 1:16

[GoogleCodeExporter]

@bertfrees: neat!

@capitan.cambio: Why again was supporting bundle:// URIs unsafe?

Original comment by josteinaj@gmail.com on 17 Apr 2013 at 1:25

[GoogleCodeExporter]

The one of the basic ideas of having scripts and modules is to encapsulate
functionality and resources, ( the same principles of OO apply ). So if you
have a xpl file that you don't want expose... just don't include it on the
catalog file.

I still don't see the necessity of going such low level as bundle:// uris,
(they may change from execution to execution btw).  Why should XProc and
the scripts know about osgi or the fwk internals? .

@Bert Can give some example of how p:load doesnt work with text files? the
file is in the same module, or somewhere else?

Original comment by capitan....@gmail.com on 17 Apr 2013 at 3:05

[GoogleCodeExporter]

Right, that seems reasonable. I agree.

So the alternative we'll provide for bundling resources should let authors:
- declare files and directories that should be exposed with public URIs
- list all bundled resources in a module
- load any bundled resource from a module
- copy any bundled resource from a module

Right?

Original comment by josteinaj@gmail.com on 17 Apr 2013 at 3:31

[GoogleCodeExporter]

The idea with the issue
http://code.google.com/p/daisy-pipeline/issues/detail?id=284 is just having
a predefined directory for static resources (css, imgs, you name it) and
fill the catalog from it.

public steps have to be inserted in the catalog by hand, I guess we could
have an attribute declaring which ones are public and which aren't.

The list of the public bundled resources is the catalog t self

as for:
- load any bundled resource from a module: no, unless it's declared in the
catalog
- copy any bundled resource from a module: idem.

Original comment by capitan....@gmail.com on 17 Apr 2013 at 3:51

[GoogleCodeExporter]

yeah, by "any bundled resource" I meant only those declared as public (I 
should've phrased myself more clearly).

Original comment by josteinaj@gmail.com on 17 Apr 2013 at 3:56

[GoogleCodeExporter]

@Javi: There must be some terrible misunderstanding, because you're basically 
saying everything is fine the way it is, while I'm struggling looking for 
workarounds to get things working. You must know something I don't know :)

So, there's two separate things here we're talking about: public resources and 
private resources.

The original question was about private resources. I always considered bundle 
URIs and relative URIs inside a bundle/JAR the same thing. (Maybe that's 
wrong?) If you say for example:

<p:variable name="some-bundle-uri" select="resolve-uri('some/relative/path')">
  <p:document href="this-file.xpl"/>
</p:variable>

you get a bundle URI. And the following works:

<p:load>
  <p:with-option name="href" select="$some-bundle-uri"
</p:load>

I never thought of using bundle URIs as a problem for encapsulation. The way I 
see it bundle URIs are always private. How would you get hold of a bundle URI 
of a resource in another bundle?

So, I guess my original question was: if p:load can handle bundle URIs & 
relative URIs, why can't p:data, px:copy etc.? At the moment there's no easy 
way that I know of to access binary resources or text resources inside the JAR.

My comment #2 was about public resources (and related your work on Issue 284): 
What's the use of having non-XML resources in a xmlcatalog if you can't access 
them? Because p:data, px:copy, etc. don't work for catalog URIs either. So 
basically it boils down to the same problem as with private resources.

I'm a bit confused about the example you ask of how p:load doesn't work with 
text files: p:load never works with text files, right? 
(http://www.w3.org/TR/xproc/#c.load)

I'm on a weekend in Belgium, so I should stop writing now. But if you want a 
more concrete example of why I need non-XML resources in my bundles, I will 
give you two next week.

Original comment by bertfrees on 18 Apr 2013 at 11:18

[GoogleCodeExporter]

@bert
yes it was missunderstanding as I thought when you where talking bundle
uri's you where talking about just using bundle://13/blablablah/file.xml
for accessing any file provided by any bundle.  my bad.

This, it sums it up
"So, I guess my original question was: if p:load can handle bundle URIs &
relative URIs, why can't p:data, px:copy etc."

In the case of copy it makes sense because:
https://github.com/ndw/xmlcalabash1/blob/master/src/com/xmlcalabash/extensions/f
ileutils/Copy.java#L63
And this step was thought as part of some fs utils, not to copy "resources".

I guess the cleanest solution would be implement our own copy (sth like
copy-resource) step which accepts any kind of schemes, not just "file:".

for p:data is a little bit less clear why it doesn't work, if you have a
concrete example that would be great.

me referring to p:load in the previous message is a mistake, it should
refer to p:data.

So, overall, if this is what you meant I'll close this issue and open
another for a custom step.

Original comment by capitan....@gmail.com on 22 Apr 2013 at 10:53

[GoogleCodeExporter]

Okay, px:copy-resource makes sense.

An example where p:data doesn't work is in the dtbook-to-odt module I'm working 
on now. I want to have an OpenOffice template (.ott file) in the bundle that I 
can use as a start for the resulting .odt file. I also want the user to be able 
to supply his own template, but there should be a default one included in the 
bundle. The .ott file is a zip, but because px:unzip only works for file:/ 
URIs, it has to be extracted to somewhere on disk first. This is why I needed 
px:copy or p:data for bundle: URIs.

Another example where px:copy-resource would be great is in liblouis-formatter, 
where I am extracting configuration files (text format) for liblouisutdml from 
the bundle. liblouisutdml requires that these configuration files are stored on 
disk. Right now I'm using a hack:

https://github.com/daisy-consortium/pipeline-mod-braille/blob/master/liblouis/li
blouis-formatter/src/main/resources/xml/xproc/attach-liblouis-config.xpl
https://github.com/daisy-consortium/pipeline-mod-braille/blob/master/liblouis/li
blouis-formatter/src/main/resources/xml/xproc/utils/copy-text-file.xpl

Thanks!

Original comment by bertfrees on 23 Apr 2013 at 2:23

[GoogleCodeExporter]

In the first example, I tried to do this but it (the p:data part) failed:

<p:store method="text" cx:decode="true">
    <p:input port="source">
        <p:data href="http://www.sbs.ch/pipeline/modules/dtbook-to-odt/dtbook-to-odt.xpl" content-type="binary/octet-stream"/>
    </p:input>
    <p:with-option name="href" select="resolve-uri('dtbook-to-odt.xpl', $temp-dir)"/>
</p:store>

Original comment by bertfrees on 23 Apr 2013 at 2:26

[GoogleCodeExporter]

And double-checked that 
http://www.sbs.ch/pipeline/modules/dtbook-to-odt/dtbook-to-odt.xpl was in the 
catalog.xml

Original comment by bertfrees on 23 Apr 2013 at 2:42