Open thomaspurchas opened 1 month ago
The underlying issue here is that file:
URIs are hierarchical; it's invalid for a file URI to not have a hierarchical path part (a path that starts with /
); see RFC-8089.
Right now, read("file:foo.txt")
will cause Pkl to read foo.txt
relative to the process CWD, but file:foo.txt
is not a valid file URI. This might be changed to a throw in the future. For similar reasons, read*("file:*")
is also not supported, and would also be changed to a throw.
When globbing using a file scheme, the path part must be a hierarchical path. For example, read*("file:/path/to/*.txt")
(or read("file:///path/to/*.txt"
for a more canonical representation; file URIs typically have a host section.)
In that situation, how are we meant to use read()
with a relative path, in particular, how can we use read()
with a relative in the REPL where the schema is mandatory?
Also these nuances aren't described in the docs. The docs mention the file:
scheme, and under the Globbed Read section provides examples of using read
without the file:
scheme. But I can't see anywhere that explicitly states relative reads shouldn't use the file:
scheme.
Also for the read
API, do we have to stick so closely to the RFC? Obviously we should correctly handle any RFC-8089 valid URI. But there's no reason we can't also accept technically invalid URI, but where the intent is still obvious and useful (e.g. using the file:
scheme with a relative path)
I don't see a strong need for relative paths with file
URIs. If this is declared from within a file-based module, it should use a relative path without the scheme part. If declared within a non file-based module (e.g. from a package), what should the path be relative to? It can't be relative to the enclosing module, and also shouldn't be relative to the CWD, because we want imports to be statically analyzable.
In particular, how can we use read() with a relative in the REPL where the schema is mandatory
I think this is a tangential but definitely a problem. Ideally, you should be able to use relative file imports from within the REPL.
That sounds reasonable. I guess then my issue is that docs aren't very clear. It's not obvious that using read()
with file:
and without are different. It also creates a slightly odd situation where file reading has a "special" status, in that it supports relative reads is you don't use the file:
scheme, but no other scheme has that capability. Although it doesn't really make sense for other schemes to support relative paths in any situation, it still creates a slightly inconsistent read()
API, that's a little surprising.
Agree that we should improve our docs here. We have a section on relative paths, but it's easily missed and a common source of confusion.
The handling of files reads using
read*
is a bit broken at the moment. Behaviour changes depending on whether or not thefile:
schema is used, and behaviour in apkl
vs the REPL doesn't always seem consistent. Additionally there are a number of simple glob patterns that causeNullPointerExceptions
s or similar incorrect behaviour.Pkl REPL:
This error is is caused by the unsafe assumption that
.getPath
always returns a value. We should probably be using. getSchemeSpecificPart
instead.Evaling the following will work
but not (addition of the
file:
schema)and running
read*("**/*.txt")
in the REPL results in