bnbalsamo
commented
6 years ago I've done rather a lot of thinking about this identifier business since our meeting, and I'd like to lay it out because I think the issue may be more nuanced than originally believed.
I believe we have two major ways we can view identifiers:
1) Hierarchical listings
mvol/0001/0002/0003 where it is possible to get a listing from mvol/0001 which includes 0002, and potentially 0001 and 0004 etc. mapped respectively to mvol/0001/0002/, mvol/0001/0001, mvol/0001/0004.
This assumption results in a sub decision that must be made, which will inform how I define the URL rules.
These two options can be roughly described as either using the URL as a proxy for the path, or URL encoding a path and passing it to an endpoint. Because URLs != paths in certain respects, these each provide different pros/cons
1.1) URLs in the form http://$HOST/mvol/0001/0002/0003/jpg
This assumption mandates hierarchical identifiers of the same depth for everything eg: each identifier must be a namespace followed by three subparts. The resulting flask URL rule and endpoint would look something like....
class NavDepthNamespace(Resource):
def get(self, namespace):
nav_path = pathlib.join(BLUEPRINT.config['OWNCLOUD_ROOT'], namespace)
# Go get and return a listing of subnodes for this path
class GetJpg(Resource):
def get(self, namespace, id_part1, id_part2, id_part3):
resource_path = pathlib.join(
BLUEPRINT.config['OWNCLOUD_ROOT'],
namespace,
id_part1,
id_part2,
id_part3
)
# Do things with the path, etc etc
API.add_resource(
GetJpg,
"/<string:namespace>/<string:id_part1>/<string:id_part2>/<string:id_part3>/jpg"
)This solution would also call for hardcoding endpoints similar to this, for navigability:
API.add_resource(
NavDepthNamespace,
"/<string:namespace>"
)
API.add_resource(
NavDepth1,
"/<string:namespace>/<string:id_part1>:
)
API.add_resource(
NavDepth2,
"/<string:namespace>/<string:id_part1>/<string:id_part2>"
)
API.add_resource(
NavDepth3,
"/<string:namespace>/<string:id_part1>/<string:id_part2>/<string:id_part3>"
)1.2) URLs in the form http://$HOST/mvol%2F0001%2F0002%2F0003/jpg
This assumption means that identifier "paths" can be arbitrary depths. It's implementation would look more like...
class Nav(Resource):
def get(self, quoted_identifier):
hierarchical_identifier = urllib.parse.unquote(quoted_identifier)
# Take the identifier hierarchy, navigate it, return child nodes
class GetJpg(Resource):
def get(self, quoted_identifier):
hierarchical_identifier = urllib.parse.unquote(identifier)
# Take the identifier, go find the thing at it, return/create and return the JPG
API.add_resource(
Nav,
"/<string:quoted_identifier>"
)
API.add_resource(
GetJpg,
"/<string:quoted_identifier>/jpg"
)2) Atomic identifiers
mvol/0001/0002/0003 is a single unit, mvol/0001/0002 is (most likely) an invalid resource identifier, or potentially another unrelated resource. In this case identifiers are not hierarchical, and therefore not intrinsically navigable.
For safeties sake I would again recommend URL escaping the identifiers for transit, though it wouldn't necessarily be required if we avoided certain characters. My examples will operate with the assumption that we would url encode them.
This results in the simplest implementation
class GetJpg(Resource):
def get(self, quoted_identifier):
identifier = urllib.parse.unquote(identifier)
# Take the identifier, go find the thing at it, return/create and return the JPG
API.add_resource(
GetJpg,
"/<string:quoted_identifier>/jpg"
)The big change here is because the identifiers are no longer navigable, and creating a flat listing of all identifiers would be too time consuming to do dynamically, @johnjung would be reliant on manifests created by @c-blair / kfeeney / kar8 in order to know what to populate interfaces with. These manifests would have to include every complete resource identifier. Extrapolation of further organizational meaning from the identifier would be left to the interface (eg, grouping by 0001/0002 or what have you).
This manifest would be very similar to the manifests we are using currently the validation workflows, and in fact could probably be used for the same purpose in this system.
Conclusion
In order of preference I'd pull for solutions... 1) 2 2) 1.2 3) 1.1
My preference for solution two is driven by a few factors.
1) It's the least functionality all rolled up into one tool.* 2) It makes the most sense for identifiers that don't currently have hierarchys, eg the finding aid material
- or material which doesn't make sense in the same hierarchy depth as mvols
- Code reuse 3) From The Zen of Python
- Explicit is better than implicit --> Explicit manifests are better than implicit path traversal
- Flat is better than nested --> No hierarchies, no hierarchy complication 4) We get to "double dip" on explicit manifests for good validation/completeness measures, rather than just validation of "what's there" 5) It's easily extensible out of band. If we wanted we could use an entirely separate system, driven by the same manifests that the interface may rely on, to provide hierarchical structure to identifiers. 6) It is the most loosely coupled implementation, relying very little on the "backdoor" knowledge that the files are on a file system in directories.
- This also has the nice side effect that it would be least disk intensive - making it probably the fastest of all the possible implementations
What are others thoughts on this? @c-blair @johnjung @verbalhanglider - We might also consider looping kefeeney, kar8, and jdartt on discussion of manifests, if we determine to go that route.
verbalhanglider
c-blair
Starting with this assumption:
mvol/0001/0002/0003 which would refer to mvol publication 1, volume 2, issue 3
Such that mvol/0001/0002/0003/pdf => return the PDF file /tiffs => return the tiffs for that issue /jejocr => return the OCR file that is needed by XTF to create the index needed by the interface /ocr => return OCR data generated by preservation department /jpg => return dynamically generated JPEG file
would retrieve relevant items from that issue.