Loris image directories

[lfarrell]

1. Does loris support multiple src_img_root settings or does everything need to be split out into sub-directories beneath one root?
2. Does it support displaying images from other servers, such a static assets server?

Thanks

[jpstroop]

Hiya, resolver.Resolver (that is, the class that is responsible for turning identifiers into paths to images and their formats) was designed to be overridden so that Loris could be adapted to different environments. Right now that just means changing the code here:

https://github.com/pulibrary/loris/blob/development/loris/resolver.py#L51-L82

to do whatever you need it to do. The two methods that need to be implemented are documented in the abstract class:

https://github.com/pulibrary/loris/blob/development/loris/resolver.py#L14-L48

So if you need to make a database call, or want to load the image over HTTP or whatever, that's fine, the code just needs to be written.

Note also that the Resolver constructor receives a config dictionary. That dictionary will automatically contain any keys/values that you put in the resolver.Resolver section of the config file, so it should be easy to pass in additional properties that resolver might need. -Jon

[lfarrell]

This is stupid question, but if I wanted to add in pulling in images via HTTP and serving them through Loris where would I put my overrides? I didn't want to mess around in the actual Loris files. Sorry, I've only used Python in the context of Django where it's usually pretty obvious where to put package overrides.

Thanks,

Dean

On Wed, Apr 2, 2014 at 3:59 PM, Jon Stroop notifications@github.com wrote:

Hiya, resolver.Resolver (that is, the class that is responsible for turning identifiers into paths to images and their formats) was designed to be overridden so that Loris could be adapted to different environments. Right now that just means changing the code here:

https://github.com/pulibrary/loris/blob/development/loris/resolver.py#L51-L82

to do whatever you need it to do. The two methods that need to be implemented are documented in the abstract class:

https://github.com/pulibrary/loris/blob/development/loris/resolver.py#L14-L48

So if you need to make a database call, or want to load the image over HTTP or whatever, that's fine, the code just needs to be written.

Note also that the Resolver constructor receives a config dictionary. That dictionary will automatically contain any keys/values that you put in the resolver.Resolver section of the config file, so it should be easy to pass in additional properties that resolver might need. -Jon

— Reply to this email directly or view it on GitHubhttps://github.com/pulibrary/loris/issues/58#issuecomment-39376147 .

[jpstroop]

Not a stupid question at all...and you're not gonna like my answer. :smirk:

I've been meaning to refactor this for a while, to make the name of the class that implements that resolver configurable so that it can be whatever, wherever you want on (as long as Python can find it). Right now, though, you do actually have to go into the loris code and replace the Resolver class in the resolver package. Just comment it out...it'll be OK.

If you've not done a lot of Python, I strongly, strongly recommend using Requests to do your HTTP work.

[lfarrell]

I had two more questions. Does Loris support authentication or would we need to to that through something like Apaches? Second, can you request a specific area of an image?

Thanks,

Dean

On Wed, Apr 23, 2014 at 12:10 PM, Jon Stroop notifications@github.comwrote:

Not a stupid question at all...and you're not gonna like my answer. [image: :smirk:]

I've been meaning to refactor this for a while, to make the name of the class that implements that resolver configurable so that it can be whatever, wherever you want on (as long as Python can find it). Right now, though, you do actually have to go into the loris code and replace the Resolver class in the resolver package. Just comment it out...it'll be OK.

If you've not done a lot of Python, I strongly, strongly recommend using Requests http://docs.python-requests.org/en/latest/ to do your HTTP work.

— Reply to this email directly or view it on GitHubhttps://github.com/pulibrary/loris/issues/58#issuecomment-41180804 .

[jpstroop]

Hiya, You would need to do auth in Apache or somewhere else upstream. IIIF is actively discussing an auth layer (one that isn't black and white, but may, for example, offer degraded access), and I've decided to hold off on any auth implementation at least until that work is done.

You can absolutely request an area of an image. The IIIF spec explains how and here's a random example from our production server. -Jon

[lfarrell]

Would you be amenable to the University of North Carolina Libraries adding in serving images from other servers and adding in some basic authorization and then contributing our code back to the Loris code base, pending a review on your end of course. I'm not sure what you have planned for Loris and if these are already on the roadmap.

Thanks,

Dean

[rlskoeser]

Are there any examples of extending/overriding the resolver?

It looks like the resolver returns a local file path and the format; that means an http resolver would need to pull the content and create a file somewhere locally, right? It seems like it might make sense to put it in the cache directory to take advantage of existing cache cleanup script, but it doesn't look like the resolver has access to that config.

I do see the SourceImageCachingResolver in the code, but can't tell that it's used anywhere, esp. since it requires an additional init parameter.

[jpstroop]

@rlskoeser This has been a topic for a while now, and your timing is very good, as @lfarrell and I had a call about it this morning! The diff above (0d3e1b1c72ee311ebb73537bd2eb6078b22d5839), which helps lay the groundwork for some stuff he's going to do, may be enough to help you get the gist of my intentions wrt making this easier, but I'll add some annotations to it, and then maybe you can ask again if you still have questions? Watch for annotations shortly.

[rlskoeser]

@jpstroop thanks for the info, this is helpful. So should I be working with the development code then?

It seems that loris relies pretty heavily on file extensions to determine image format (at least in the stable release - maybe this has been addressed in dev version?). Seems like this would already be problematic (I tested with a file that had a .jpeg instead of .jpg and got a 500 error), but will be even more of an issue for pulling images from urls. I can pass the format from the resolver by pulling it from the url response mimetype, but once the file is cached without a standard extension it seems like there will be a problem. Any thoughts of switching to python-magic and using mimetypes instead of extensions?

[jpstroop]

@rlskoeser Yes, I would recommend working on the development branch as it'll (soon--I didn't get to write a unit test today for the branch I mentioned above, so it's not been merged--hopefully this weekend) be the one that has a more flexible resolver configuration. One big caveat: the info.json syntax has changed significantly since the 1.1 release of the IIIF Image API. I haven't yet updated the code with OpenSeadragon (which tends to be Loris's gateway drug :smile:) to work with the IIIF 2.0 syntax, and I'm not likely to do so until we finalize the IIIF 2.0 release, which we're planning for late August or early September. The development branch is using the current draft 2.0 syntax.

Regarding file extensions...yes, it took me a long time to get comfortable with the idea of relying on three-character extensions instead of MIME/media types, but in the end, because that's what the IIIF API relies on, it seemed best to keep that consistent throughout the application and its APIs. In earlier versions I was using media types, but kept having to translate back and forth between that and, e.g. 'jpeg' or 'jpg', resulting in lots of extra switches. The solution was to at least make it possible to encapsulate all of that logic in one place--the resolver--rather than let it proliferate throughout the application.

So, sort of related to what I was saying yesterday, your identifier may be 761390 and you may have to look in some other database field the for the format info, and the value there may be jpeg or image/jpeg or THIS_IS_A_JPEG. As long as the resolver can turn that into a path to a file on the local filesystem and 'jpg', everything will work. Hopefully that's a reasonable tradeoff for the other stuff Loris gets you!

Have a nice weekend. - Jon

[jpstroop]

@rlskoeser, just FYI: #95

[jpstroop]

@rlskoeser unrelated to this ticket and it may not even apply to your use case, but since I mentioned it above... :smile:

@azaroth42 is working on a new TileSource for OpenSeadragon and it looks nearly complete (not on github yet, or I'd link). I suspect there will be IIIF 2.0-draft support on the master branch of OpenSeadragon within a week or so.

-Jon

[jpstroop]

@lfarrell this issue may have splintered into #95 and to some extent #98. Are you OK with closing, or is there still a separate issue here?

[lfarrell]

@jpstroop, sorry for the long delay. I went on vacation and then got caught up in other things. Closing this issue is fine

[lfarrell]

@rlskoeser I was having similar issues with extensions, as a lot of our images have this format https://url.edu/uuid:21a69f8b-028d-4a67-962b-e88ca16c73e8/IMAGE_JP2000 to access the remote image. I've got it working locally here at UNC, but it's pretty naive. It uses @jpstroop file extension check against a list of file types ['png', 'jpg','jpeg', 'jp2']. If no extension is found it defaults to jp2 and uses the uuid as the filename.