Have webapi send JWT to Sipi for auth

[benjamingeer]

The Knora API server can use JWT to authorise a user to upload files to Sipi, as well as to view files stored in Sipi.

Parts:

• 1165

• 1166

• 1169

[benjamingeer]

It seems that JWT isn't going to work, because of HTML img tags, so it looks like Sipi will need to make a request to the Knora API server for this.

[subotic]

relevant: #171

[subotic]

As I understand it, this is only a problem in Salsah 1.5, where the img tag is used. In Salsah 2.0 this shouldn't be a problem. I will talk to @lrosenth about this.

[lrosenth]

I think SIPI is accepting webtokens as cookie or as URL option (don't remember the "keyword" to be used...). May be this can be used for Salsah 1.5 bay adding the token to the URL in the img-ag.

Sent from my iPad

On 9 Aug 2017, at 11:00, Ivan Subotic notifications@github.com<mailto:notifications@github.com> wrote:

As I understand it, this is only a problem in Salsah 1.5, where the img tag is used. In Salsah 2.0 this shouldn't be a problem. I will talk to @lrosenthhttps://github.com/lrosenth about this.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/dhlab-basel/Knora/issues/520#issuecomment-321281868, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AFN9zARfe9YfP9lZ6GnhJyzky3OlQvBHks5sWclZgaJpZM4NrPjd.

[subotic]

May be this can be used for Salsah 1.5 bay adding the token to the URL in the img-tag.

Yes, this is what I also had in mind. We could generate so called access tokens and add them to every image URL. Those access tokens would only be valid for one particular image and only for a short amount of time. The pre-flight script in SIPI could then check the validity of the token and give access, all without the need to contact webapi.

[benjamingeer]

Just talked to Lukas about this again. Here's the problem:

Putting a JWT in an image URL is fine for SALSAH 1.5, but it won't work for a IIIF viewer. The IIIF auth standard expects to get a JWT in a cookie or in an HTTP auth header. There's no way to set an auth header for an <img> tag, so the cookie is the only option. But we can't make one giant cookie containing the user's permissions on all images. We can make a cookie that says who the user is, but then Sipi still has to make a request to Knora to ask for the user's permissions on a particular image. And then we're back where we started.

So my question is this: If the purpose of using JWT is to avoid the Sipi-to-Knora request, but we're actually going to have to make that request anyway, why go to the trouble of using JWT? Why not just keep using cookies as we do now?

[subotic]

As discussed with @benjamingeer, I will take a look at the IIIF standard and OpenSeadragon to see if and how we can use JWT, and then decide from then on.

[subotic]

There seems to be a nice solution with Angular HTTP Interceptors (provided in Angular 4.3). As I understand it, it should be possible to intercept each call from SALSAH to SIPI and remove the token from the URL and add it to the header (see: https://github.com/dougmoscrop/angular-img-http-src).

If the HTTP interceptor idea doesn't work, then we will proceed as recently discussed with @lrosenth, and "break" the Authentication API 1.0 by using tokens attached to the URL. This will also mean, that we will maybe need to patch OpenSeadragon, in the case that it doesn't leave the tokens.

[kilchenmann]

In Salsah resp. in the knora-ui modules we implemented the Angular HTTP Interceptors and I hope we can use it for the Sipi authentication as well

[benjamingeer]

@mrivoal In API v2, the user submits a JWT token to Sipi when uploading a file:

https://docs.knora.org/paradox/03-apis/api-v2/editing-values.html#creating-file-values

We haven't yet implemented using JWT to authorise a user to view a file. Is it correct that you need this feature for your release at the end of February?

[gfoo]

We haven't yet implemented using JWT to authorise a user to view a file. Is it correct that you need this feature for your release at the end of February?

yes, I get an image this way: http://localhost:1024/knora/AU8mw3gyud-GKKZruS24Z9.jpx/full/full/0/default.jpg and thanks to Angular Interceptor, I add an auth method in HTTP Header: Authorization: Bearer <token>.

n.b.: I have a solution to combine easily <img> and Authorization: Bearer <token> , let me know if you are interested

[benjamingeer]

n.b.: I have a solution to combine easily <img> and Authorization: Bearer <token> , let me know if you are interested

Yes please!

[gfoo]

https://stackoverflow.com/questions/46563607/angular-4-image-async-with-bearer-headers

By using an Angular Pipe allowing an asynchronous loading of the image content with the http client which depends on the interceptor (which adds the bearer auth conf):

@Pipe({
  name: 'AsyncLoadingImage'
})
export class AsyncLoadingImagePipe implements PipeTransform {
  constructor(private http: HttpClient) {}

  transform(url: string) {
    return this.http.get(url, { responseType: 'blob' }).pipe(
      switchMap(blob => {
        // return new observable which emits a base64 string when blob is converted to base64
        return Observable.create(observer => {
          const reader = new FileReader();
          reader.readAsDataURL(blob); // convert blob to base64
          reader.onloadend = function() {
            observer.next(reader.result); // emit the base64 string result
          };
        });
      })
    );
  }
}

In the html:

<img *ngIf="sipiImageURL | AsyncLoadingImage | async as imgSrc" [attr.src]="imgSrc">

Can probably done from pure javascript too.

[subotic]

cool, thanks. I think @kilchenmann will like this a lot. We still need to add this to Salsah.

[benjamingeer]

@gfoo If you do it this way, I guess Knora has to provide the token separately from the image URL. Maybe something like this?

  "knora-api:hasStillImageFileValue" : [ {
    "@id" : "http://rdfh.ch/7bbb8e59b703/reps/22bc3f713f07",
    "@type" : "knora-api:StillImageFileValue",
    "knora-api:attachedToUser" : {
      "@id" : "http://rdfh.ch/users/91e19f1e01"
    },
    "knora-api:fileValueAsUrl" : {
      "@type" : "xsd:anyURI",
      "@value" : "http://localhost:1024/knora/incunabula_0000004159.jp2/full/3625,5251/0/default.jpg"
    },
    "knora-api:fileValueHasToken" : "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJLbm9yYSIsInN1YiI6Imh0dHA6Ly9yZGZoLmNoL3VzZXJzLzlYQkNyRFYzU1JhN2tTMVd3eW5CNFEiLCJhdWQiOlsiS25vcmEiLCJTaXBpIl0sImV4cCI6NDY5NTE5MzYwNSwiaWF0IjoxNTQxNTkzNjA1LCJqdGkiOiJsZmdreWJqRlM5Q1NiV19NeVA0SGV3IiwiZm9vIjoiYmFyIn0.qPMJjv8tVOM7KKDxR4Dmdz_kB0FzTOtJBYHSp62Dilk",
    "knora-api:fileValueHasFilename" : "incunabula_0000004159.jp2",
    "knora-api:hasPermissions" : "CR knora-base:Creator|M knora-base:ProjectMember|V knora-base:KnownUser|RV knora-base:UnknownUser",
    "knora-api:stillImageFileValueHasDimX" : 3625,
    "knora-api:stillImageFileValueHasDimY" : 5251,
    "knora-api:stillImageFileValueHasIIIFBaseUrl" : {
      "@type" : "xsd:anyURI",
      "@value" : "http://localhost:1024/knora"
    }
  }

[gfoo]

Not sure to understand, I get the token when I connect to Knora with /v2/authentication, why do you want to provide it when we get the knora-api:hasStillImageFileValue?

[benjamingeer]

When you log in, you get a token that just says you're logged in. But it can't say anything about image permissions, because every image can have different permissions. Therefore, every image needs a different token.

[benjamingeer]

The token for an image needs to specify which permissions you have on that particular image.

[gfoo]

So, in the previous solution there was a cookie per image?

[gfoo]

I thought that Sipi have to ask to Knora if the user (represented by the token) is allowed to see or not an image (represented by its sipi id), in this case, no need for a token per image?

[loicjaouen]

that was the case in v1, a session id was added in cookies, then sipi was forwarding it to knora to ask for permissions.

The JWT is supposed to give the access right information to sipi without sipi having to ask knora.

[benjamingeer]

in the previous solution, every time you request an image from Sipi, the browser sends your login cookie to Sipi, and Sipi makes an HTTP request to Knora (using the session ID from your login cookie), to ask what permissions you have on that particular image. But this is inefficient, because it requires an extra HTTP request every time you request an image. This is why we want to use JWT instead: Knora will serve the token with the image, the client will give the token to Sipi, and the token will tell Sipi what the user's permissions are on the image. This eliminates the need for an extra HTTP request.

[gfoo]

But now the Angular interceptor is no more interesting then... the idea was to take care behind the scene of a global token precisly!

"knora-api:fileValueHasToken": ok fine

[benjamingeer]

But now the Angular interceptor is no more interesting then... the idea that was to take care behind the scene of a global token precisly!

I thought the idea was to be able to add an authentication header (containing the token) to the HTTP request that is sent because of the <img> tag. This is the reason why it has taken so long to resolve this issue. To make the token work with <img> tags, it seemed that the only option was to put the token in the image URL, like this:

http://localhost:1024/knora/incunabula_0000004159.jp2/full/3625,5251/0/default.jpg?token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJLbm9yYSIsInN1YiI6Imh0dHA6Ly9yZGZoLmNoL3VzZXJzLzlYQkNyRFYzU1JhN2tTMVd3eW5CNFEiLCJhdWQiOlsiS25vcmEiLCJTaXBpIl0sImV4cCI6NDY5NTE5MzYwNSwiaWF0IjoxNTQxNTkzNjA1LCJqdGkiOiJsZmdreWJqRlM5Q1NiV19NeVA0SGV3IiwiZm9vIjoiYmFyIn0.qPMJjv8tVOM7KKDxR4Dmdz_kB0FzTOtJBYHSp62Dilk

But this is non-standard in IIIF, so the question is whether this will work with IIIF viewers.

I thought that your Angular interceptor would solve this problem by putting the token in a header, so it doesn't need to be in the URL.

[gfoo]

Interceptor is not related to authentication header. It works behind the scene for every http request without knowing the context, this is nice for the Knora API to automatically add a token in the header.

For image from Sipi I won't use the interceptor, for each http request I will add the particular token each time. Just few modifications in my angular pipe, no problem.

[benjamingeer]

For image from Sipi I won't use the interceptor, for each http request I will add the particular token each time. Just few modifications in my angular pipe, no problem.

Could you give an example of how to do this with an IIIF viewer? I would like to make sure whichever approach we decide to use will work with IIIF viewers like https://openseadragon.github.io/ and http://projectmirador.org/. I am assuming that the IIIF viewer is not going to know about Angular, so I wonder if Angular pipes can actually solve this problem.

[musicEnfanthen]

Just to make sure, I understand it right: Finally the Client has to check the permissions for every file himself instead of Knora overtaking this task?

[benjamingeer]

Just to make sure, I understand it right: Finally the Client has to check the permissions for every file himself instead of Knora overtaking this task?

No, Sipi will check the permissions as it does now. The difference is that Sipi will get the permissions from a token (in the client’s request to Sipi) instead of having to make an HTTP request to Knora to get the permissions. The token will be cryptographically signed by Knora so that Sipi can ensure that Knora really generated the token. From the client’s point of view, the token is just an opaque string that it receives from Knora along with the image URL.

[benjamingeer]

The token will basically say something like: “I, Knora, have determined that the user carrying this token has permission to view file xyz.jp2.”

[benjamingeer]

The only thing that changes is how Knora communicates this information to Sipi.

[gfoo]

Could you give an example of how to do this with an IIIF viewer? I would like to make sure whichever approach we decide to use will work with IIIF viewers like https://openseadragon.github.io/ and http://projectmirador.org/. I am assuming that the IIIF viewer is not going to know about Angular, so I wonder if Angular pipes can actually solve this problem.

My solution is strictly Angular oriented and you must have a deep access of the code if you want to hack an existing software.

I never used openseadragon or projectmirador, if the common piece is the IIIF API, the logical solution is to implement the IIIF Auth API (see @subotic above comment), but don't know if it covers your requirements.

A short time and straightforward solution is clearly to add ?token=<token> because all informations are in the URL, not sure that it really breaks the IIIF API, it is an optional parameter after all :)

[benjamingeer]

if the common piece is the IIIF API, the logical solution is to implement the IIIF Auth API (see @subotic above comment), but don't know if it covers your requirements.

According to Lukas, it doesn't have the concept of different permissions for each image. We have been talking about suggesting an enhancement to the IIIF consortium to allow for this. But that will take time.

A short time and straightforward solution is clearly to add ?token=<token> because all informations are in the URL, not sure that it really breaks the IIIF API, it is an optional parameter after all :)

We have to try it and see if the IIIF viewers actually accept it and if they pass it to the IIIF server (Sipi).

[musicEnfanthen]

Does https://iiif.io/api/auth/1.0/#example-description-resource-with-authentication-services and point 3.0 below help?

[benjamingeer]

I don't think it helps, because as far as I can tell, the IIIF Authentication API is based on the idea that the user logs in and gets a single token that they will use for many different images. The documentation doesn't seem to describe a use case like ours, where the user gets a different token for each image requested.

[musicEnfanthen]

Thanks @benjamingeer for your explanations. I confess, I am still a little bit confused about the different parts/applications involved here and their role in the operational flow. Possibly it is my bad understanding, I apologize.

Why does the IIIF viewer have to know anything about authentication when that is handled by the client, so Angular in these cases? The Angular app checks the auth and, if allowed, sends the image to the view, so the IIIF viewer. Probably I miss something here.

@gfoo Thanks for the approach with interceptors. But don't know if making an http request from within a pipe is a good idea? Couldn't it be done as well from within a service that handles all http stuff and just returns the correct imageSrc to the view to be displayed in img tag?

[benjamingeer]

Why does the IIIF viewer have to know anything about authentication when that is handled by the client, so Angular in these cases? The Angular app checks the auth and, if allowed, sends the image to the view, so the IIIF viewer. Probably I miss something here.

The Angular app doesn't get the image, the IIIF viewer does. The Angular app is just responsible for providing the IIIF viewer with a JSON document describing the image. The IIIF viewer is then responsible for getting the image (or parts of it) directly from an IIIIF server. To do this, the IIIF viewer has to generate HTML (presumably using <img> tags) that causes the browser to make an HTTP request to the IIIF server (Sipi). So if we are going to use a token as a credential, the IIIF viewer somehow has to have this token and include it in the request to Sipi.

Try viewing the source code of this page to see an example:

https://showcase.iiif.io/showcase/osd-viewer/

[gfoo]

Note that we of course don't need anymore of AsyncLoadingImagePipe with the ?token solution:

<img src="sipiImageURL+'?token='+sipiImageToken" >

And, also get a direct access even from your browser to the image just by using the built URL, can be more efficient, for debug, for internal browser cache image...

@musicEnfanthen with a Bearer auth way, the token is not in the imageSrc value but in the HTTP header request, can not be done directly from your browser for example.

[benjamingeer]

@musicEnfanthen Also, keep in mind that Knora and Sipi cannot trust client code to check authorisation (anyone could write a client that simply allows everything), so the checking has to be done on the server.

[benjamingeer]

@subotic Do you have time to work on this, or shall I do it?

[subotic]

I could but not before next week. Would like to first finish the non-actor responders.

Although, I don't really know how. I should probably first try openseadragon and mirador with ?token=xyzin the image URL. If those work, then we could implement it this way.

[benjamingeer]

I could but not before next week. Would like to first finish the non-actor responders.

That sounds OK to me.

I should probably first try openseadragon and mirador with ?token=xyzin the image URL. If those work, then we could implement it this way.

That’s what I was going to do. I had a quick look at OpenSeadragon and it looks like it has an option for adding a custom header to the IIIF image request. I haven’t looked at Mirador, though. Another possibility is that there really is some non-obvious way to do this with the IIIF Auth spec.

[benjamingeer]

I asked about this on the IIIF-Discuss mailing list, and got some interesting replies. See especially the reply from Tom Crane, Technical Director at Digerati, the company that makes the IIIF viewer Universal Viewer:

https://groups.google.com/forum/#!topic/iiif-discuss/f9-eQbdqgME

[lrosenth]

Very, very interesting!

Thanks for sharing. We’ll hve to discuss this!

Best Regards, Lukas (still at home...)

Sent from my iPad

On 11 Jan 2019, at 11:01, Benjamin Geer notifications@github.com<mailto:notifications@github.com> wrote:

I asked about this on the IIIF-Discuss mailing list, and got some interesting replies. See especially the reply from Tom Crane, Technical Director at Digerati, the company that makes the IIIF viewer Universal Viewerhttps://digirati.com/work/galleries-libraries-archives-museums/products/universal-viewer/:

https://groups.google.com/forum/#!topic/iiif-discuss/f9-eQbdqgME

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/dhlab-basel/Knora/issues/520#issuecomment-453461196, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AFN9zIIOXJ0lSpGFl3p_9JQzUy9FWZEBks5vCGD8gaJpZM4NrPjd.

[subotic]

So, as I understand the reply from Tom Crane, our use case should be possible with the current specification. Not exactly as we would have liked it, but workable.

This is what I have understood:

1. In the IIIF manifest, each image gets its own authentication endpoint (however this is done)
2. The viewer then gets a token for each image
3. The viewer uses the token when accessing the image

Also, info.json needs to be accessible without a token.

The Sipi scripts would need to be changed so that if an expired token is used, the correct status code is returned so that the viewer knows to get a new token and re-requests the image.

[benjamingeer]

@subotic I think Knora-ui actually generates info.json in Angular, based on the information from the Knora API, but you'd have to ask the Knora-ui people about that.

The key question seems to be whether IIIF viewers can actually handle a token in the image URL.

[benjamingeer]

Now Andy Irving at the British Library has replied to the mailing list thread, saying "Feel free to get in touch off list if you want any more information". So I think we should just ask him for details about how they're doing this.

[subotic]

Andy Irving wrote:

technically we support different rights in a region of an image but thankfully we don't currently have any of those in use.

This is on a whole other level.

Then I have understood it correctly. They are doing what I have described in my previous comment.

[benjamingeer]

In his latest message he says:

Sorry for not being clear! When the client makes the XHR call to the info.json, it has to use the token because it can't access the correct cookie. when making the requests for pixels, it can send the cookies assigned during the login action.

We use SAML2 (shibboleth) as our authentication system, and it sets a client cookie, which gets sent with the request for pixels. our image server is proxied by a shibbolth service provider that uses that cookie to set upstream data (role). our rights enforcement is done at that point.

So they're still using a cookie in the request to get the image file.

[subotic]

So they intercept each call to the IIIF server, set the authorization, and then the IIIF server can act accordingly:

client -> shibboleth (proxy) -> image server

[subotic]

we could implement it by not allowing direct access to Sipi, but only through Knora which will then talk to Sipi. Basically, create an authorization proxy route.