Unicon / tool13demo

LTI 1.3 Java tool with Spring boot
Apache License 2.0
21 stars 17 forks source link

Return 404 by visiting https://localhost:443. #46

Closed Qitx14 closed 1 year ago

Qitx14 commented 1 year ago

image I have followed the steps given, but finally the access returned 404.

These two linked accesses return JSON data: https://localhost:443/config/ https://localhost:443/jwks/jwk

This link access returns 405: https://localhost:443/config/lti_advantage/client_assertion

My LMS is Open Edx, and I don't know how to integrate this LTI tool with my LMS, so I don't yet know how to use links that are only accessible within the LMS.

Anything pointing me in the right direction would be much appreciated!

ddelblanco commented 1 year ago

There is nothing that should answer in the / path (localhost:443) so the 404 is totally expected and not an issue.

Documentation about how to configure Open Edx with LTI 1.3 is in this page in the documentation of Edx: https://edx.readthedocs.io/projects/edx-partner-course-staff/en/latest/exercises_tools/lti_component.html

Information about the endpoints accessible with and without LMS is in the README. (For the client assertion, it is a POST, so maybe that is why). You will need to use those URLs when configuring Open Edx. On the other hand, you will need to create an entry in the configuration table with the values that your LMS provides (by default you have some sample data there in that table so you can look how it is done).

I really recommend to read the LTI 1.3 standard or watch some videos that explain how it works, (This one is a quick intro that can be useful, https://www.youtube.com/watch?v=9QLZPrMlxNk&t=453s) because if you don't know how the LTI messages happens and why, this tool won't make sense.

Closing the issue because it is not an issue

Qitx14 commented 1 year ago

Thank you very much for your reply. But you may not fully understand what I mean, I have done the configuration related to using LTI1.3 on Open Edx, and generated some information that requires the configuration of the LTI tool. But I didn't find an access path on the LTI tool that would allow me to fill in the corresponding configuration. image image

Qitx14 commented 1 year ago

There is nothing that should answer in the / path (localhost:443) so the 404 is totally expected and not an issue.

Documentation about how to configure Open Edx with LTI 1.3 is in this page in the documentation of Edx: https://edx.readthedocs.io/projects/edx-partner-course-staff/en/latest/exercises_tools/lti_component.html

Information about the endpoints accessible with and without LMS is in the README. (For the client assertion, it is a POST, so maybe that is why). You will need to use those URLs when configuring Open Edx. On the other hand, you will need to create an entry in the configuration table with the values that your LMS provides (by default you have some sample data there in that table so you can look how it is done).

I really recommend to read the LTI 1.3 standard or watch some videos that explain how it works, (This one is a quick intro that can be useful, https://www.youtube.com/watch?v=9QLZPrMlxNk&t=453s) because if you don't know how the LTI messages happens and why, this tool won't make sense.

Closing the issue because it is not an issue

Hi,Because I didn't find a page where I could configure the information provided by Open Edx, I added the data directly to the iss_configuration table in the database. Now I click on Deep Linking Launch to get to a page. image But whether I click the Submit POST button or the Perform GET as Step 2 button, it shows me that something has gone wrong. image Either I did something wrong or this is just a normal operation.

ddelblanco commented 1 year ago

There is a way to POST the configuration to the /config endpoint in the app (the user is admin and the password can be configured in the properties or if it is not configured you can see it in the log messages when starting the app), but adding it in the database is usually the easiest way and it is what I do myself.

The screen you are seeing is a part of the demo features, it stops in the process to show the values received and then you can GET or POST this to the LMS. The GET works in Canvas, BB, Moodle, Sakai and D2L, but it has never been tested in openedx (and although LTI is a standard, some LMS's can be picky and expect some optional things that others are not using).

Anyway, without log information (from openedx or from the tool itself) or something that explains what openedx did not liked from that GET or maybe our tool did not liked from the answer (but that error you are showing seems to be in the openedx side), I can't tell you what is wrong.

Do you have access to the openedx logs? Or maybe with the browser dev tools can you check, for more descriptive errors.? Sadly, LMSs don't like to display detailed error messages to the final user as a security good practice, to don't give clues to the hackers, but that makes the testing of these things a lot more complicated. "There was an error while launching the LTI tool" is the classical error that does not help to know what is happening.

Qitx14 commented 1 year ago

Whether I only need to add one piece of data when adding to the database to ensure that my database is missing data. This is a piece of data I added in the database. I only added one piece of data in the iss_configuration table. Do I need to add data in other tables? image

This is an error checked by the browser development tool, but there doesn't seem to be any useful information. image

I tried to send requests using other browsers and Postman, and sometimes got different errors. image

This link will be generated, but I can't access it. https://example.unicon.net/lti3/

Qitx14 commented 1 year ago

There is a way to POST the configuration to the /config endpoint in the app (the user is admin and the password can be configured in the properties or if it is not configured you can see it in the log messages when starting the app), but adding it in the database is usually the easiest way and it is what I do myself.

The screen you are seeing is a part of the demo features, it stops in the process to show the values received and then you can GET or POST this to the LMS. The GET works in Canvas, BB, Moodle, Sakai and D2L, but it has never been tested in openedx (and although LTI is a standard, some LMS's can be picky and expect some optional things that others are not using).

Anyway, without log information (from openedx or from the tool itself) or something that explains what openedx did not liked from that GET or maybe our tool did not liked from the answer (but that error you are showing seems to be in the openedx side), I can't tell you what is wrong.

Do you have access to the openedx logs? Or maybe with the browser dev tools can you check, for more descriptive errors.? Sadly, LMSs don't like to display detailed error messages to the final user as a security good practice, to don't give clues to the hackers, but that makes the testing of these things a lot more complicated. "There was an error while launching the LTI tool" is the classical error that does not help to know what is happening.

I found some corresponding logs in OpenEdx.

INFO 7 [openedx.core.djangoapps.cors_csrf.helpers] [user None] [ip xxx] helpers.py:64 - Origin None was not in CORS_ORIGIN_WHITELIST; full referer was None and requested host was 'openedx.study'; CORS_ORIGIN_ALLOW_ALL=False INFO 7 [tracking] [user None] [ip xxx] logger.py:41 - {"name": "edx.lti.xblock.launch_request", "context": {"user_id": null, "path": "/api/lti_consumer/v1/launch/", "course_id": "", "org_id": "", "enterprise_uuid": ""}, "username": "", "session": "", "ip": "xxx", "agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36", "host": "openedx.study", "referer": "", "accept_language": "en;q=0.9", "event": {"lti_version": "lti_1p3", "user_roles": "instructor", "launch_url": "https://example.unicon.net/lti3/"}, "time": "", "event_type": "edx.lti.xblock.launch_request", "event_source": "server", "page": null}

Qitx14 commented 1 year ago

A new error occurred when I clicked the Submit POST button and the Perform GET as Step 2 button without changing any configuration. image image After click access links from https://example.unicon.net/lti3/ into https://localhost/lti3.

ddelblanco commented 1 year ago

About: "https://example.unicon.net/lti3/" , yes, you can't because your tool is not there. You don't own "example.unicon.net" so your tool can't be listening in that endpoint.

About "https://localhost/lti3": I don't think that the LMS can access to your localhost unless you use a solution like ngrok or anything like that (so, including if you pass this step, others will fail because the LMS will never be able to reach your JWKS endpoint.

The browser still can access to your localhost, so, some parts of the launching work, meanwhile the LMS won't try to reach the JWKS endpoint, you will be ok. So, the normal LTI launch should work... but... continue reading for my theory about the issue.

If you are receiving that error in the screenshot, it means that the LMS was able (through the browser) to send to you the LTI Request (you can see that the error is in the LTIRequest.java). That is good news, OpenEdx liked your GET and is sending to you the LTI Request.

The error says that or the Nonce value in the JWT is null (I don't think so) or your browser is not able to find the value of the nonce in the session (I think so). I have the feeling that it could be due to don't have a valid https certificate or similar. (Note, Safari does not work very well with cookies , but Chrome and Firefox still work). So, the session cookie is not stored correctly and is lost during the process. Using localhost to test an LTI tool that is communicating with a real and external LMS is always an issue.

I recommend you to use "ngrok", you can start ngrok in front of your server and it will create a real https endpoint that the lms and the browser can call with a valid certificate. Configure OpenEdx to use the ngrok url instead localhost, and see if that changes anything.

Qitx14 commented 1 year ago

So where should I modified to prevent access to this link https://example.unicon.net/lti3/ page. What should I change the link to. I found a place to use this link in application.properties and modified it, but it didn't work. image

The same error 500 occurs whether or not I modify the link in application.properties. I used ngrok as you said and used the ngrok url instead of localhost in OpenEdx. image image image A new error occurred when I clicked the Submit POST button and the Perform GET as Step 2 button. image

I tested it in the code, lti_state is null, state has data output. What's the difference between lti_state and state? image

ddelblanco commented 1 year ago

In those configurations you need to put your ngrok domain. With an incorrect configuration a lot of things won't work as expected.... so maybe that is why you are having all those problems.

The lti_state is the value that is supposed to be in the cookie (or in java, in the session, that is managed by a session cookie). It is a comma separated list of valid states.

You don't have anything there, and that means that your cookies are still not working as expected. That can be maybe (not sure if that is the issue) a problem with the previous configuration with the wrong domaines, because cookies are domain related and maybe once you have the right configuration with the right domain (the ngrok one) it will work.

The state is the value that the LMS returns to us. The value in the LMS answer must be in the list of valid states contained in the lti_state string.

So, please, configure the domains and urls correctly and try again. Let's see if that was the issue :)

Qitx14 commented 1 year ago

I have changed the application.url in application.properties to a local page in a SpringBoot project. Now I have a question. I know that LTI1.3 is composed of Assignments and Grades services, so how does LTI tool send grades to OpenEdx? How is this implemented in the code? How do I test this feature?

ddelblanco commented 1 year ago

With "local page" do you mean the ngrok url or a localhost url? that value will be used in the code to injected in urls that are sent to the LMS or the browser... so, adding the ngrok url is preferred than a localhost url.
Did that change made it work?

The AGS service is this one: net/unicon/lti/service/lti/impl/AdvantageAGSServiceImpl.java

You can use that service form your tool code. In the example there is a AGS controller that is called by the demo page, your tool could be using it too if you want.

Before that, you need the token. In the example code you can find how the token is requested for AGS (or the names and roles service)

I've never worked with OpenEdx, but if it follows the standard... the code follows the standard, so it should work.

Qitx14 commented 1 year ago

“本地页面”是指 ngrok url 还是 localhost url?该值将在代码中用于注入发送到 LMS 或浏览器的 URL...因此,添加 ngrok url 比本地主机 url 更好。 这个改变让它发挥作用了吗?

AGS 服务是这样的: net/unicon/lti/service/lti/impl/AdvantageAGSServiceImpl.java

您可以通过工具代码使用该服务。 在示例中,演示页面调用了一个 AGS 控制器,如果您愿意,您的工具也可以使用它。

在此之前,您需要令牌。在示例代码中,您可以找到如何为 AGS(或名称和角色服务)请求令牌

我从未使用过 OpenEdx,但如果它遵循标准......代码遵循标准,所以它应该可以工作。

The local page is the localhost url.

I have a problem. I went to a page in another SpringBoot project by modifying the application.url in application.properties and clicking access. So what is my LTI tool? tool13demo or another of my SpringBoot projects? My personal feeling is that the page should go to your tool13demo instead of another SpringBoot project.

So far I've managed to access only one page in the code: oidcRedirect.html. The other pages don't know how to access it because clicking the Post and Get buttons in the oidcRedirect.html page causes an error and doesn't know which page to visit. image

ddelblanco commented 1 year ago

Hi Qitx,

The application url will need to be the ngrok url, not the localhost.

I have the feeling that you are asking some questions because you are not understanding perfectly what LTI does and how it works. And that is something I can't be explaining here, because is not so simple as to be explained in a github issue. I will try this time, but please, take a look at the specification and some videos that explain what LTI does and maybe some of your questions will go away.

This demo tool is a DEMO tool that implements the LTI standard. The LMS calls this tool and this tool answers following the specifications of the LTI 1.3 workflow, that involves a series of messages exchanged between both, once all that workflow ends, the tool will display what is asked to display.

https://www.imsglobal.org/sites/default/files/specs/images/security/1p0/fig5p2-oidcflowv1.jpg

So, the LMS calls the OIDC endpoint, the tool answers to the LMS, the LMS POST to the tool again and the tool displays the result. For this to happen, everything needs to be configured correctly, so the LMS can continue the dialog beyond the Oidc call. To do that you need the right application url (the ngrok one, not localhost...) because your LMS can't contact with your localhost urls directly, but it can through the ngrok ones! So, please configure your tool with the ngrok urls and not with "localhost".

Most of the controllers that you are naming are related with that workflow or with the DEMO frontend features for Deep Link, AGS and NRS (the other advantages services)... so yes, they won't be listening to "normal" requests unless they come from the right place and with the right information. Trying to access them directly and failing is expected.

About your other question: The code in this demo tool can be used as base for any other LTI tool... but this is not a middleware... so you won't launch OTHER tool with this. If you want to use this code you will need to "copy/paste" and adapt it to work on your other tool or replace the demo frontend code with your tool code and adjust it to work as expected with the LTI launch.

Qitx14 commented 1 year ago

嗨,齐特克斯,

应用程序 url 必须是 ngrok url,而不是 localhost。

我觉得您问一些问题是因为您没有完全理解 LTI 的作用及其工作原理。这是我无法在这里解释的事情,因为它并不像在 github 问题中解释那么简单。 这次我会尝试一下,但是请看一下规范和一些解释 LTI 功能的视频,也许您的一些问题就会消失。

该演示工具是一个实现LTI标准的DEMO工具。LMS 调用该工具,该工具按照 LTI 1.3 工作流程的规范进行应答,其中涉及两者之间交换的一系列消息,一旦所有工作流程结束,该工具将显示要求显示的内容。

https://www.imsglobal.org/sites/default/files/specs/images/security/1p0/fig5p2-oidcflowv1.jpg

因此,LMS 调用 OIDC 端点,工具回复 LMS,再次向工具发送 LMS POST,工具显示结果。为此,需要正确配置所有内容,以便 LMS 可以在 Oidc 调用之外继续对话。为此,您需要正确的应用程序 url(ngrok 的,而不是 localhost...),因为您的 LMS 无法直接与您的 localhost url 联系,但它可以通过 ngrok 的 url!因此,请使用 ngrok url 配置您的工具,而不是使用“localhost”。

您命名的大多数控制器都与该工作流程或 Deep Link、AGS 和 NRS(其他优势服务)的 DEMO 前端功能相关……所以是的,它们不会监听“正常”请求,除非他们来自正确的地方并拥有正确的信息。尝试直接访问它们,预计会失败。

关于您的其他问题: 此演示工具中的代码可以用作任何其他 LTI 工具的基础...但这不是中间件...所以您不会用它启动其他工具。如果您想使用此代码,您将需要“复制/粘贴”并调整它以在您的其他工具上工作,或者将演示前端代码替换为您的工具代码,并调整它以按 LTI 启动的预期工作。

Ok, I will continue to find materials to learn the working principle of LTI.

I ran into some issues while looking at the tool13demo code: An error occurred after clicking Submit POST. image image According to the error, the following code is found:

image In this code I need to get the parameter 'link' from 'HttpServletRequest', but I didn't find the parameter link in the HttpServletRequest request I sent. This is the parameter in HttpServletRequest. image Is there no link parameter or did I not add the link parameter to HttpServletRequest in a previous step?

ddelblanco commented 1 year ago

When you create a LTI element in the LMS using deeplink, the target_url will contain "link" on it to specify what resource the LMS wants to open. The target URL will end in ?link=12123, for example...

It is not something that you need to do by yourself at all unless you create the LTI elements in the LMS manually. Usually the link does not come in the request, so it is null most of the time (and that is ok), but in other place in the code we extract it from the target url. (look at line 175 approx).

The error you are sending seems to be in this place: at net.unicon.lti.security.lti.LTI3OAuthProviderProcessinaFilter.java: line 118

And that file, in my code does not have 118 lines. Did you added something to that file?

Qitx14 commented 1 year ago

当您使用 Deeplink 在 LMS 中创建 LTI 元素时,target_url 将包含“link”以指定 LMS 要打开的资源。目标 URL 将以 结尾?link=12123,例如...

这根本不需要您自己做,除非您在 LMS 中手动创建 LTI 元素。通常,链接不会出现在请求中,因此大多数情况下它为 null(这没关系),但在代码的其他位置,我们从目标 url 中提取它。(大约参见第 175 行)。

您发送的错误似乎是在这个地方: at net.unicon.lti.security.lti.LTI3OAuthProviderProcessinaFilter.java:第118行

在我的代码中,该文件没有 118 行。您是否向该文件添加了某些内容?

These are the errors reported: image image image image image In LTI3OAuthProviderProcessinaFilter. Java file line 118 appeared a mistake, I didn't change any content of this file.In LTI3OAuthProviderProcessinaFilter. Java file line 118 appeared a mistake, I didn't change any content of this file.

You said somewhere else in the code, extract from target_url which file is it in? Can you take a picture?

ddelblanco commented 1 year ago

This error does not seem related with the previous one.

Based on that error, the LTIContextID is null. That is a value that comes in the id_token that the LMS is sending to you, and it MUST NOT BE null unless you are calling the tool from outside a course. (If that is the case, then surely you have an error because we are not considering the option to call the tool outside a course) .

Please, could you paste here the full id_token (encrypted or decrypted, as you prefer) that you are receiving from the LMS, or at least the values from the section named: https://purl.imsglobal.org/spec/lti/claim/context

The id value on that section shouldn't be null. Should be the identifier from the course you are calling the tool. (As said, if you are calling the tool from a inside a course).