Failing to load parent form with a CORS issue on 'jr://' resources

Yann-J commented 1 month ago

Description

Steps to Reproduce

Created a parent form (enketo link) and submitted a bunch of responses
Created a child form using Dynamic Data Attachment feature, linking forms appropriately.
- The form displays correctly in the preview window:

including pulling data from the parent form:
But NOT when opening the published enketo link

Expected behavior

Form should load correctly

Actual behavior

The published Enketo form (here), throws a Failed to load parent_form.xml error:

Looking at the browser console, I can see that all network calls succeed, but an error is raised that seems to block the loading of the resource, throwing this error: Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at jr://file/parent_form.xml. (Reason: CORS request not http).

Looking at the source XML of the form as downloaded from Kobocat, I can indeed see that the resource URI is jr://:

I do not know what these jr:// resource URIs are, but these don't seem to be expected, since indeed browsers will appropriately block these for violating CORS...

Additional details

Using the following docker images on our own infra:

kobotoolbox/kpi:2.024.19b
kobotoolbox/kobocat:2.024.19a
kobotoolbox/enketo-express-extra-widgets:6.2.2

I would appreciate some guidance troubleshooting this...

Are these jr:// URIs expected?
If yes, how are these supposed to circumvent CORS violation?
If not, what am I missing / doing wrong with my setup?

When inspecting the browser logs while loading the preview, I can see that this is references as a separate media URL in the form definition downloaded from Enketo, and there is a separate http call to download the linked parent_form.xml, which is going through without any CORS issue...

Many thanks! 🙏

noliveleger commented 3 weeks ago

Hello @Yann-J, Are you facing the same problem with (static) pull data? If you do, you'd rather open the issue on the enketo repo.

But yes, jr://URLs are expected.

Yann-J commented 3 weeks ago

Good question @noliveleger indeed I had not tried.

Just did a quick experiment using this form and can confirm that pulldata works fine with a static CSV (both in preview and Enketo URL).

Is there an easy way for me to check that the dynamic data attachment job has worked and see the generated XML?

Yann-J commented 3 weeks ago

OK, doing some further digging, here are some more details:

From the Enketo server logs, I can see the two source files being loaded from kobocat:

The downloadUrl has the XML source, including the jr:// links to attached files (jr://images/... for pictures, jr://file/parent_form.xml for my parent form data)

The manifestUrl has the list of attached files and their locations, including images and the parent form

I'm not entirely sure how the jr:// URIs are being mapped to entries in this manifest, but while the pictures can probably be matched by filename, the filename for the parent form data is intriguing: it's a URL, using the KPI_INTERNAL_URL hostname (therefore unreachable from internet), and external.xml filename... The downloadUrl from that manifest points to a valid file, showing the data attachments worked.

Contrasting this to when I load the preview in the Kobotoolbox UI, where the links point to kpi snapshot URLs rather than kobocat URLs:

This time the manifest includes a friendlier filename for the parent form data:

I don't know if all that is expected, but could this odd filename be why the jr:// link cannot be mapped to a resource from the manifest (and therefore the browser falls back to fetching it directly, failing CORS)?

noliveleger commented 2 weeks ago

@Yann-J, thanks for the super detailed investigation. We will look at it.

noliveleger commented 2 weeks ago

@Yann-J We could not reproduce the error on our server. You can give it a try with exact same project and let us know if you do face the same error, but from my test, I'm able to load the form in EE and preview without any problem.

Yann-J commented 2 weeks ago

OK thanks @noliveleger , I'm sure it's something to do with my own server setup... everything else seems to work fine so I think the setup is generally fine, but there might be some configuration I don't fully understand... I don't expect the issue to affect your server, but let me give it a try and see if I can reproduce... I would just appreciate some guidance / suggestions as to where to look at so I can keep troubleshooting this...

noliveleger commented 2 weeks ago

Hello @Yann-J, as you may already know, due to the small size of our team and large volume of users, we cannot provide support or guidance via GitHub issues. Especially with custom setup. You may search for help on our community forum. If it is really a bug that affects also our servers, then it should be considered as bug and it should be added to our backlog. Having that said, maybe not related but I would have a look at CSP_EXTRA_DEFAULT_SRC and ENABLE_CSP on both KoboCAT and KPI. Maybe it is at your reverse proxy level.

kobotoolbox / kpi