Resize image files on perseus file export

rtibbles commented 7 months ago

Observed behavior

Currently when inserting images into the answer section of a multiple choice question that gets exported into a Perseus format, if the images have been resized they have height and width dimensions attached to them, similarly to how they are annotated in the question block.

Unfortunately, Perseus ignores the height and width annotation when images are inserted into the answer section (rather than the question section).

Additionally, this also means that if someone inserted a very large image and then resized it to be significantly smaller, we would be shipping a much larger image than is necessary in the bundled perseus exercise.

Expected behavior

During the backend process to create a perseus exercise bundle, we should resize any images that have been resized, in order to bundle them at the appropriate resolution. This will solve both of the issues above.

User-facing consequences

This will fix several user reports of image insertion into answers that have not been properly resized in Kolibri, and also save our users valuable space!

Acceptance criteria

[ ] Read the width and height information for each image from the question data
[ ] Pass the width and height information for each image into the create_perseus_zip function - this could be done as a dict mapping from the image file name to the dimensions
[ ] Before the image file is written to the zip resize it according to the dimensions, using Python Image Library (PIL)
[ ] Add extra handling for the case where the same image file is reused in the same perseus file but with different dimensions - this is probably best done by renaming the image files with the checksum of the resized image and updating the exercise data with the relevant rename

KshitijThareja commented 7 months ago

Hi @rtibbles. I would like to work on this issue. Could you please assign it to me if possible? Thanks :)

rtibbles commented 7 months ago

Hi @KshitijThareja - yes that would be fine - note that most of this should be relatively straight forward, but the edge case handling of the last item in the checklist is where it will be trickier (as it will require more interaction between the image resizing code and the code that is outputting the JSON data).

KshitijThareja commented 7 months ago

Hey @rtibbles. I have a fair idea of what my approach should be. I just had one question. How do I test my changes? I am unsure if I can add a studio channel (from my local) containing a sample exercise to kolibri for testing it. And in studio itself, there's no preview available for the exercise. I can only preview the quiz questions as shown here: test

rtibbles commented 7 months ago

It is possible to install a channel from your development Studio. You can either override the STUDIO url to use your development server on Kolibri using this environment variable: KOLIBRI_CENTRAL_CONTENT_BASE_URL or add it as another server.

I think it would also be a good idea to write some python unit tests for this code.

KshitijThareja commented 6 months ago

Hey @rtibbles! Sorry I had been a bit inactive here because of my exams. While working on this issue, I had stumbled over another problem. I changed the CENTRAL_CONTENT_BASE_URL env variable to import contents from my local studio. I was able to see the published channels from my studio in kolibri's import channel section. But when I tried importing one of the available channels, I ran into an import error.

What's happening is when kolibri downloads the data for a specific channel, the data that is downloaded (the sqlite database) does not conform to the original database that should have been fetched, thereby giving a SchemaNotFound error. I am unsure how to proceed with this. Could you please double check this from your end?

rtibbles commented 6 months ago

Hrm, it should be configured to serve the files via a redirect: https://github.com/learningequality/studio/blob/unstable/contentcuration/contentcuration/dev_urls.py#L76

Can you post the SQlite database for the channel here as an attachment? You should be able to download it from 127.0.0.1:8080/content/databases/<channel_id>.sqlite3 from your Studio server.

KshitijThareja commented 6 months ago

Yeah sure. Here's the Link. Couldn't add it as an attachment directy.. This is the database that should be fetched on channel import. But the one that is actually fetched is empty.

rtibbles commented 6 months ago

Hrm, I wonder if the redirect is causing issues when downloading, and hence causing the empty file.

Just to confirm, if you go to this URL: 127.0.0.1:8080/content/databases/66a31fdbdd745c7c9e499206f3c4cced.sqlite3 you get the file above downloaded?

If in a python shell you do:

import requests
response = requests.get("127.0.0.1:8080/content/databases/66a31fdbdd745c7c9e499206f3c4cced.sqlite3")
print(response.content)

Does it print the binary content of the file?

KshitijThareja commented 6 months ago

Does it print the binary content of the file?

Yeah, it does. I had checked it out earlier too. I wasn't able to find any specific reason for the previously mentioned error though. It should be able to fetch the database correctly by default..

KshitijThareja commented 5 months ago

Hi @rtibbles! Sorry I wasn't able to pay attention to this issue because of my university exams. The error that we discussed about earlier while trying to import a channel still persists. What should be the next steps? I am unable to move forward with this issue currently.

rtibbles commented 5 months ago

Can you confirm what you're setting your CENTRAL_CONTENT_BASE_URL to?

KshitijThareja commented 5 months ago

I have set it to work with my local development server. The value is set to: "http://localhost:8080".

rtibbles commented 5 months ago

Just in case there's a silly bug at play, could you try adding a trailing slash to the end? http://localhost:8080/ - I don't think this is the issue, but just want to double check.

KshitijThareja commented 5 months ago

I tried that, still getting the same error. I wonder if it's the same for everyone, or if it's some misconfiguration from my end.

rtibbles commented 5 months ago

The only other thing I can think of is that somehow the CENTRAL_CONTENT_BASE_URL is not properly getting used to fetch the channel database, and instead, it's attempting to fetch from Studio (so a bug in Kolibri).

Could you try adding the address as a "Peer import server" through the UI instead, and see if that works?

KshitijThareja commented 5 months ago

Hi @rtibbles I tried adding the same address (http://localhost:8080) as a Peer import server. It still gives the same error (SchemaNotFound error). In both the cases, i.e. changing the CENTRAL_CONTENT_BASE_URL env variable's value to use the local studio server and using Peer import option, I am able to see the published channel in kolibri, but unable to download the resources because of the aforementioned error.

rtibbles commented 5 months ago

I will test locally and see if this is a more systematic issue, or if there is some configuration that is still missing for you!

MisRob commented 5 days ago

@KshitijThareja would you like to return to this or should we unassign?

I just checked with @rtibbles and it seems it should be possible to continue if you'd like to:

The fix for the local import [from a local studio instance] should be fixed on Kolibri develop, so testing should be possible now

KshitijThareja commented 5 days ago

Hi @MisRob! Sorry I was occupied with some other tasks. I'll get back to working on it. Thanks :)

MisRob commented 5 days ago

Thanks @KshitijThareja and no pressure at all, take all the time you wish (I'm just doing my regular status and clarification rounds :)

learningequality / studio