Base64 images migration breaks image URLs

[sdelcroix]

Describe the bug

After upgrading from 0.25 to 0.26.7, and running the Base64 images migration task, some images have wrong URL. The filename is not present at the end of URL, but a "hash" or a "key".

Example : /rails/active_storage/blobs/eyJfcmFpbHMiOnsibWVzc2FnZSI6IkJBaHBBaW9JIiwiZXhwIjpudWxsLCJwdXIiOiJibG9iX2lkIn19--3effc0f11422c1febf5c0b07889068782d40a4bc/ab3c4f7c3a620fc9486731c88d519ad0

Additionally, the protocol and domain are missing too.

This is resulting in a 404 not found error.

To Reproduce

1. Update from 0.25 to 0.26 with participatory processes containing blog posts with base64 encoded images inside
2. Run the Base64 images migration task
3. See the wrong URLs in admin text editors and in frontend

Expected behavior

Right URLs for base64 images converted in ActiveStorage attachments

Screenshots

No response

Stacktrace

Here some log lines :

I, [2023-07-20T16:06:46.296880 #3534270]  INFO -- : [3a7fe61a-0930-4e8c-8b25-0e8415b33989] Started GET "/rails/active_storage/blobs/eyJfcmFpbHMiOnsibWVzc2FnZSI6IkJBaHBBaW9JIiwiZXhwIjpudWxsLCJwdXIiOiJibG9iX2lkIn19--3effc0f11422c1febf5c0b07889068782d40a4bc/ab3c4f7c3a620fc9486731c88d519ad0" for XX.XX.XX at 2023-07-20 16:06:46 +0200
I, [2023-07-20T16:10:47.595417 #3534270]  INFO -- : [462b6c43-a0bf-4fd6-90d5-5da333d5e260] Processing by ActiveStorage::BlobsController#show as HTML
I, [2023-07-20T16:10:47.595532 #3534270]  INFO -- : [462b6c43-a0bf-4fd6-90d5-5da333d5e260]   Parameters: {"signed_id"=>"eyJfcmFpbHMiOnsibWVzc2FnZSI6IkJBaHBBaW9JIiwiZXhwIjpudWxsLCJwdXIiOiJibG9iX2lkIn19--3effc0f11422c1febf5c0b07889068782d40a4bc", "filename"=>"ab3c4f7c3a620fc9486731c88d519ad0"}
I, [2023-07-20T16:10:47.596198 #3534270]  INFO -- : [462b6c43-a0bf-4fd6-90d5-5da333d5e260] Filter chain halted as :set_blob rendered or redirected
I, [2023-07-20T16:10:47.596382 #3534270]  INFO -- : [462b6c43-a0bf-4fd6-90d5-5da333d5e260] Completed 404 Not Found in 1ms (ActiveRecord: 0.0ms | Allocations: 144)

And the blob found in console :

irb(main):065:0> ActiveStorage::Blob.where(filename: "ab3c4f7c3a620fc9486731c88d519ad0")
  ActiveStorage::Blob Load (1.4ms)  SELECT "active_storage_blobs".* FROM "active_storage_blobs" WHERE "active_storage_blobs"."filename" = $1 LIMIT $2  [["filename", "ab3c4f7c3a620fc9486731c88d519ad0"], ["LIMIT", 11]]
=> #<ActiveRecord::Relation [#<ActiveStorage::Blob id: 2090, key: "e6w84ejub4ojx7c0pfxdv6ovxyn0", filename: "ab3c4f7c3a620fc9486731c88d519ad0", content_type: "image/jpeg", metadata: {"identified"=>true, "width"=>800, "height"=>600, "analyzed"=>true}, byte_size: 95703, checksum: "SFtrudnC5w5sKv30j3+7Xw==", created_at: "2023-07-17 13:32:04">]>

Extra data

• Device:
• Device OS:
• Browser:
• Decidim Version: 0.26.7
• Decidim installation: upgrade from 0.25.2 (in several steps : 0.26.0, then 0.26.3, then 0.26.4, then 0.26.7)

Additional context

• The Base64 images migration task ran with success
• Also, this is a multitenant Decidim instance.

[sdelcroix]

This case is even worse :(

I really like Decidim and I'm really sad to admit that the base64 migration has been a total mess for us. Additionally to the initial issue, there were others :

• some images are still encoded in base64
• some blog posts have completely different images between 0.25 and 0.26 versions
• some converted images are inserted at the wrong place in the text
• some images have been removed from content (especially agenda items)

After digging hours to figure out an automated way, we had to fix all of these by hand. The topic is closed for us but if someone wants to search why the migration task silently failed, I can provide content data that have been incorrectly migrated.

[ahukkanen]

Could you provide the original content which you have been migrating with the migration script and it resulted in the wrong outcome?

It would potentially help fixing this issue.

[sdelcroix]

Thanks for your time ;) Here are some sample data :

Accountability with images still encoded in base64 after migration

decidim upgrade 0.26 - migration base64 issue - accountability 72.txt

I think this is because the task doesn't handle this case.

Page where images are wrong (and inserted at wrong places) after migration

decidim upgrade 0.26 - migration base64 issue - page 15.txt

Blog posts where images are wrong after migration

decidim upgrade 0.26 - migration base64 issue - blog post 19.txt decidim upgrade 0.26 - migration base64 issue - blog post 20.txt decidim upgrade 0.26 - migration base64 issue - blog post 29.txt decidim upgrade 0.26 - migration base64 issue - blog post 31.txt

I'm on vacation from July, 29th to August, 13th. If needed, I could provide more data until tomorrow or when I'm back ;)