Product images missing on Aus production

[kirstenalarsen]

Description

Aus had a new server created / deployed on Monday night (24/2/2020). After the new server was created all the product images were missing, showing the broken image icon

@Matt-Yorkley did something (maybe also @mkllnk ?) and they came back in the shop I was looking at (Warrenbayne Farm Collective)

I have just received a report from a user that some are missing in their shop (https://openfoodnetwork.org.au/synchronicity-farm/shop#/shop) and I have looked through a few others and found there are more similar problems eg. https://openfoodnetwork.org.au/foodconnectwholesale/shop#/shop

I cannot be sure but I suspect this is an issue hanging around from the deployment problem that wasn't picked up until now?

Expected Behavior

Product images should display

Actual Behaviour

Some do not

Steps to Reproduce

I don't have any information about which images are absent / broken or why those ones

Animated Gif/Screenshot

Workaround

not that i know of - I am not sure if users could re-upload images?

Severity

s2 - product images are pretty important and people get really shirty when they're missing

Your Environment

MAC OS, Chrome and also confirmed problem on firefox

Possible Fix

[Matt-Yorkley]

It should be fixed now @kirstenalarsen, if you want to re-check.

Technical notes are in Slack here for reference on how it was fixed: https://openfoodnetwork.slack.com/archives/CDLKH9MM0/p1582724641019500?thread_ts=1582715255.015800&cid=CDLKH9MM0

[kirstenalarsen]

It was fixed for that user, but we have had another report today that it is happening to them. I have just checked and confirmed that yes there are broken image links in their shop right now. Here's what they said . .

" I keep getting broken links for images I've uploaded in our shop front. I reloaded them all yesterday and now others are broken. Not sure why this is happening? "

I too would love to know why this is happening :) @Matt-Yorkley @mkllnk @luisramos0

https://openfoodnetwork.org.au/fawkner-wholefoods-collective/shop#/shop

[kirstenalarsen]

bit more information from the currently affected customer " I was uploading either screen res jpgs or png files and they were all small file sizes. "

[mkllnk]

I'm currently checking exactly which images are missing:

Spree::Image.count
#=> 6589

missing = Spree::Image.all.reject { |i| i.attachment.exists? }; nil
missing.count
#=> 1932

[mkllnk]

It looks like new images are still uploaded as png files but our settings then always refer to jpg files.

[mkllnk]

I noticed another difference in the configuration. The config said that all images are jpgs and now I changed it to "Unchanged" which means that we will use the original file type.

I checked the images again and now there is only one missing image (id: 7400). It's indeed not saved on S3 and was created yesterday. It's a jpeg file instead of jpg.

• Product 10741: Zucchini & Green Bean Curry (Sabzi) - Vegan
• Supplier: Flock Stock & Basil
• Distributor: Prom Coast Food Collective

[mkllnk]

Since the main issue seems to be solved, I'm closing this. Fee free to re-open with further details if need, @kirstenalarsen.

[mkllnk]

Even though my script said that all images are now present, that is only true for the originally uploaded image and the default size product. In the shop we are show the size small and when we click on the product and the modal opens it shows the image in size large. Unfortunately, some of those images are still missing: https://openfoodnetwork.org.au/growlightly-vegiebags/shop#/shop

I believe that this problem is limited to files ending in .jpeg. Since the file format is the same for the two endings jpg and jpeg, the image library seems to be confused and tries to find image.jpg when it actually uploaded image.jpeg.

We can define the file ending for all images to .jpg but that breaks all .png and .gif images. I tried to regenerate all images but that didn't help:

bundle exec rake paperclip:refresh:thumbnails CLASS=Spree::Image

Now I'm wondering if it would work to set the sizes mini, small, and large to .jpg and then regenerate all thumbnails. Maybe that would convert all images to .jpg so that they all work. The downside would be that all non-jpg images would be broken during the conversion and the conversion takes several hours. Worth a try?

Why do other instances not have this problem? Do UK people never upload .jpeg images, @lin-d-hop?

[kirstenalarsen]

For people who are really cranky, can I suggest that they try uploading images as .JPG?

[kirstenalarsen]

@mkllnk @Matt-Yorkley @lin-d-hop ?

[mkllnk]

I started the process of regenerating thumbnails with these settings:

[mkllnk]

I now found that the regeneration of thumbnails failed at some point because of an invalid image. All images after that were not processed. I patched the code to rescue from that error and continue. Now I'm running it again.

~/.gem/ruby/2.3.0/gems/paperclip-3.4.2/lib/tasks/paperclip.rake

  namespace :refresh do
    desc "Regenerates thumbnails for a given CLASS (and optional ATTACHMENT and STYLES splitted by comma)."
    task :thumbnails => :environment do
      klass = Paperclip::Task.obtain_class
      names = Paperclip::Task.obtain_attachments(klass)
      styles = (ENV['STYLES'] || ENV['styles'] || '').split(',').map(&:to_sym)
      names.each do |name|
        Paperclip.each_instance_with_attachment(klass, name) do |instance|
          puts "Processing ID: #{instance.id}"
          begin
            instance.send(name).reprocess!(*styles)
          rescue Paperclip::Errors::NotIdentifiedByImageMagickError => e
            puts "errors while processing #{klass} ID #{instance.id}:"
            puts " " + e + "\n"
          end
          unless instance.errors.blank?
            puts "errors while processing #{klass} ID #{instance.id}:"
            puts " " + instance.errors.full_messages.join("\n ") + "\n"
          end
        end
      end
    end

[mkllnk]

I had an error in that code and it still failed. But at least now I know at which point it failed. I extracted the important code to run it in the console for the rest of the images:

Spree::Image.unscoped.where("attachment_file_name is not null").where("attachment_file_name not like '%.jpg'").where("id > 2634").find_each { |instance| puts "Processing ID: #{instance.id}"; instance.attachment.reprocess! unless instance.attachment.exists?("small"); puts " " + instance.errors.full_messages.join("\n ") + "\n"; }

[mkllnk]

Broken images:

• 175
• 2634
• 4335
• 4743
• 7467

[mkllnk]

It looks like I finally re-processed all the images in the wrong format and all images are showing now. The few broken images belong to deleted products.