Gallery images unreliable, ZIP download files are missing

[DrLex0]

When creating a new Thing or adding images and models to an existing one, the changes are added to processing queues. As far as I know, there are two queues, one for images and one for creating/updating the ZIP file that is provided through the main ‘Download’ button. The processing time for these queues used to be in the order of seconds, maybe a few minutes during moments of heavy load.

Now, both these processing queues are lagging horribly. The one for images seems to lag a few days, the one for ZIP files more than a week (somewhere around 9 days at this moment, but the exact values change all the time). During the time the images are stuck in their queue, only the main thumbnail can be seen in the overview if the user uploaded one. If not, only a grey rectangle with a cogwheel is shown. During the time the ZIP file has not yet been generated, the main Download button will only send users to the download tab where they have to download each file separately.

This seems due to the servers being completely overloaded and having an enormous backlog of files to process (#2).

Edit 2019/08/29: now images are visible again, but they often still don't work. If they don't work immediately after uploading, they never seem to work and must be deleted and re-uploaded until they work. They also look bad if they are not uploaded in 4:3 aspect ratio. As observed by @Lyl3S in a comment below, this ‘fix’ looks more like a hack to get images to show up quicker while the root cause of the problem has not been solved. Please try to do better than this.

[clsn]

Is it just hours or days? I've been waiting for zips of a whole bunch of things what seems like a lot longer than just days. Some are definitely over a week. Do they ever show up? Or is this bug-report being optimistic?

[DrLex0]

The processing of zip files is much slower than for images. Currently the newest Things I find that have zip files, are from August 14th. It has not completely stagnated because a few days ago the newest ZIPs were from the 13th. These bug reports are a best effort within the scarce time I can assign to this crap that should not even have been necessary. There is no point in trying to document everything down to exact accuracy because the situation changes every day and who knows, maybe they have fixed something or made something worse, but we cannot know due to #4.

[aeakin1]

I am fairly certain that a number of files uploaded in late june/early july STILL do not have AIO ZIPs. I will attempt to locate on and add it to this issue if I can.

[Lyl3S]

An additional problem, but it's related to this issue...

It looks like the way it's currently working is when an image file is uploaded, it's stored as a byte stream in the database. Some time later it is then processed into various sized images and each resolution is stored as a temporary file with a name similar to card_preview_AnArticulatedPosableStickman.jpg, where it adds a prefix (cardpreview) indicating the resolution to the original filename that was uploaded. Some time after that it goes through another round of processing and each resolution is stored as a permanent file with a name similar to f879d6c5bcd2de257f08859c7b2515da_preview_card.jpg, where it adds a suffix (_preview_card) indicating the resolution to a random string of characters.

I have a thing that I just published and the images are still stored as the temporary files. However all the notifications in my dashboard where someone has collected it show an incorrect image (f38ea72df76e176e52ce26e35cf9c218_preview_medium.jpg) from some other thing that is not mine.

[DrLex0]

Lol, I recognize that model: it's some customized copy of my customizable glass bed bracket.

A while ago we also had random thing descriptions being pre-filled in on the New Thing page. I can't even imagine how such mistakes can be made. It really looks as if some random persons are hacking away at the site without much of a clue what they're doing. The mere fact that they are obviously just making untested changes on the production servers is also grade A amateurism. The only vague sign of life we have seen from what we assume is a Makerbot employee, was a single post by someone we've never seen before. Maybe all the others have been fired…

[Lyl3S]

Lol. Funny coincidence.

Yeah, their code changes have been horrific lately. I'm pretty sure the temporary files are a new thing brought in to get around the enormous backlog of the processing for the permanent files. So essentially their solution was to double the amount of processing required, further increasing the backlog.

[Lyl3S]

@DrLex0, I just looked at the collection of remixes of your customizable glass bed bracket and the image showing up in my dashboard doesn't match any of the published remixes, so that means it's from an unpublished remix. That makes the situation even worse because I should not even be able to see that image.

This is the "card" resolution of that image in case anyone wants to double-check that it's not one of the published remixes: https://cdn.thingiverse.com/renders/d9/81/f4/3e/53/f38ea72df76e176e52ce26e35cf9c218_preview_card.jpg

[DrLex0]

I managed to get something published with working images. I had to delete the images and re-upload them, and then the ridiculous temporary previews were actually generated. So, next to all this double processing, they have also encouraged users to re-upload files multiple times, further increasing load!

[Lyl3S]

Yeah I did the same thing. I had uploaded the files for a thing and had been waiting days for them to be processed. In the meantime, I was uploading files for a different thing and noticed that the image was immediately processed for the temporary files. So I repeatedly re-uploaded the images for the other thing that I had been waiting days for. It took several tries before they were processed right away.

Also, I've noticed that the temporary files don't retain the aspect ratio of the original (most obvious in the thumb images in the analytics page), whereas the permanent ones do, so it's not like they're just renaming the temporary ones into the permanent ones. They are definitely re-processing the original image.

[DrLex0]

It seems mesh previews are working again. Even Things from less than a minute ago already have the previews. No Thingiview though, and gallery images are still messy, with anything that is not uploaded in a 4:3 aspect ratio being distorted. Also the ‘Download all’ ZIP file isn't working yet.

[DrLex0]

ZIP download is working again, sort of. The archive now contains very broken LICENSE and README files, and also all possible downscaled copies of the gallery images, plus sometimes totally random images of completely unrelated Things?! For instance this one, which also shows up in my dashboard with one of those incorrect thumbnails. What a mess! Thingiview also works again on the most recent things. But because issue #4 still holds, we have no clue as to what has actually been fixed and what not.

[DrLex0]

I'm going to close this one because the gallery images sort of work now. I'll replace it with a new issue about the mess and outdated files inside the ZIP files.