search

mozilla / api.webmaker.org

Services for Webmaker
https://api.webmaker.org
16 stars 14 forks source link

Generate missing thumbnails #172

Closed xmatthewx closed 9 years ago

xmatthewx commented 9 years ago

Projects created since snippet launch are missing thumbnails.

cc @cadecairos @ashleygwilliams

xmatthewx commented 9 years ago

related: https://github.com/mozilla/api.webmaker.org/pull/94

cadecairos commented 9 years ago

I thought doing this would be fairly straight forward (and it kinda is)... Here's a breakdown of the steps

  1. Select min(page.id), and its associated project_id and user_id for all projects that have no thumbnail
  2. for Each set of ids, generate a thumbnail URL using thumbnail service and save row to database
  3. bust the projects.findOne and projects.findUsersProjects server caches for each set of data

Which is going to be one heck of a script.

Is it worth the effort?

cc @jbuck @ashleygwilliams @xmatthewx

xmatthewx commented 9 years ago

@secretrobotron - We had a snafu with thumbnails. Trying to weight the cost & benefit of generating them for existing projects. It will certainly impact users who return to the app after playing around ... and make it easier for you to browse new projects to feature.

@cadecairos - how much time would you estimate?

cadecairos commented 9 years ago

half a days worth of work to write and test the script

secretrobotron commented 9 years ago

:+1:

@cadecairos lots of the projects are empty. detecting those and skipping over them will be useful, i bet.

xmatthewx commented 9 years ago

Thanks for the speedy work on this. Are we throttling the script? Should we, to avoid impact on active users?

cadecairos commented 9 years ago

It shouldn't interfere with any active users, afaict. I'm monitoring prod stats just in case

cadecairos commented 9 years ago

also, it's throttled to only update one project at a time.

xmatthewx commented 9 years ago

Cool cool :relieved:

cadecairos commented 9 years ago

We're all fixed up!

~ $ node scripts/regenerate_thumbnails.js "projects.thumbnail = '{}'"
Found 0 rows to update
Thumbnail queue empty. Nice job!
xmatthewx commented 9 years ago

:dart: :clap: :cake: