Image Infrastructure Update

[schanzer]

The Problem

Demands on our use of images have grown a LOT since 2020:

• We added image captions in 2021
• Flannery led the addition of screenreader-friendly descriptions in 2022
• Flannery's work on the Social Studies branch has led to the need for image citations as well

Dorai has heroically extended the @image{...} directive with optional arguments this whole time. But each argument adds a layer of complexity and an opportunity for mistakes. (QUICK! What is the size and description of the following image: @image{images/box-n-whisker-plot.png, 300, "A sample box-and-whisker plot"} -- lmfao no ur wrong the caption is "300" silly!)

We've also accumulated some technical debt, with zero standardization on where image files live within a lesson. Sometimes they're in a dedicated images/ directory, but sometimes they're in the pages/ directory. And sometimes they're in both!

With the addition of image citations coming down the pike in 2023, it's time to address this problem.

Proposed Solution

• All lesson images should live inside an images folder within the lesson plan (just like the pages directory).
• This folder should include a lesson-images.json file, which acts as a dictionary for all the relevant lesson information
• The dictionary has the following structure[*]:

[
  "box-plot.png" : {
        "caption" : "distribution of US Income in 2003"
        "description" : "a box plot showing a right skew"
        "source" : "retrieved from data.gov on April 15th, 2022"
        "license" : "Creative Commons 2.0 - ND - SA"
  },
  "red-star.png" : {
        "caption" : ""
        "description" : "an image of a solid, red star"
        "source" : "Created by the Bootstrap Team"
        "license" : "Creative Commons 4.0 - NC - SA"
  },
]

• All @image directives include nothing more than the image path and (optionally) the size of the image. For example:

  @image{../images/box-plot.png, 250}
  OR
  @image{../images/box-plot.png}

• When building a lesson, we perform the normal integrity checks (does the image exist? Does it have the required fields?) and generate the appropriate markup. For example, the directive @image{image1.png, 300} would yield the following HTML:

  <div class="image" width="300">
  <img src="images/box-plot.png" aria-labelledby="box-plot-caption" aria-describedby="box-plot-description">
  <span id="box-plot-caption"        class="image-caption">distribution of US Income in 2003</span>
  <span id="box-plot-source"         class="image-source">retrieved from data.gov on April 15th, 2022</span>
  <span id="box-plot-description"    class="image-description">a box plot showing a right skew</span>
  </div>

With this implementation, we can use CSS to ensure that empty captions aren't rendered. Screenreaders will always vocalize using the label first, then the description. Visually-impaired users will hear "distribution of US Income in 2003, a box plot showing a right skew". Sighted users will see the image, with the text "distribution of US Income in 2003, retrieved from data.gov on April 15th, 2022".

---

When including an image, we can just use the standard @image tag, with arguments solely for filename and size. The build script can compare the filename to the dictionary, and include proper captions, alt text, etc. It can also produce a warning if an image isn't in the dictionary. And finally, it could combine all the dictionaries into a single citation page for each pathway, allowing us to show exactly where every piece of media we use came from.

This has the benefit of backwards-compatibility: none of the existing @image tags need to change. Everything will still work! Over time, we can move all the captions out of those tags and into these dictionaries.

[flannery-denny]

@schanzer nudging you to add the following to the spec above:

• standardized language for how to report the source of images of our own creation
• how you want dorai's script to handle those sources in the build

[flannery-denny]

@ds26gte please implement this on the image branch so that I can confirm that my changes are working as expected. Thanks!

[flannery-denny]

@retabak @schanzer

• I am noticing image descriptions that would not be very helpful to blind people. Encouraging you both to push yourselves to describe displays in more detail for screen readers. I did a full audit of all of our image descriptions last year, so I am sure these are new.
• We need to figure out what license is used for images from CODAP and for the data cycle images from Mobilizing IDS project and GAISE

[flannery-denny]

@schanzer is it intentional that the page "Do NYC Schools reflect the City's diversity?" was removed from the bar and pie chart lesson?

[flannery-denny]

• [x] bar and pie charts
• [ ] box plots (there are a lot of pyret plots - confirm that we want to move all of the image descriptions)
• [x] choosing your dataset (figure out how to credit ids)
• [ ] codap lessons (why don't images / descriptions match other lessons?)
• [ ] combinatorics (where do food images come from?)
• [ ] permutations (where do food images come from?)
• [x] composing table operations
• [x] computing needs all voices
• Do we want to use the same infrastucture for this lesson - it currently has its own way of crediting sources... and three separate image folders
• If so, is there a way to not have to list all thumbs as separate from pioneers?
• What to do about pioneers we didn't track sources/licenses for?
• [x] contracts
• [x] coordinates (where does ninja cat come from?)
• [ ] correlations
• [ ] custom scatter plots
• [ ] data collection (what's the source / license for screenshots from google?)
• [x] data cycle (figure out how to credit ids)
• [ ] debugging (bug & Hopper credits)
• [x] defining values
• [x] defining table functions (figure out how to credit ids)
• [ ] distance formula (what to do about game screenshots?)
• [x] ds-intro (who to credit for pyret and notice & wonder images?)
• [ ] ethics privacy bias (who to credit for atomic bomb?)
• [x] flags
• [ ] function compositions (what do about cubes?)
• [ ] function-definition-linear
• [ ] function notation
• [ ] functions can be linear
• [ ] functions dr
• [ ] functions examples definitions
• [x] functions for character animation (what to do about ninja cat dog)
• [ ] functions make life easier
• [ ] functions vertical line test
• [x] grouped samples (figure out how to credit ids)
• [ ] histograms
• [ ] histograms 2 (figure out how to credit ids)
• [ ] inequalities collision ( did we make the pythag images?)
• [ ] inequalities 1
• [ ] inequalities 2
• [ ] inequalities sam (masks? ninja cat?)
• [ ] linear regression (figure out how to credit ids)
• [x] lookups (figure out how to credit ids)
• [ ] making game images (fair use diagram, google)
• [ ] measures of center (figure out how to credit ids, codap)
• [x] numbers inside videogames ( ninja cat)
• [ ] order of operations (pemdas triangle)
• [ ] piecewise functions conditionals (images are of code - should they now be written using dorai's infrastructure instead?)
• [ ] player animation
• [ ] problem decomposition
• [ ] random samples (codap)
• [x] scatterplots ((figure out how to credit ids)
• [ ] simple data types (google & roller coaster image)
• [x] standard deviation (figure out how to credit ids)
• [ ] surface are rect prism
• [ ] table functions (codap)
• [x] threats to validity

Ninja Cat Images created for Bootstrap by Colleen Murphy

[flannery-denny]

@schanzer Flagging that the coordinates lesson includes images that are linked to with @link… not sure how this will work with the image infrastructure. Seems these could be pdfs instead of pngs linked to with @handout?

[schanzer]

@flannery-denny good call! I've implemented that change and pushed.

[flannery-denny]

@schanzer There are sometimes images that only exist in slides. Do we want to describe them, too? for an example, see functions for character animation, which only has one image, and it's only used in slides.

[schanzer]

@flannery-denny No need to describe them (this includes china-code.png in the defining values lesson)

[flannery-denny]

Does that mean we should leave them out of lesson-images.json or make a commented section of that file that says images only used in slides or list them in the file with "slides" in the 3 required fields?

[schanzer]

Leave them out.

(Also FYI - you can't add comments to a .json file)

[flannery-denny]

@ds26gte

• Thanks for volunteering to populate lesson-images.json files from information in the @image directives in lesson plans and workbook pages.

• Any image created by us can get the following attribution.

    "source" : "Created by the Bootstrap Team"
      "license" : "Creative Commons 4.0 - NC - SA"

• Please do not fill in bogus information for images that we did not create, such as images of food in @schanzer's combinatorics lesson.

• Please reach out for help if you run into stuff that feels unclear!

@schanzer Thanks for agreeing to take on the computing needs all voices lesson. Encouraging you to also look at the comments I made in the comment with the checkboxes above about images I had questions about the provenance of, as I expect Dorai will also not know what to do about them.

[flannery-denny]

@schanzer @ds26gte Had a chance to check out the image branch. Here are some things I have questions about:

• [ ] Should I be able to use @link in a caption?

"dollar-infographic-mona-chalabi.png" : { "caption" : "Made by Data Designer @link{https://www.instagram.com/monachalabi/, Mona Chalabi}", "description" : "Infographic: For every dollar a white man earns... visual shows parts of dollar bills corresponding to the comparable earning of other groups including (from highest to lowest earnings): Asian Men, Asian Women, White Women, Black Men, Native American Men, Black Women, Hispanic Men, Native American Women, Hispanic Women", "source" : "https://www.instagram.com/monachalabi/", "license" : "Used with permission from the creator." },

• [ ] Why don't I get a warning/error for images like these that are missing descriptions?
• [ ] Why are there so many images that are missing some of the four fields? The entire correlations lesson, for examples is missing descriptions... I thought the plan was to make sure that all authors thought about all 4 of these.
• [ ] Confirming it's intentional that we turned off the warning for missing sources for images like these and someone will turn those warnings back on in the future and clean up the files?

  "ethics-privacy-and-bias": { "AtomicBomb.png" : { "caption" : "", "description" : "Mushroom cloud from an atomic bomb", "source" : "", "license" : "" }

• [ ] Assuming that the images.js file gets built from other files, which means it's pointless for authors to edit it. Can we make that clear within the file somehow? I found myself feeling tempted to fix things when I looked at it.

[flannery-denny]

@ds26gte before we close this issue, let's add something about these files to the README doc, including wording standards for authors.