search

mitodl / open-discussions

BSD 3-Clause "New" or "Revised" License
10 stars 2 forks source link

Create and populate a search index with mappings for courses #1717

Closed mbertrand closed 5 years ago

mbertrand commented 5 years ago

Preliminary mapping:

COURSE_OBJECT_TYPE = {
    "course_id": {"type": "keyword"},
    "course_title": ENGLISH_TEXT_FIELD,
    "short_description": ENGLISH_TEXT_FIELD,
    "full_description": ENGLISH_TEXT_FIELD,
    "language": {"type": "keyword"},
    "level": {"type": "keyword"},
    "semester": {"type": "keyword"},
    "year": {"type": "integer"},
    "start_date": {"type": "date"},
    "end_date": {"type": "date"},
    "enrollment_start": {"type": "date"},
    "enrollment_end": {"type": "date"},
    "topics": {"type": "keyword"},
    "instructors": {"type": "keyword"},
    "price": {"type": "nested"},
    "course_image": {"type": "keyword"},
}
mbertrand commented 5 years ago

Sample course document in ES:

{
   "course_id":"MITx/3.072x/1T2015",
   "short_description":"An upper-level undergraduate engineering course that explores the structure and symmetry of materials and the profound effect that structure has on physical properties of materials.",
   "full_description": "<p>In <em>3.072x: Symmetry, Structure, and Tensor Properties of Materials</em>, you will study the underlying structures of materials and deepen your understanding of the relationship between the properties of materials and their structures. Topics include lattices, point groups, and space groups in both two and three dimensions; the use of symmetry in the tensor representation of crystal properties; and the relationship between crystalline structure and properties, including transport properties, piezoelectricity, and elasticity. Two course projects will allow students to explore their particular interests in greater depth. </p>\n<h3><strong>FAQ</strong></h3>\n<p><strong>Who can register for this course?</strong><br />\nUnfortunately, learners from Iran, Sudan and the Crimea region of Ukraine will not be able to register for this course at the present time. While edX has received a license from the U.S. Office of Foreign Assets Control (OFAC) to offer courses to learners from Iran and Sudan our license does not cover this course. Separately, EdX has applied for a license to offer courses to learners in the Crimea region of Ukraine, but we are awaiting a determination from OFAC on that application. We are deeply sorry the U.S. government has determined that we have to block these learners, and we are working diligently to rectify this situation as soon as possible.</p>",
   "language":"en-us",
   "semester":"spring",
   "year":2015,
   "start_date":"2015-02-10T00:00:00Z",
   "end_date":"2015-05-20T14:00:00Z",
   "enrollment_start":null,
   "enrollment_end":null,
   "course_title":"Symmetry, Structure and Tensor Properties of Materials",
   "course_image":"https://prod-discovery.edx-cdn.org/media/course/image/d0568514-4374-4166-a141-3355bd2af6f4-63d2d02a0d8e.small.jpg",
   "object_type":"course",
   "instructors":[
      "Eugene Fitzgerald",
      "Richard Taylor"
   ],
   "topics":[
      "Engineering",
      "Physics",
      "Chemistry"
   ],
   "prices":[
      {
         "price":0,
         "mode":"honor"
      },
      {
         "price":100,
         "mode":"verified"
      }
   ]
}
Ferdi commented 5 years ago

@mbertrand in the context of search, maybe full_description is more useful than short description ? or maybe both ?

mbertrand commented 5 years ago

I'll include both

pdpinch commented 5 years ago

I have a couple of questions / thoughts:

mbertrand commented 5 years ago

Here's a sample OCW doc:

 {
    "course_id": "4007b067a3db229b6e789bfbb56a409d",
    "short_description": "General principles of motor control in biological systems. Structure and function of sensory receptors. Muscle structure and reflex arcs. Spinal cord. Locomotion. Oculomotor control. Cerebellar structure and function. Motor thalamus. Basal ganglia. Somatosensory cortex: maps and neuronal properties. Cortical plasticity. Motor psychophysics and computational approaches to motor control, and motor planning.",
    "full_description": null,
    "platform": "ocw",
    "language": null,
    "semester": "Spring",
    "year": 2002,
    "start_date": null,
    "end_date": null,
    "enrollment_start": null,
    "enrollment_end": null,
    "course_title": "Somatosensory and Motor Systems",
    "course_image": "",
    "object_type": "course",
    "instructors": [
        "Emilio Bizzi",
        "Mriganka Sur",
        "Peter Schiller",
        "Ann Graybiel"
    ],
    "topics": [],
    "prices": [{
        "price": 0,
        "mode": "audit"
    }]
 }
mbertrand commented 5 years ago

@pdpinch:

pdpinch commented 5 years ago

department_numbers is a fine identifier for OCW, but it would be nice to expand that to the full text of the department, because it would help search.

For OCW, I am confused by the master_course_number examples that you mention. I'm expecting something more like 6.002 Maybe you can point me to a complete example?

For edX, we'll have to extract the number from the course id/key. I can help with a regex, but it's basically the middle part of the key (either + or / separated) and cut off the _ and anything that follows. So course-v1:MITx+6.341x_2+2T2016 => 6.341x and MITx/8.MReV/2013_Summer => 8.MReV).

pdpinch commented 5 years ago

department_numbers is a fine identifier for OCW, but it would be nice to expand that to the full text of the department, because it would help search.

For OCW, I am confused by the master_course_number examples that you mention. I'm expecting something more like 6.002 Maybe you can point me to a complete example?

For edX, we'll have to extract the number from the course id/key. I can help with a regex, but it's basically the middle part of the key (either + or / separated) and cut off the _ and anything that follows. So course-v1:MITx+6.341x_2+2T2016 => 6.341x and MITx/8.MReV/2013_Summer => 8.MReV).

pdpinch commented 5 years ago

Some feedback on the sample OCW doc:

mbertrand commented 5 years ago

Here is the raw_json from an OCW course, the master_course_number is 430

   'ocw_feature_notes': '',
   'ocw_feature_url': '',
   'ocw_speciality': '',
   'ocw_subfeature': 'Art History'},
  {'ocw_feature': 'Fine Arts',
   'ocw_feature_notes': '',
   'ocw_feature_url': '',
   'ocw_speciality': '',
   'ocw_subfeature': 'Media Studies'},
  {'ocw_feature': 'Fine Arts',
   'ocw_feature_notes': '',
   'ocw_feature_url': '',
   'ocw_speciality': 'Digital Media',
   'ocw_subfeature': 'Media Studies'}],
 'course_embedded_media': {},
 'course_features': [{'ocw_feature': 'Assignments',
   'ocw_feature_notes': '',
   'ocw_feature_url': '/courses/literature/21l-430-popular-culture-and-narrative-serial-storytelling-spring-2013/assignments/',
   'ocw_speciality': '',
   'ocw_subfeature': 'written (no examples)'}],
 'course_files': [{'alt_text': 'Popular Culture and Narrative: Serial Storytelling',
   'caption': 'Detail of a disc depicting a couple waltzing, created by Eadweard Muybridge for the phenakistoscope, an early motion picture machine that produced the illusion of movement through the rapid progression of a series of images.',
   'credit': 'Image via wikimedia commons.',
   'description': 'Detail of a disc depicting a couple waltzing, created by Eadweard Muybridge for the phenakistoscope, an early motion picture machine that produced the illusion of movement through the rapid progression of a series of images. Image via wikimedia commons.',
   'file_location': 'https://mb-odl-video-dev-mb-1-thumbnails.s3.amazonaws.com/21l-430-popular-culture-and-narrative-serial-storytelling-spring-2013/06ed0c2eba8b8dad5d257234e4178bde_21-430s13-th.jpg',
   'file_type': 'image/jpeg',
   'parent_uid': 'f42d61319f1b751daba0ce873de26e21',
   'platform_requirements': 'Any number of image viewers/editors can be used to display these .jpg image files, including most html browsers.',
   'title': 'Series of illustrations of a woman and man dancing the waltz',
   'uid': '06ed0c2eba8b8dad5d257234e4178bde'},
  {'alt_text': None,
   'caption': None,
   'credit': None,
   'description': "Prefaces to Charles Dickens' The Pickwick Papers.",
   'file_location': 'https://mb-odl-video-dev-mb-1-thumbnails.s3.amazonaws.com/21l-430-popular-culture-and-narrative-serial-storytelling-spring-2013/24d925ce049395d86959d29e19caa32b_MIT21L_430S13_dickens_2.pdf',
   'file_type': 'application/pdf',
   'parent_uid': '367a04c5c9835df8074e920fddee31ad',
   'platform_requirements': 'Adobe Reader software is required to view this .pdf file.',
   'title': 'Collected Prefaces to the Pickwick Papers',
   'uid': '24d925ce049395d86959d29e19caa32b'},
  {'alt_text': None,
   'caption': None,
   'credit': None,
   'description': "Chapters 1 and 2 of Charles Dickens' The Pickwick Papers.",
   'file_location': 'https://mb-odl-video-dev-mb-1-thumbnails.s3.amazonaws.com/21l-430-popular-culture-and-narrative-serial-storytelling-spring-2013/26a79bb32740da724657896f332ef750_MIT21L_430S13_pickwick_1.pdf',
   'file_type': 'application/pdf',
   'parent_uid': '367a04c5c9835df8074e920fddee31ad',
   'platform_requirements': 'Adobe Reader software is required to view this .pdf file.',
   'title': 'The Pickwick Papers, Chapters 1 & 2',
   'uid': '26a79bb32740da724657896f332ef750'},
  {'alt_text': None,
   'caption': None,
   'credit': None,
   'description': 'Arthur Conan Doyle\'s "The Adventure of the Speckled Band."  The Strand Magazine, February 1892, 108-123.',
   'file_location': 'https://mb-odl-video-dev-mb-1-thumbnails.s3.amazonaws.com/21l-430-popular-culture-and-narrative-serial-storytelling-spring-2013/fc1c78ed095eb824c7124468e8a400e0_MIT21L_430S13_Adventure.pdf',
   'file_type': 'application/pdf',
   'parent_uid': '367a04c5c9835df8074e920fddee31ad',
   'platform_requirements': 'Adobe Reader software is required to view this .pdf file.',
   'title': 'The Adventure of the Speckled Band',
   'uid': 'fc1c78ed095eb824c7124468e8a400e0'},
  {'alt_text': 'Series of illustrations of a woman and man dancing the waltz.',
   'caption': 'Detail of a disc depicting a couple waltzing, created by Eadweard Muybridge for the phenakistoscope, an early motion picture machine that produced the illusion of movement through the rapid progression of a series of images.',
   'credit': 'Image via wikimedia commons.',
   'description': 'Detail of a disc depicting a couple waltzing, created by Eadweard Muybridge for the phenakistoscope, an early motion picture machine that produced the illusion of movement through the rapid progression of a series of images. Image via wikimedia commons.',
   'file_location': 'https://mb-odl-video-dev-mb-1-thumbnails.s3.amazonaws.com/21l-430-popular-culture-and-narrative-serial-storytelling-spring-2013/9a0d3854209d52a515e29c722d6fe500_21l-430s13.jpg',
   'file_type': 'image/jpeg',
   'parent_uid': 'f42d61319f1b751daba0ce873de26e21',
   'platform_requirements': 'Any number of image viewers/editors can be used to display these .jpg image files, including most html browsers.',
   'title': 'Series of illustrations of a woman and man dancing the waltz',
   'uid': '9a0d3854209d52a515e29c722d6fe500'}],
 'course_foreign_files': [{'file_location': 'https://mb-odl-video-dev-mb-1-thumbnails.s3.amazonaws.com/21l-430-popular-culture-and-narrative-serial-storytelling-spring-2013/MIT21L_430S13_Rembrandt.pdf',
   'link': 'http://ocw.mit.edu/ans7870/21l/21l.430/s13/MIT21L_430S13_Rembrandt.pdf',
   'parent_uid': '367a04c5c9835df8074e920fddee31ad'},
  {'file_location': 'https://mb-odl-video-dev-mb-1-thumbnails.s3.amazonaws.com/21l-430-popular-culture-and-narrative-serial-storytelling-spring-2013/MIT21L_430S13_Goya_1.pdf',
   'link': 'http://ocw.mit.edu/ans7870/21l/21l.430/s13/MIT21L_430S13_Goya_1.pdf',
   'parent_uid': '367a04c5c9835df8074e920fddee31ad'},
  {'file_location': 'https://mb-odl-video-dev-mb-1-thumbnails.s3.amazonaws.com/21l-430-popular-culture-and-narrative-serial-storytelling-spring-2013/MIT21L_430S13_Goya_2.pdf',
   'link': 'http://ocw.mit.edu/ans7870/21l/21l.430/s13/MIT21L_430S13_Goya_2.pdf',
   'parent_uid': '367a04c5c9835df8074e920fddee31ad'},
  {'file_location': 'https://mb-odl-video-dev-mb-1-thumbnails.s3.amazonaws.com/21l-430-popular-culture-and-narrative-serial-storytelling-spring-2013/MIT21L_430S13_Goya_3.pdf',
   'link': 'http://ocw.mit.edu/ans7870/21l/21l.430/s13/MIT21L_430S13_Goya_3.pdf',
   'parent_uid': '367a04c5c9835df8074e920fddee31ad'},
  {'file_location': 'https://mb-odl-video-dev-mb-1-thumbnails.s3.amazonaws.com/21l-430-popular-culture-and-narrative-serial-storytelling-spring-2013/MIT21L_430S13_Holmes-new.pdf',
   'link': 'http://ocw.mit.edu/ans7870/21l/21l.430/s13/MIT21L_430S13_Holmes-new.pdf',
   'parent_uid': '367a04c5c9835df8074e920fddee31ad'}],
 'course_level': 'Undergraduate',
 'course_owner': 'lfp',
 'course_pages': [{'description': "<p>Serial Storytelling examines the ways the passing and unfolding of time structures narratives in a range of media. From Rembrandt's lifetime of self-portraits to The Wire, Charles Dickens' Pickwick Papers to contemporary journalism and reportage, we will focus on the relationships between popular culture and art, the problems of evaluation and audience, and the ways these works function within their social context.</p>",
   'parent_uid': 'f42d61319f1b751daba0ce873de26e21',
   'short_url': 'index.htm',
   'text': None,
   'title': 'Course Home',
   'type': 'CourseHomeSection',
   'uid': '2d300cdcabd1df5772a5907a34d232ea',
   'url': '/courses/literature/21l-430-popular-culture-and-narrative-serial-storytelling-spring-2013/index.htm'},
  {'description': 'This section provides information on course meeting times, required texts, major assignments, readings and other materials, and the schedule of seminar topics.',
   'parent_uid': 'f42d61319f1b751daba0ce873de26e21',
   'short_url': 'syllabus-and-calendar',
   'text': '<h2 class="subhead">Course Meeting Times</h2> <p>Seminars: 2 sessions / week, 2 hours / session</p> <h2 class="subhead">Required Texts</h2> <p>[AT] = <a href="http://www.amazon.com/exec/obidos/ASIN/0393979261/ref=nosim/mitopencourse-20"><img border="0" style="vertical-align: middle;" src="/images/a_logo_17.gif" alt="Buy at Amazon" /></a> Tennyson, Alfred. <em>In </em><em>Memoriam</em>. W.W. Norton &amp; Company, 2003. ISBN: 9780393979268.</p> <p>[DS] = <a href="http://www.imdb.com/title/tt0306414/"><em>The Wire: The Complete First Season</em></a>. Created by David Simon. Home Box Office, Inc., 2004. DVD.</p> <p><a href="http://www.amazon.com/exec/obidos/ASIN/1576876020/ref=nosim/mitopencourse-20"><img border="0" style="vertical-align: middle;" src="/images/a_logo_17.gif" alt="Buy at Amazon" /></a> DeLyria, Joy, and Sean Michael Robinson. <em> Down in the Hole: The UnWired World of H. B. Ogden</em>. powerHouse Books, 2012. ISBN: 9781576876022.</p> <h2 class="subhead">Major Assignments</h2> <ul> <li><em>The Wire</em> viewing journal</li> <li>Two shorter essays</li> <li>Class presentations on final project</li> <li>Final essay</li> </ul> <h2 class="subhead">Calendar</h2> <div class="maintabletemplate"><table summary="See table caption for summary."> <caption class="invisible">Course calendar.</caption> <thead> <tr> <th scope="col">SES&nbsp;#</th> <th scope="col">TOPICS</th> <th scope="col">READINGS/MATERIALS</th> </tr> </thead> <tbody> <tr class="row"> <td>1</td> <td>Introducing the Serial</td> <td>&nbsp;</td> </tr> <tr class="alt-row"> <td>2</td> <td>Thresholds of Serial: Rembrandt</td> <td><p><a href="http://ocw.mit.edu/ans7870/21l/21l.430/s13/MIT21L_430S13_Rembrandt.pdf"><img alt="This resource may not render correctly in a screen reader." src="/images/inacessible.gif" />Rembrandt: The Self-Portraits (PDF - 21.1MB)</a><br /> <a href="http://www.rembrandtpainting.net/complete_catalogue/start_self_portraits.htm">Gallery at Rembrandtpainting.net</a></p> <p>Vincent, Jean. &quot;<a href="http://thinking-about-art.blogspot.com/2007/11/aging-face-of-rembrandt.html">The Aging Face of Rembrandt</a>.&quot; <em>Thinking About Art</em>. Blog post with embedded video not produced by the author. November 27, 2011.</p> <p>Stevens, Wallace. &quot;<a href="http://www.writing.upenn.edu/~afilreis/88/stevens-13ways.html">Thirteen Ways of Looking at a Blackbird</a>.&quot; 1917.</p></td> </tr> <tr class="row"> <td>3</td> <td>Thresholds of Serial: Rembrandt (cont.)</td> <td><p>[DS] Episodes 1 &amp; 2<em>.<br /> </em></p> <p><a href="http://www.amazon.com/exec/obidos/ASIN/0691040613/ref=nosim/mitopencourse-20"><img border="0" style="vertical-align: middle;" src="/images/a_logo_17.gif" alt="Buy at Amazon" /></a> Chapman, H. Perry. &quot;The Self Fashioned.&quot; In <em>Rembrandt\'s Self-Portraits: A Study in Seventeenth-Century Identity</em>. Princeton University Press, 1990. ISBN: 9780691040615.</p> <p>Koerner, Joseph Leo. &quot;<a href="http://www.jstor.org/stable/20166751">Rembrandt and the Epiphany of the Face</a>.&quot; <em>RES: Anthropology and Aesthetics</em>, no. 12 (1986): 5&ndash;32.</p> <p>Schwartz, Frederic. &quot;\'<a href="http://www.jstor.org/stable/20166816">The Motions of the Countenance\': Rembrandt\'s Early Portraits and the Tronie</a>.&quot; <em>RES: Anthropology and Aesthetics</em>, no. 17/18 (1989): 89&ndash;116.</p> <p>Auden, W. H. &quot;<a href="http://english.emory.edu/classes/paintings&amp;poems/auden.html">Musée des Beaux Arts</a>.&quot; 1938.</p></td> </tr> <tr class="alt-row"> <td>4</td> <td>Sequence and Argument: Goya</td> <td><p>Goya, Francisco. <a href="http://www.richardharrisartcollection.com/portfolio-view/francisco-goya-2/"><em>The Disasters of War</em> (1810&ndash;1820)</a>.<br /> <img src="/images/inacessible.gif" alt="This resource may not render correctly in a screen reader." /><a href="http://ocw.mit.edu/ans7870/21l/21l.430/s13/MIT21L_430S13_Goya_1.pdf">Prints 1&ndash;30 (PDF - 17.2MB)</a><br /> <img src="/images/inacessible.gif" alt="This resource may not render correctly in a screen reader." /><a href="http://ocw.mit.edu/ans7870/21l/21l.430/s13/MIT21L_430S13_Goya_2.pdf">Prints 31&ndash;60 (PDF - 17.5MB)</a><br /> <img src="/images/inacessible.gif" alt="This resource may not render correctly in a screen reader." /><a href="http://ocw.mit.edu/ans7870/21l/21l.430/s13/MIT21L_430S13_Goya_3.pdf">Prints 61&ndash;83 (PDF - 14.7MB)</a><br /> <a href="http://commons.wikimedia.org/wiki/Los_desastres_de_la_guerra">Gallery at Wikimedia Commons</a></p> <p><a href="http://www.amazon.com/exec/obidos/ASIN/ 0375711287/ref=nosim/mitopencourse-20"><img border="0" style="vertical-align: middle;" src="/images/a_logo_17.gif" alt="Buy at Amazon" /></a> Hughes, Robert. &quot;War with Napoleon.&quot; In <em>Goya</em>. Knopf, 2006. ISBN: 9780375711282.</p></td> </tr> <tr class="row"> <td>5</td> <td>Sequence and Argument: Goya (cont.)</td> <td><p>[DS] Episode 3.</p> <p>Goya, <em>The Disasters of War (cont.)</em></p> <p><a href="http://www.amazon.com/exec/obidos/ASIN/0299137007/ref=nosim/mitopencourse-20"><img border="0" style="vertical-align: middle;" src="/images/a_logo_17.gif" alt="Buy at Amazon" /></a> Hults, Linda C. &quot;The Print and Socio-Political Reform: Hogarth and His Heirs and Goya.&quot; In <em>The Print in the Western World: An Introductory History</em>. University of Wisconsin Press, 1996. ISBN: 9780299137007.</p></td> </tr> <tr class="alt-row"> <td>6</td> <td>Inventing the Multiverse: Comics</td> <td><p>[DS] Episode 4.</p> <p>Eco, Umberto, and Natalie Chilton. &quot;<a href="http://www.jstor.org/stable/464920">The Myth of Superman</a>.&quot; <em>Diacritics</em> 2, no. 1 (1972): 14&ndash;22.</p> <p>Burgas, Greg. &quot;<a href="http://goodcomics.comicbookresources.com/2007/10/25/compressed-storytelling-versus-decompressed-storytelling-pros-and-cons/">Compressed Storytelling versus Decompressed Storytelling</a>.&quot; Reblogged by Brian Cronin at ComicBookResources.com. October 25, 2007.</p> <p><a href="http://www.amazon.com/exec/obidos/ASIN/1433101076/ref=nosim/mitopencourse-20"><img border="0" style="vertical-align: middle;" src="/images/a_logo_17.gif" alt="Buy at Amazon" /></a> Rhoades, Shirrel. Chapters 2, 4, 5, and 6 in <em>A Complete History of American Comic Books</em>. Peter Lang, 2008. ISBN: 9781433101076. [Preview with <a href="http://books.google.com/books?id=O16BXbITZwEC&amp;&amp;pg=PAfrontcover">Google Books</a>]</p></td> </tr> <tr class="row"> <td>7</td> <td>Motion Studies: Photography</td> <td><p><a href="http://www.amazon.com/exec/obidos/ASIN/0674007840/ref=nosim/mitopencourse-20"><img border="0" style="vertical-align: middle;" src="/images/a_logo_17.gif" alt="Buy at Amazon" /></a> Doane, Mary Ann. &quot;Zeno\'s Paradox: The Emergence of Cinematic Time,&quot; and &quot;Temporality, Storage, Legibility: Freud, Marey, and the Cinema.&quot; In <em>The Emergence of Cinematic Time: Modernity, Contingency, the Archive</em>. Harvard University Press, 2002. ISBN: 9780674007840.</p> <p><a href="http://www.amazon.com/exec/obidos/ASIN/0767929403/ref=nosim/mitopencourse-20"><img border="0" style="vertical-align: middle;" src="/images/a_logo_17.gif" alt="Buy at Amazon" /></a> Ball, Edward. <em> The Inventor and the Tycoon</em>. Anchor, 2013, pp. 15&ndash;21, 71&ndash;2, 119&ndash;25, and 306&ndash;29. ISBN: 9780767929400.</p> <p>Edweard Muybridge, photographs.</p> <p>Duchamp, Marcel. <a href="http://2.bp.blogspot.com/_-XqtscwAOzg/TLg6sYaOWuI/AAAAAAAAAwA/kNB8kOJOWu0/s1600/nude+_no2.jpg"><em>Nude Descending a Staircase, No. 2</em></a>. 1912.</p></td> </tr> <tr class="alt-row"> <td>8</td> <td>The Challenge of Formula: Journalism<br /> <em>Guest speaker: Alexandra Garcia, The Washington Post</em></td> <td><p>[DS] Episode 5.</p> <p>Darnton, Robert. &quot;<a href="http://www.jstor.org/stable/20024337">Writing News and Telling Stories</a>.&quot; <em>Daedalus</em> 104, no. 2 (1975): 175&ndash;94.</p> <p>&quot;<a href="http://www.washingtonpost.com/wp-srv/special/artsandliving/scene-in/index.html?order=chronological&amp;episode=5">Scene In</a>&quot; (4 episodes). WashingtonPost.Com. Web video series.</p> <p>Heath, Chris. &quot;<a href="http://www.gq.com/news-politics/newsmakers/201203/terry-thompson-ohio-zoo-massacre-chris-heath-gq-february-2012">18 Tigers, 17 Lions, 8 Bears, 3 Cougars, 2 Wolves, 1 Baboon, 1 Macaque, and 1 Man Dead in Ohio</a>.&quot; <em>GQ</em>, March 2012.</p> <p>Jones, Chris. &quot;<a href="http://www.esquire.com/features/zanesville-0312">Animals</a>.&quot; <em>Esquire</em>, March 2012.</p> <p>Ogles, Jonah. &quot;<a href="http://www.cincinnatimagazine.com/features/man-or-beast3/">Man or Beast?</a>&quot; <em> Cincinnati</em>, March 1, 2012.</p></td> </tr> <tr class="row"> <td>9</td> <td>The Challenge of Formula: Journalism (cont.)</td> <td><p><a href="http://www.amazon.com/exec/obidos/ASIN/0156033070/ref=nosim/mitopencourse-20"><img border="0" style="vertical-align: middle;" src="/images/a_logo_17.gif" alt="Buy at Amazon" /></a> Orwell, George. &quot;Boys\' Weeklies.&quot; In <em>All Art Is Propaganda</em>. Mariner Books, 2009. ISBN: 9780156033077.</p> <p><a href="https://mitpress.mit.edu/9780262631594"><img alt="Buy at MIT Press" src="/images/mp_logo.gif" /> </a><a href="http://www.amazon.com/exec/obidos/ASIN/0262631598/ref=nosim/mitopencourse-20"><img border="0" style="vertical-align: middle;" src="/images/a_logo_17.gif" alt="Buy at Amazon" /></a> McLuhan, Marshall. &quot;The Medium is the Message.&quot; In <em>Understanding Media: The Extensions of Man</em>. The MIT Press, 1964. ISBN: 9780262631594.</p> <p>Stevens, Wallace. &quot;<a href="http://www.poetryfoundation.org/poetrymagazine/poem/3778">Anecdote of the Jar</a>.&quot; 1919.</p></td> </tr> <tr class="alt-row"> <td>10</td> <td>Stasis and Progress: Tennyson, <em>In Memoriam</em></td> <td><p>[DS] Episode 6.</p> <p>[AT] Prologue. pp. 1&ndash;7, 9, 18, 27&ndash;30, 34, and 47&ndash;8.</p> <p><a href="http://www.amazon.com/exec/obidos/ASIN/0801834716/ref=nosim/mitopencourse-20"><img border="0" style="vertical-align: middle;" src="/images/a_logo_17.gif" alt="Buy at Amazon" /></a> Sacks, Peter. &quot;Interpreting the Genre: The Elegy and the Work of Mourning.&quot; In <em>The English Elegy</em>. The Johns Hopkins University Press, 1987. ISBN: 9780801834714.</p> <p><a href="http://www.amazon.com/exec/obidos/ASIN/0156005557/ref=nosim/mitopencourse-20"><img border="0" style="vertical-align: middle;" src="/images/a_logo_17.gif" alt="Buy at Amazon" /></a> Woolf, Virginia. <em>Mrs. Dalloway</em>. Harcourt Brace &amp; Company, 1997, pp. 73&ndash;4. ISBN: 9780156005555.</p></td> </tr> <tr class="row"> <td>11</td> <td>Stasis and Progress: Tennyson, <em>In Memoriam </em>(cont.)</td> <td><p>[AT] &quot;Epilogue.&quot; pp. 50, 54&ndash;6, 69, 78, 104&ndash;7, 117&ndash;20, and 128&ndash;31.</p> <p><a href="http://www.amazon.com/exec/obidos/ASIN/0198127472/ref=nosim/mitopencourse-20"><img border="0" style="vertical-align: middle;" src="/images/a_logo_17.gif" alt="Buy at Amazon" /></a> Tennyson, Alfred. &quot;Introduction.&quot; In<em> In Memoriam</em>. Edited by Susan Shatto and Marion Shaw. Clarendon Press, 1982. ISBN: 9780198127475.</p> <p>Landseer, Edwin Henry. &quot;<a href="http://www.english-for-students.com/man-proposes-but-god-disposes.html">Man Proposes God Disposes</a>.&quot; 1864.</p> <p>Dyce, William. &quot;<a href="http://www.tate.org.uk/art/images/work/N/N01/N01407_10.jpg">Pegwell Bay, Kent&mdash;A Recollection of October 5th 1858</a>.&quot; 1860.</p></td> </tr> <tr class="alt-row"> <td>12</td> <td>The Hero\'s Journey: Tennyson, <em>Morte D\'Arthur</em></td> <td><p>[DS] Episode 7.</p> <p>Tennyson, Alfred Lord. &quot;<a href="http://d.lib.rochester.edu/camelot/text/tennyson-epic">The Epic</a>,&quot; and  &quot;<a href="http://d.lib.rochester.edu/camelot/text/tennyson-epic">Morte D\'Arthur</a>.&quot; 1842.</p></td> </tr> <tr class="row"> <td>13</td> <td>Whole Into Parts: Charles Dickens, <em>The Pickwick Papers</em></td> <td><p>[DS] Episode 8.</p> <p>Dickens, Charles. <a href="/courses/literature/21l-430-popular-culture-and-narrative-serial-storytelling-spring-2013/syllabus-and-calendar/MIT21L_430S13_dickens_2.pdf">Prefaces (PDF)</a> in <em>The Pickwick Papers.</em></p> <p>&mdash;&mdash;&mdash;. <a href="/courses/literature/21l-430-popular-culture-and-narrative-serial-storytelling-spring-2013/syllabus-and-calendar/MIT21L_430S13_pickwick_1.pdf">Chapters 1&ndash;2 (PDF)</a> in <em>The Pickwick Papers</em>.<br />             (Full text at <a href="http://www.gutenberg.org/ebooks/580">Project Gutenberg</a>.)</p> <p><a href="http://www.amazon.com/exec/obidos/ASIN/0813913144/ref=nosim/mitopencourse-20"><img border="0" style="vertical-align: middle;" src="/images/a_logo_17.gif" alt="Buy at Amazon" /></a> Hughes, Linda K., and Michael Lund. &quot;Introducing the Serial.&quot; In <em>The Victorian Serial</em>. University of Virginia Press, 1991. ISBN: 9780813913148.</p></td> </tr> <tr class="alt-row"> <td>14&ndash;16</td> <td>Whole Into Parts: Charles Dickens, <em>Bleak House</em></td> <td><p>Dickens, Charles. <a href="http://www.gutenberg.org/ebooks/1023"><em>Bleak House</em></a>. 1853.</p> <p>[DS] Episode 9.</p> <p><a href="http://www.amazon.com/exec/obidos/ASIN/0822311100/ref=nosim/mitopencourse-20"><img border="0" style="vertical-align: middle;" src="/images/a_logo_17.gif" alt="Buy at Amazon" /></a> Miller, J. Hillis. &quot;Interpretation in Bleak House.&quot; In <em>Victorian Subjects</em>. Duke University Press, 1990. ISBN: 9780822311102.</p> <p><a href="http://www.amazon.com/exec/obidos/ASIN/0814204597/ref=nosim/mitopencourse-20"><img border="0" style="vertical-align: middle;" src="/images/a_logo_17.gif" alt="Buy at Amazon" /></a> Altick, Richard D. &quot;English Publishing and the Mass Audience in 1852.&quot; In <em>Writers, Readers, and Occasions</em>. Ohio State University Press, 1988. ISBN: 9780814204597.</p></td> </tr> <tr class="row"> <td>17</td> <td>The Reader Replies: Sherlock Holmes</td> <td><p>Moretti, Franco. <a href="https://msu.edu/course/eng/487/johnsen/61.1moretti.pdf">&quot;The Slaughterhouse of Literature.&quot; (PDF)</a> <em>MLQ: Modern Language Quarterly</em> 61, no. 1 (2000): 207&ndash;27.</p> <p>Conan Doyle, Arthur. <a href="/courses/literature/21l-430-popular-culture-and-narrative-serial-storytelling-spring-2013/syllabus-and-calendar/MIT21L_430S13_Adventure.pdf">&quot;The Adventure of the Speckled Band&quot; (PDF - 1.5MB)</a>.&nbsp; <em>The Strand Magazine</em>, February 1892, 108-123.             (note: this PDF is a scan of the original 1892 publication)</p> <p>Conan Doyle, Arthur.&nbsp; &quot;<a href="http://www.gutenberg.org/ebooks/1661">The Adventure of the Red-Headed League</a>.&quot;&nbsp; In <em>The Adventures of Sherlock Holmes</em>. George Newnes Ltd, 1892.</p> <p>Conan Doyle, Arthur. &quot;<a href="http://www.gutenberg.org/ebooks/834">The Final Problem</a>.&quot; In <em>The Memoirs of Sherlock Holmes</em>.&nbsp; George Newnes Ltd, 1894.</p> <p><a href="http://ocw.mit.edu/ans7870/21l/21l.430/s13/MIT21L_430S13_Holmes-new.pdf">Selected excerpts (PDF - 13.2MB)</a> from other Holmes Stories.</p> <h3 class="subsubhead">Optional Reading</h3> <p>Orwell, George. &quot;<a href="http://orwell.ru/library/articles/decline/english/e_doem">Decline of the English Murder</a>,&quot; <em>Tribune</em>, February 15, 1946.</p></td> </tr> <tr class="alt-row"> <td>18</td> <td>The Reader Replies: Sherlock Holmes (cont.)</td> <td><p>[DS] Episode 10.</p> <p>Jurgensen, John. &quot;<a href="http://online.wsj.com/news/articles/SB10001424052702303740704577521300806686174">Binge Viewing: TV\'s Lost Weekends</a>,&quot; <em>The Wall Street Journal</em>, July 13, 2012.</p> <p>Armisen, Fred, and Carrie Brownstein. &quot;<a href="https://www.youtube.com/watch?v=aQttrkzWOo4">Battlestar Galactica</a>.&quot; <em>Portlandia</em>. Selection of comedic sketch originally produced for prestige television, subsequently made available for marketing purposes. Broadway Video Television and IFC Original Productions, 2011.</p> <p>Armisen, Fred, Carrie Brownstein, et al. &quot;<a href="http://www.youtube.com/watch?v=dM0AI5T4wYo">Spoiler Alert</a>.&quot; <em>Portlandia</em>. Selection of comedic sketch originally produced for prestige television, subsequently made available for marketing purposes. Broadway Video Television and IFC Original Productions, 2013.</p> <p>Garcia, Alexandra, Jason Spingarn-Koff, et al. &quot;<a href="http://www.ew.com/article/2013/01/27/downton-abbey-season-4-spoilers">Downton Abbey Spoiler Alert</a>.&quot; <em>Convergence Journalism? Emerging Documentary and Multimedia Forms of News</em>. One minute segment of recorded forum session. MIT Communications Forum, 2013, 01:27:00&ndash;01:28:00.</p> <p><a href="http://www.amazon.com/exec/obidos/ASIN/0822353806/ref=nosim/mitopencourse-20"><img border="0" style="vertical-align: middle;" src="/images/a_logo_17.gif" alt="Buy at Amazon" /></a> Condry, Ian. &quot;Making a Cutting-Edge Anime Studio: The Value of the Gutter.&quot; In <em>The Soul of Anime</em>. Duke University Press Books, 2013. ISBN: 9780822353805.</p></td> </tr> <tr class="row"> <td>19</td> <td>Parts Into Whole: Charlie Chaplin, <em>The Gold Rush</em></td> <td><p>Chaplin, Charlie. Selected short films &amp; <a href="http://www.youtube.com/watch?v=nt-_DXC-aik"><em>The Gold Rush</em></a>. 1925.</p> <p><a href="http://www.amazon.com/exec/obidos/ASIN/0023770759/ref=nosim/mitopencourse-20"><img border="0" style="vertical-align: middle;" src="/images/a_logo_17.gif" alt="Buy at Amazon" /></a> Mast, Gerald and Bruce F. Kawin. &quot;Mark Sennett and the Chaplin Shorts,&quot; and &quot;The Comics.&quot; In <em>A Short History of the Movies</em>. 6th ed. Allyn &amp; Bacon, 1996. ISBN: 9780023770753.</p></td> </tr> <tr class="alt-row"> <td>20</td> <td>Parts Into Whole: Charlie Chaplin, <em>Modern Times</em></td> <td><p>[DS] Episode 11.</p> <p><a href="http://www.imdb.com/title/tt0027977/"><em>Modern Times</em></a>. Directed by Charlie Chaplin. Black and White, 87 min. 1936.</p></td> </tr> <tr class="row"> <td>21</td> <td>Remediation: The Wire</td> <td><p>[DS] Episode 12.</p> <p>Bolter, Jay David, and Richard Grusin. <a href="http://lmc.gatech.edu/~objork3/1101/fall07/remediation.pdf">&quot;Remediation.&quot; (PDF)</a> <em>Configurations</em> 4, no. 3 (1996): 311&ndash;58.</p></td> </tr> <tr class="alt-row"> <td>22</td> <td>Remediation: The Wire (cont.)</td> <td><p>[DS] Episode 13.</p> <p>Mittell, Jason. &quot;<a href="http://justtv.wordpress.com/2007/05/22/the-wire-and-the-serial-procedural-an-essay-in-progress/">All in the Game: The Wire, Serial Storytelling, and Procedural Logic</a>.&quot; 2007.</p> <p>Anderson, Paul Allen. &quot;<a href="http://dx.doi.org/10.1353/crt.2010.0043">\'The Game Is the Game\': Tautology and Allegory in The Wire</a>.&quot; <em> Criticism</em> 52, no. 3&ndash;4 (2010): 373&ndash;98.</p></td> </tr> <tr class="row"> <td>23</td> <td>Remediation: The Wire (cont.)</td> <td><p><a href="http://www.amazon.com/exec/obidos/ASIN/1576876020/ref=nosim/mitopencourse-20"><img border="0" style="vertical-align: middle;" src="/images/a_logo_17.gif" alt="Buy at Amazon" /></a> DeLyria, Joy, and Sean Michael Robinson. <em>Down in the Hole: The UnWired World of H. B. Ogden</em>. powerHouse Books, 2012. ISBN: 9781576876022.</p> <p>Owen, Paul. &quot;<a href="http://www.theguardian.com/media/organgrinder/2010/apr/06/wire-charles-dickens-season-5-episode-8-dickensian">The Wire Re-Up: Season Five, Episode Eight&mdash;The Dickensian Aspects</a>.&quot; <em>TheGuardian.com: Organ Grinder Blog</em>, April 5, 2010.</p></td> </tr> <tr class="alt-row"> <td>24&ndash;25</td> <td>Class presentations</td> <td>&nbsp;</td> </tr> </tbody> </table></div> <p>&nbsp;</p>',
   'title': 'Syllabus and Calendar',
   'type': 'CourseSection',
   'uid': '367a04c5c9835df8074e920fddee31ad',
   'url': '/courses/literature/21l-430-popular-culture-and-narrative-serial-storytelling-spring-2013/syllabus-and-calendar'},
  {'description': 'This section provides information on the writing assignments for the course.',
   'parent_uid': 'f42d61319f1b751daba0ce873de26e21',
   'short_url': 'assignments',
   'text': '<h2 class="subhead">Topics: Essay 1 (5 Pages)</h2> <p><em>Due: Session 9</em></p> <ol>     <li><strong>Thirteen Ways of Looking at a Blackbird</strong></li>     <p>Rearrange the poems in &quot;<a href="http://www.writing.upenn.edu/~afilreis/88/stevens-13ways.html">Thirteen Ways of Looking at a Blackbird</a>&quot; to create two new possible narrative arcs. Write a persuasive explanation of each arc. What aspects of the poem do not change regardless of the ordering of the stanzas? What aspects of the poem absolutely change with the ordering of the stanzas? Where do you think the more substantial meaning of the poem resides&mdash;in the aspects that change with reordering, or the aspects that do not?</p>     <li><strong>Rembrandt</strong></li>     <p>A striking characteristic of Rembrandt\'s self-portraits is that a given image often seems incomplete by itself, while also seeming to gesture toward shared features in the larger series. Using specific artworks as examples, describe at least three ways in which Rembrandt creates this effect. As an approach to self-portraiture, how does this differ from a painting like Velázquez\' <em>Las Meninas</em> or Vermeer\'s <em>The Artist in His Studio</em>?</p>     <li><strong>The Disasters of War</strong></li>     <p>Describe at least three &quot;sub-plots&quot; or subsidiary sequences Goya builds into the <em>Disasters of War</em>. For each sequence, provide a careful exegesis. How is Goya using sequence to frame and develop larger social or political arguments?</p> </ol> <h2 class="subhead">Essay 2</h2> <p>Assignment Description Unavailable</p> <h2 class="subhead">Final Essay Assignment</h2> <p><em>Due: Session 25</em></p> <p>Choose a media work of a kind that has not yet been discussed in class. Write a ten-page or twelve-page essay that compares this work with one of the major pieces of work we have studied in class from Tennyson onward.</p> <p>Instead of 50/50, please split the comparison 60/40 or 70/30&mdash;that is, spend 60 percent of the paper on one media work and 40 percent on the other. (This will help you to avoid making the structure of the paper too rote or schematic.) Your argument should include discussion of (1) the new work as a narrative, (2) the new work as a material artifact, and (3) at least two issues relating to the work\'s seriality that we have <strong>not</strong> yet seen in other works on the syllabus.</p> <p>Examples of kinds of works you <strong>might</strong> choose include:</p> <ol>     <li>A web miniseries</li>     <li>A radio serial (e.g. <em>The Adventures of Superman</em>)</li>     <li>A book series written by multiple authors under one name (e.g. <em>Nancy Drew</em> or <em>The Hardy Boys</em>)</li>     <li>A newspaper comic strip</li>     <li>A video game series (e.g. <em>Bioshock</em> or <em>Metal Gear Solid</em>)</li>     <li>A film franchise that aims to accommodate plot and character development over many years of new installments (e.g. <em>The Avengers</em> Franchise)</li> </ol>',
   'title': 'Assignments',
   'type': 'CourseSection',
   'uid': 'fdabab4b6ae08a9e7ed4b8ed361a8565',
   'url': '/courses/literature/21l-430-popular-culture-and-narrative-serial-storytelling-spring-2013/assignments'},
  {'description': None,
   'parent_uid': 'f42d61319f1b751daba0ce873de26e21',
   'short_url': 'download-course-materials',
   'text': '<div id="download_inner_main"><p class="ptext">This package contains the same content as the online version of the course, except for any audio/video materials and other interactive file types.</p><p class="ptext">For help downloading and using course materials, read our <a href="http://ocw.mit.edu/help/faq-technology/">frequently asked questions</a>.</p></div>',
   'title': 'Download Course Materials',
   'type': 'DownloadSection',
   'uid': '7871b682cf0e301c19e6a1c8d9572888',
   'url': '/courses/literature/21l-430-popular-culture-and-narrative-serial-storytelling-spring-2013/download-course-materials'}],
 'department_number': '21L',
 'description': 'Detail of a disc depicting a couple waltzing, created by Eadweard Muybridge for the phenakistoscope, an early motion picture machine that produced the illusion of movement through the rapid progression of a series of images. Image via wikimedia commons.',
 'extra_course_number': [{'linked_course_number_col': 'CMS.920',
   'linked_course_type_col': 'Graduate',
   'sort_as_col': ''}],
 'from_semester': 'Spring',
 'from_year': '2013',
 'image_description': '',
 'image_src': '',
 'instructors': [{'department': '',
   'directory_title': '',
   'first_name': 'Elyse',
   'last_name': 'Graham',
   'middle_initial': '',
   'mit_id': '',
   'suffix': '',
   'title': '',
   'uid': '5cb4e7c9e7ad887678c4a48aea2f3820'},
  {'department': 'Literature Section',
   'directory_title': 'Section Head/Professor',
   'first_name': 'James',
   'last_name': 'Buzard',
   'middle_initial': '',
   'mit_id': '900019054',
   'suffix': '',
   'title': 'Buzard, James',
   'uid': '7391b4544dd0b061944ab6a2e4be0c01'}],
 'language': 'en-US',
 'master_course_number': '430',
 'short_url': '21l-430-popular-culture-and-narrative-serial-storytelling-spring-2013',
 'sort_as': '21L.430',
 'tags': [{'name': 'serial'},
  {'name': 'storytelling'},
  {'name': 'narrative'},
  {'name': 'seriality'},
  {'name': 'Tennyson'},
  {'name': 'memoriam'},
  {'name': 'wire'},
  {'name': 'David Simon'},
  {'name': 'Rembrandt'},
  {'name': 'self-portraits'},
  {'name': 'blackbird'},
  {'name': 'Wallace Stevens'},
  {'name': 'Omar'},
  {'name': 'Auden'},
  {'name': 'Goya'},
  {'name': 'Disasters of War'},
  {'name': 'War'},
  {'name': 'Hogarth'},
  {'name': 'Superman'},
  {'name': 'Myth of Superman'},
  {'name': 'Myth of Sisyphus'},
  {'name': 'Myth of Fingerprints'},
  {'name': 'photography'},
  {'name': 'Muybridge'},
  {'name': 'Edweard'},
  {'name': 'Edweird'},
  {'name': 'Leland Stanford'},
  {'name': 'Camp Stanford'},
  {'name': 'Duchamp'},
  {'name': 'Nude Descending'},
  {'name': 'Nude Ascending'},
  {'name': 'Nude Suspended'},
  {'name': 'journalism'},
  {'name': 'animals eating people'},
  {'name': 'Orwell'},
  {'name': 'Art is Propaganda'},
  {'name': 'McLuhan'},
  {'name': 'Elegy'},
  {'name': 'Mourning'},
  {'name': "Morte D'Arthur"},
  {'name': 'Epic'},
  {'name': 'Dickens'},
  {'name': 'Pickwick'},
  {'name': 'Bleak House'},
  {'name': 'Sherlock'},
  {'name': 'Holmes'},
  {'name': 'Slaughterhouse'},
  {'name': 'Literature'},
  {'name': 'Murder'},
  {'name': 'English Murder'},
  {'name': 'Portlandia'},
  {'name': 'Battlestar Galactica'},
  {'name': 'Spoiler Alert'},
  {'name': 'Downton Abbey'},
  {'name': 'Downtown Abby'},
  {'name': 'social game'},
  {'name': 'anime'},
  {'name': 'chaplin'},
  {'name': 'gold rush'},
  {'name': 'comics'},
  {'name': 'remediation'},
  {'name': 'mediation'},
  {'name': 'dismediation'},
  {'name': 'transmediation'},
  {'name': 'procedural'}],
 'title': 'Popular Culture and Narrative: Serial Storytelling',
 'to_semester': '-1',
 'to_year': '-1',
 'uid': 'f42d61319f1b751daba0ce873de26e21',
 'url': '/courses/literature/21l-430-popular-culture-and-narrative-serial-storytelling-spring-2013'}
mbertrand commented 5 years ago
mbertrand commented 5 years ago

I will map department ids to names based on this page: http://web.mit.edu/facts/academic.html

pdpinch commented 5 years ago

As discussed in slack, we're going to drop the department and course numbers fields for now. They're not worth the extra effort.

mbertrand commented 5 years ago

@pdpinch, the topics field for Course objects currently just saves the topic name, but the raw_json will have details on the nested topics. Here's one example:

course.topics: ['Engineering', 'Business & Management']

course.raw_json.course_collections: 

[{'ocw_feature': 'Business',
   'ocw_feature_notes': '',
   'ocw_feature_url': '',
   'ocw_speciality': '',
   'ocw_subfeature': 'Operations Management'},
  {'ocw_feature': 'Engineering',
   'ocw_feature_notes': '',
   'ocw_feature_url': '',
   'ocw_speciality': '',
   'ocw_subfeature': 'Aerospace Engineering'},
  {'ocw_feature': 'Engineering',
   'ocw_feature_notes': '',
   'ocw_feature_url': '',
   'ocw_speciality': '',
   'ocw_subfeature': 'Systems Engineering'}]

For purposes of searching by these, what do you think of appending the subfeatures to the topics in the ES doc?

topics: ['Engineering', 'Business & Management', 'Operations Management', 'Aerospace Engineering', 'Systems Engineering']

mbertrand commented 5 years ago

Actually it would probably be better to have these in a separate subtopics field for when it comes to displaying topics in the search result details.

pdpinch commented 5 years ago

That OCW data structure seems overly-complicated. Let’s skip it until we have a better use case in mind.

@pdpinch https://github.com/pdpinch, the topics field for Course objects currently just saves the topic name, but the raw_json will have details on the nested topics. Here's one example: