mitodl / ocw-to-hugo

A command line utility for taking master.json output from ocw-data-parser and producing markdown for use with hugo-course-publisher
3 stars 0 forks source link

21W.758 S13 doesn't exist in studio production #470

Closed pdpinch closed 2 years ago

pdpinch commented 2 years ago

For some reason, this course doesn't seem to be have imported in production

s3://open-learning-course-data-production/21w-758-genre-fiction-workshop-spring-2013/21w-758-genre-fiction-workshop-spring-2013_parsed.json

It does exist on ocw-studio-rc:

https://ocw-studio-rc.odl.mit.edu/sites/21w-758-genre-fiction-workshop-spring-2013/

gumaerc commented 2 years ago

From 21w-758-genre-fiction-workshop-spring-2013_parsed.json:

"last_unpublishing_date": "2015/04/16 07:00:44.510 GMT-4",

Likely this course still exists on RC because in the Plone OCW system the course data isn't frequently backpopulated into QA. So, the course was likely unpublished in production and the same thing never was done in QA. ocw-to-hugo simply skips unpublished courses so they don't end up in the output bucket and are either not imported in the first place or removed from studio if they are detected as having been removed from the bucket.

pdpinch commented 2 years ago

Amazing. There are two versions of this course in Plone. One was unpublished, but the other is still active. It's probably random which one we got in the Plone export.

I'll try to update the raw JSON on S3 with the correct version, by running the export_content script. It will take until tomorrow to arrive.

pdpinch commented 2 years ago

Ok, both versions are in the Plone export, but they have the same slug. I'm not sure how we determine which one we get.

As noted in mitodl/open-discussions#3533, we should have imported the courses in the department CMS-W and not the courses in in 21W or CMS.

In this particular case, that means

this one: s3://ocw-content-storage/PROD/CMS-W/21W.758/Spring_2013/21w-758-genre-fiction-workshop-spring-2013

instead of: s3://ocw-content-storage/PROD/21W/21W.758/Spring_2013/21w-758-genre-fiction-workshop-spring-2013

mbertrand commented 2 years ago

I think we can import the correct one by running this task from a django shell on open-discussions production:

get_ocw_courses(
    course_prefixes=["PROD/CMS-W/21W.758/Spring_2013/21w-758-genre-fiction-workshop-spring-2013"],
)

I can try it on RC first by deleting the current 21w-758-genre-fiction-workshop-spring-2013 course and running the above. If successful, we'd then have to run import_ocw_course_content for studio afterward - the ban on importing sites doesn't apply to sites that were never imported, right?

mbertrand commented 2 years ago

Found a few more?

https://ocw.mit.edu/courses/comparative-media-studies-writing/21w-730-1-expository-writing-social-and-ethical-issues-in-print-photography-and-film-fall-2005/ https://ocw.mit.edu/courses/comparative-media-studies-writing/21w-730-3-expository-writing-autobiography-theory-and-practice-spring-2001/ https://ocw.mit.edu/courses/comparative-media-studies-writing/cms-360-introduction-to-civic-media-fall-2012/ https://ocw.mit.edu/courses/comparative-media-studies-writing/cms-407-media-and-methods-sound-fall-2012/

Found these by doing a query on open-discussions for LearningResourceRuns which had duplicate slugs, of which at least one was published. They are not on ocw-studio production.

pdpinch commented 2 years ago

Please go for it. I don’t think there’s any version of this course on production studio.

mbertrand commented 2 years ago

All the above sites are published now on production, I'll close this issue if they look good to you.

pdpinch commented 2 years ago

👍