mitodl / ocw-studio

Open Source Courseware authoring tool
BSD 3-Clause "New" or "Revised" License
9 stars 3 forks source link

Google Drive Backfill Misses Some Files in S3 #2088

Closed pt2302 closed 5 months ago

pt2302 commented 5 months ago

Expected Behavior

Running ./manage.py backfill_gdrive_folder --filter <course name> for a legacy course should create Google Drive files for the all the non-video resources in the course.

Current Behavior

The backfill fails, with the error message botocore.exceptions.ClientError: An error occurred (404) when calling the HeadObject operation: Not Found.

Steps to Reproduce

Locally, this can be tested by setting the ENVIRONMENT variable to anything but dev, entering appropriate credentials for AWS (instead of MinIO), and then running the backfill_gdrive_folder management command for a legacy course.

Possible Solution

This is caused by a difference in the way that MinIO and AWS S3 handle paths. A related prior discrepancy for caption file paths was resolved here: https://github.com/mitodl/ocw-studio/pull/1797.