mitodl / odl-video-service

building blocks for a basic video service for ODL
BSD 3-Clause "New" or "Revised" License
4 stars 1 forks source link

Can't upload more than 5 files from dropbox #1027

Closed pdpinch closed 1 year ago

pdpinch commented 1 year ago

Steps to Reproduce

  1. Create a new collection
  2. Upload the 32 videos from https://www.dropbox.com/sh/27n8kzccjxv7dm1/AADn0oY1V2gjjZXkbuP8gu_Sa?dl=0

Expected Behavior

Actual Behavior

pdpinch commented 1 year ago

The number of videos that get queued seems to be variable. On a subsequent attempt, I uploaded 16 and 7 were queued.

pdpinch commented 1 year ago

No apparent errors in Sentry, but when I watch in my browser, I see a 504 response from https://video.odl.mit.edu/api/v0/upload_videos/

mbertrand commented 1 year ago

I think the celery task to stream videos to S3 and start transcoding jobs is not running asynchronously:

        # Kick off chained async celery tasks to transfer file to S3, then start a transcode job
        task_result = chain(
            tasks.stream_to_s3.s(video.id), tasks.transcode_from_s3.si(video.id)
        )()

The () at the end should probably be replaced with .apply_async()

pdpinch commented 1 year ago

@collinpreston @mbertrand do you think we can close this?

mbertrand commented 1 year ago

I think so, it seems to be working well now

collinpreston commented 1 year ago

Passed testing again today. Closing.