StreamHttpResponse with sync operations break in Daphne

[themagicalmammal]

I am basically using this

def process(
    self,
    request: Request,
    pk: str,
    <parameters>
):
    video = Video.objects.get(id=pk)
    return StreamingHttpResponse(process_video(<parameters>))

As you can tell process_video is a sync function but the generator breaks for this and it only yields the final result. This is working fine with --no-asgi flag.

To give a example of how it looks like (a much smaller version of the code). But it processed video and send the frame to the frontend.

def process_video(
        <parameters>
):
    # Open the video file
    cap = cv2.VideoCapture(video_path)

    while cap.isOpened():
                  success, frame = cap.read()
                  print (frame)
                  yeild (frame)
    yield ("Complete")

It is basically a sync generator, but it fails when daphne is introducted. It only yeilds "Complete" once complete. The print function is printing the frame perfectly fine. So I know it runs. I also tried to convert it to async but async generator is a error.

[themagicalmammal]

Issue References

1. 413

I initially has a async_to_sync bug which got fixed by updated django to the latest lts version (4.2.5). But the generator still seems to be a issue. I have faced caroutine bug while trying to tackle this as well.

[carltongibson]

Yes, you need to use Django 4.2's support for async streaming responses (using an async generator).

[themagicalmammal]

I have converted to code to async generator and now it works fine. But I think we should have some way to convert a sync generator to async generator or add support for sync generator in StreamHTTPResponse.

[themagicalmammal]

I am closing this issue for now but I think we should look into one of those things I mentioned above.

[carltongibson]

@themagicalmammal Glad you've resolved your issue.

I'm afraid there's no magic way to convert sync iterators to async iterators. As per the Django docs on this topic, you need to adapt the generator passed to your response depending on whether you're running under WSGI or ASGI. (Alas the abstraction breaks down at this point.)