support for streaming a json response in the urllib.request.urlopen flow

[yyellin]

The read() operation of the Response object returned from urllib.request.urlopen returns a byte rather than the expected str, hence I check for return type and decode if necessary.

[daggaz]

Hi, what's the exact scenario here?

[yyellin]

I have a server route which streams a json object as its response, and I would like my client code to consume the json as it becomes available.

Here's an example server:

import datetime
from time import sleep

from flask import Flask, Response

app = Flask(__name__)

@app.route("/test", methods = ['GET'])
def test():

  def streamer():
      yield '['
      until  = datetime.datetime.now() + datetime.timedelta(seconds=20)
      while datetime.datetime.now() < until:
          yield '{"foo":"bar"}, '
          sleep(1)

      yield '{"foo":"bar"}]'

  return Response(streamer(), mimetype='application/json')

if __name__ == '__main__':
    app.run()

And this would be my client (based on the example you provide):

import urllib.request
import json_stream.requests

def visitor(item, path):
    print(f"{item} at path {path}")

with urllib.request.urlopen('http://127.0.0.1:5000/test') as response:
    data = json_stream.visit(response, visitor)

Without my change (or something similar) the client will fail to process the json stream

[daggaz]

Right, not sure how I've not spotted this before!

I've created an issue #10 to track this, and a PR #11 for a complete fix.

[daggaz]

@yyellin this is now available on pypi as version 1.2.0

Thanks for your help!

[yyellin]

Installed 1.2.0 - working brilliantly, thanks!