mozilla / bugbug

Platform for Machine Learning projects on Software Engineering
Mozilla Public License 2.0
503 stars 312 forks source link

Compress HTTP service responses #1532

Closed marco-c closed 4 years ago

marco-c commented 4 years ago

The JSON responses could be gzipped or brotlified.

marco-c commented 4 years ago

I tested on a random response. The original size was 200 KB, 20 KB when compressed with gzip, 16,4 KB when compressed with brotli. I'd go with gzip.

rajeshkumargp commented 4 years ago

I have read the source https://github.com/mozilla/bugbug/blob/master/http_service/bugbug_http/app.py.

The requirement is to compress data in 1) to make prediction for a bug - https://github.com/mozilla/bugbug/blob/master/http_service/bugbug_http/app.py#L351 2) to make prediction for batch of bugs - https://github.com/mozilla/bugbug/blob/master/http_service/bugbug_http/app.py#L525

as gzip compressed files.

Please clarify on my understanding.

Related References:

  1. https://github.com/closeio/Flask-gzip/blob/master/flask_gzip.py#L25-L31
marco-c commented 4 years ago

Also https://github.com/mozilla/bugbug/blob/e0bc6777e833628d143fb1f63a2322988974f1b6/http_service/bugbug_http/app.py#L579.

Note we want to compress in a way that something like:

response = requests.get("http://localhost:80/push/try/46bef19ad47bbdb7ef5193b1ad63ddd9f1c1adba/schedules", headers={"X-Api-Key": "test"})
response.json()

keeps working as before.