GZIP encoding support

[ghost]

Hi,

first of all thanks for your work on keeping VBA Web up-to-date, you are doing a great job. I'd like to follow up on an old item (#61) on gzip encoding. I have integrated the decoding for large JSON files in my application as I needed it, and thought I share what I did and give some input for further improving VBA-Web

First of all I found a module that can decode GZIP on Stack Overflow, that does the real work (see here: https://stackoverflow.com/questions/58026702/how-to-decompress-http-responses-in-vba-excel). The function "inflate" takes a GZIP byte array and transforms it unicode. Unfortunately it can only decode gzip, but not encode.

To integrate it with VBA Web, this is how I created the request:

    Dim Request As New WebRequest
    Dim Response As New WebResponse

    Request.Resource = "/function"
    Request.AddHeader "Accept-Encoding", "gzip"

    Request.RequestFormat = Custom
    Request.CustomResponseFormat = "gzip"

    WebHelpers.RegisterConverter "gzip", "application/json", "GzipConverter.ConvertToGzip", "GzipConverter.ParseGzip", , "Binary"

    Set Response = Client.Execute(Request)

I've then created a function "ParseGzip" in one the module GzipConverter that will receive the binary (that's important!) body from the requst and decode it.

Public Function ParseGzip(Bytes As Variant) As Object

    Dim byteArray() As Byte
    Dim strResponse As String
    byteArray = Bytes ' convert variant array to byte array

    Call Inflate(byteArray) 'decode the GZIP byte array
    strResponse = StrConv(byteArray, vbUnicode) 'create a string from the encoded byte array

    Set ParseGzip= ParseJson(strResponse) 'parse string to json

End Function

This functions returns the parsed JSON to the VBA web for further processing. This works, but it is a all bit hacky. Although proposed as a solution in #61, the RegisterConverter function is not really optimal here, because gzip is not an content type (in my case I am working with JSON). If the server does not return gzip (which is visibile in the encoding header of the response), it would still try to parse it as gzip and would through an error.

I think it would make sense to allow an additional hook for an encoding function, which is triggered by content-encoding: gzip in the header. Would love to see a more proper intergration into VBA Web...

[zgrose]

Create a pull-request with your code added to the class(es)?

[ghost]

It is to hacky (focused on JSON only, can‘t handle non decoded response) to add it to the project and I am not experienced enough to do a proper integration. Would do if I could.

Am 23.04.2020 um 19:05 schrieb Zoltan Grose notifications@github.com:

 Create a pull-request with your code added to the class(es)?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[ghost]

Well, I found a better solution that integrates it properly and created a pull request #433 (first time done something like this, hope everything worked).