Rewrites proxy returning cut off response to browser

[JackParn33]

Discord thread

When implementing NextJS rewrites to an external URL we found the response being returned to the browser was being cut off causing the page to display incorrectly.

This was fixed for us by adding a custom converter to remove the Accept-Encoding header from the proxy request (code below), however I think there needs to be a wider fix as I would expect others to encounter this hard to diagnose bug.

Also found that using the Streaming wrapper resolves the issue.

@conico974 has pointed out it is likely awaiting the wrong thing when the proxy is both using streaming and accept-encoding (especially when it's a full body like here) https://github.com/opennextjs/opennextjs-aws/blob/0536cc68732cc468c091916a7a4e8b0e9b7bd305/packages/open-next/src/core/routing/util.ts#L254. It probably should be res.on instead of _res, we need to wait for the end of the decompression here.

Custom converter fix

import converter from '@opennextjs/aws/converters/aws-apigw-v2.js';
import type { Converter } from '@opennextjs/aws/types/open-next';

const customConverter: Converter = {
  convertFrom: async event => {
    const result = await converter.convertFrom(event);

    if (result.headers) {
      const { 'Accept-Encoding': _, ...newHeaders } = result.headers;
      return { ...result, headers: newHeaders };
    }

    return result;
  },

  convertTo: converter.convertTo,

  name: 'custom-apigw-v2-no-accepts-encoding',
};

export default customConverter;

[conico974]

Fixed by #555