nodejs / readable-stream

Node-core streams for userland
https://nodejs.org/api/stream.html
Other
1.03k stars 227 forks source link

AsyncIterator can't support Node v8 with some polyfill? #400

Closed ogawa0071 closed 5 years ago

ogawa0071 commented 5 years ago

Can I run this sample code in Node v8 (Google Cloud Functions) with some polyfill? I tried @bable/polyfill and core-js, but occurred error Symbol.asyncIterator is not defined..

Sample code:

import * as fs from 'fs'
import axios from 'axios'
import * as mime from 'mime'

const download = async (url: string) => {
  const response = await axios({
    method: 'get',
    url,
    responseType: 'stream'
  })

  try {
    const file = `file.${mime.getExtension(response.headers['content-type'])}`
    const ws = fs.createWriteStream(file)

    for await (const data of response.data) {
      ws.write(data)
    }
  } catch (err) {
    console.error(err)
  }
}

;(async () => {
  await download('https://avatars1.githubusercontent.com/u/3258736?v=4')
})()
mcollina commented 5 years ago

Can you please check if the following works:

import { Readable } from 'readable-stream'

const r = new Readable({
  read() {
    this.push('hello')
    this.push(null)
  }
})

(async () => {
   for await (const chunk of r) {
    console.log(chunk)
  }
})()

Also, what are you babel settings? We are currently testing against https://github.com/nodejs/readable-stream/blob/master/.babelrc.

Note, streams using readable-stream@2 would not support this. if that's the case, you should just:

stream[Symbol.asyncIterator] = Readable.prototype[Symbol.asyncIterator]

Let me know if it works.

@MylesBorins @fhinkel maybe you can take a look, it would be an awesome guide for Google Cloud Functions.

ogawa0071 commented 5 years ago

Thank you for your reply!

I tried your code, and it doesn't work it. I write code in TypeScript almost always, and TypeScript's tsc command also functions as a transpiler.

My tsconfig.json ( It is Google Cloud Functions for Firebase's default excluding lib option. I added esnext.asynciterable. ) is this:

{
  "compilerOptions": {
    "lib": [
      "dom",
      "es6",
      "dom.iterable",
      "scripthost",
      "esnext.asynciterable"
    ],
    "module": "commonjs",
    "noImplicitReturns": true,
    "noUnusedLocals": true,
    "outDir": "lib",
    "sourceMap": true,
    "strict": true,
    "target": "es2015"
  },
  "compileOnSave": true,
  "include": [
    "src"
  ]
}

And tried .babelrc too.

{
  "presets": [
    [
      "@babel/preset-env",
      {
        "targets": {
          "node": "8"
        }
      }
    ]
  ]
}

And I tried with core-js. It works!

import 'core-js/modules/es7.symbol.async-iterator'
// or
// import '@babel/polyfill'

// import { Readable } from 'stream'
import { Readable } from 'readable-stream'

const r = new Readable({
  read() {
    this.push('hello')
    this.push(null)
  }
})

;(async () => {
  for await (const chunk of r) {
    console.log(chunk)
  }
})()

So, I found out the answer!

~ readable-stream: enable readable-stream: disable
core-js: enable 👍 👎
core-js: disable 👎 👎

"readable-stream: disable" mean that use Node.js standard stream library in Node.js v9 or less like this: import { Readable } from 'stream'

ogawa0071 commented 5 years ago

I will return to the first question. Now, I found out readable-stream library and core-js polyfill include, AsyncIterator works in Node.js v8.

But, I want to use axios. It is npm package. How to replace Node.js standard stream library to readable-stream used in the package?

Can't without fork axios package?

mcollina commented 5 years ago

As I said, you would have to do:

stream[Symbol.asyncIterator] = Readable.prototype[Symbol.asyncIterator]

for await (let chunk of stream) {
  // ...
}

Let me know if that works.

ogawa0071 commented 5 years ago

I understand!!!

This code work in Node.js v8! Thank you!!!

import 'core-js/modules/es7.symbol.async-iterator'
// or
// import '@babel/polyfill'

import { Readable } from 'readable-stream'
import * as fs from 'fs'
import axios from 'axios'
import * as mime from 'mime'

const download = async (url: string) => {
  const response = await axios({
    method: 'get',
    url,
    responseType: 'stream'
  })

  response.data[Symbol.asyncIterator] = Readable.prototype[Symbol.asyncIterator]

  try {
    const file = `file.${mime.getExtension(response.headers['content-type'])}`
    const ws = fs.createWriteStream(file)

    for await (const data of response.data) {
      ws.write(data)
    }
  } catch (err) {
    console.error(err)
  }
}

;(async () => {
  await download('https://avatars1.githubusercontent.com/u/3258736?v=4')
})()

* Need compile with TypeScript or Babel.