sindresorhus / got

🌐 Human-friendly and powerful HTTP request library for Node.js
MIT License
14.27k stars 935 forks source link

RequestError: unexpected end of file #1826

Closed TKul6 closed 3 years ago

TKul6 commented 3 years ago

Describe the bug

Actual behavior

When I execute number of requests one after the other, after X request (the number is changes after each running) I get the following error from got: RequestError: unexpected end of file at PassThrough.eval (webpack://justeat/./node_modules/got/dist/source/core/index.js?:800:31) The status code is 200 and the status is OK. ...

Expected behavior

To parse the response successfully. ...

Code to reproduce

let options = {
    url: "<ANY_URL>",
    return_full_response: true,
    resolve_with_full_response: true,
    timeout: 120000,
    retries: 1,
    responseType: "text",
    throwHttpErrors: false,
    agent: {
      https: tunnel.httpsOverHttp({
        rejectUnauthorized: false,
        proxy: {
          host: "<PROXY_IP>",
          port: <PROXY_PORT>
        }
      })
    }
  };
  for (let attempt_number = 0; attempt_number < 100; attempt_number++) {
    console.log(`Fetching attempt number ${attempt_number}.`);
    const res = await got(options);
    console.log(res.statusCode);
  }

Checklist

szmarczak commented 3 years ago

Please post the full stack trace.

TKul6 commented 3 years ago

Thanks @szmarczak, here is the full stack:


    at PassThrough.eval (webpack://justeat/./node_modules/got/dist/source/core/index.js?:800:31)
    at Object.onceWrapper (events.js:482:26)
    at PassThrough.emit (events.js:387:35)
    at PassThrough.emit (domain.js:470:12)
    at emitErrorNT (internal/streams/destroy.js:106:8)
    at emitErrorCloseNT (internal/streams/destroy.js:74:3)
    at processTicksAndRejections (internal/process/task_queues.js:82:21)
    at Zlib.zlibOnError [as onerror] (zlib.js:188:17)```
szmarczak commented 3 years ago

Duplicate of #224 but this shouldn't happen.

szmarczak commented 3 years ago

Can you provide a fully working code to reproduce?

TKul6 commented 3 years ago

Sure, @szmarczak here you go:


const tunnel = require("tunnel");

async function work() {
  console.log("testing");
  let options = {
    url: <ANY_LINK>,
    return_full_response: true,
    resolve_with_full_response: true,
    timeout: 120000,
    retries: 1,
    responseType: "text",
    throwHttpErrors: false,
    agent: {
      https: tunnel.httpsOverHttp({
        rejectUnauthorized: false,
        proxy: {
          host: <PROXY_IP>,
          port: <PROXY_PORT>
        }
      })
    }
  };
  for (let attempt_number = 0; attempt_number < 100; attempt_number++) {
    console.log(`Fetching attempt number ${attempt_number}.`);
    const res = await got(options);
    console.log(res.statusCode);
  }
}

work();
szmarczak commented 3 years ago
I ran locally (with a proxy) and no bugs on main branch ```js import got from './dist/source/index.js'; import tunnel from 'tunnel'; async function work() { console.log("testing"); let options = { url: 'https://httpbin.org/anything', // return_full_response: true, // resolve_with_full_response: true, timeout: 120000, retry: {limit: 1}, responseType: "text", throwHttpErrors: false, agent: { https: tunnel.httpsOverHttp({ rejectUnauthorized: false, proxy: { host: '127.0.0.1', port: 8001, proxyAuth: 'username:password' } }) } }; for (let attempt_number = 0; attempt_number < 100; attempt_number++) { console.log(`Fetching attempt number ${attempt_number}.`); const res = await got(options); console.log(res.statusCode); } } work(); ``` ``` testing Fetching attempt number 0. 200 Fetching attempt number 1. 200 Fetching attempt number 2. 200 Fetching attempt number 3. 200 Fetching attempt number 4. 200 Fetching attempt number 5. 200 Fetching attempt number 6. 200 Fetching attempt number 7. 200 Fetching attempt number 8. 200 Fetching attempt number 9. 200 Fetching attempt number 10. 200 Fetching attempt number 11. 200 Fetching attempt number 12. 200 Fetching attempt number 13. 200 Fetching attempt number 14. 200 Fetching attempt number 15. 200 Fetching attempt number 16. 200 Fetching attempt number 17. 200 Fetching attempt number 18. 200 Fetching attempt number 19. 200 Fetching attempt number 20. 200 Fetching attempt number 21. 200 Fetching attempt number 22. 200 Fetching attempt number 23. 200 Fetching attempt number 24. 200 Fetching attempt number 25. 200 Fetching attempt number 26. 200 Fetching attempt number 27. 200 Fetching attempt number 28. 200 Fetching attempt number 29. 200 Fetching attempt number 30. 200 Fetching attempt number 31. 200 Fetching attempt number 32. 200 Fetching attempt number 33. 200 Fetching attempt number 34. 200 Fetching attempt number 35. 200 Fetching attempt number 36. 200 Fetching attempt number 37. 200 Fetching attempt number 38. 200 Fetching attempt number 39. 200 Fetching attempt number 40. 200 Fetching attempt number 41. 200 Fetching attempt number 42. 200 Fetching attempt number 43. 200 Fetching attempt number 44. 200 Fetching attempt number 45. 200 Fetching attempt number 46. 200 Fetching attempt number 47. 200 Fetching attempt number 48. 200 Fetching attempt number 49. 200 Fetching attempt number 50. 200 Fetching attempt number 51. 200 Fetching attempt number 52. 200 Fetching attempt number 53. 200 Fetching attempt number 54. 200 Fetching attempt number 55. 200 Fetching attempt number 56. 200 Fetching attempt number 57. 200 Fetching attempt number 58. 200 Fetching attempt number 59. 200 Fetching attempt number 60. 200 Fetching attempt number 61. 200 Fetching attempt number 62. 200 Fetching attempt number 63. 200 Fetching attempt number 64. 200 Fetching attempt number 65. 200 Fetching attempt number 66. 200 Fetching attempt number 67. 200 Fetching attempt number 68. 200 Fetching attempt number 69. 200 Fetching attempt number 70. 200 Fetching attempt number 71. 200 Fetching attempt number 72. 200 Fetching attempt number 73. 200 Fetching attempt number 74. 200 Fetching attempt number 75. 200 Fetching attempt number 76. 200 Fetching attempt number 77. 200 Fetching attempt number 78. 200 Fetching attempt number 79. 200 Fetching attempt number 80. 200 Fetching attempt number 81. 200 Fetching attempt number 82. 200 Fetching attempt number 83. 200 Fetching attempt number 84. 200 Fetching attempt number 85. 200 Fetching attempt number 86. 200 Fetching attempt number 87. 200 Fetching attempt number 88. 200 Fetching attempt number 89. 200 Fetching attempt number 90. 200 Fetching attempt number 91. 200 Fetching attempt number 92. 200 Fetching attempt number 93. 200 Fetching attempt number 94. 200 Fetching attempt number 95. 200 Fetching attempt number 96. 200 Fetching attempt number 97. 200 Fetching attempt number 98. 200 Fetching attempt number 99. 200 ```
szmarczak commented 3 years ago

I tested again with got@11.8.2 and no issues as well.