search

yumauri / gotenberg-js-client

A simple JS/TS client for interacting with a Gotenberg API
MIT License
110 stars 9 forks source link

EPIPE Error when generating 10s pdf in parallel #29

Closed MollardMichael closed 3 years ago

MollardMichael commented 3 years ago

I'm trying this library to call Gotenberg and it works great.

I'm generating around 300 pdf and this trying to call Gotenberg in batch of 10s.

When I do this, the calls start to fail with {"errno":-32,"code":"EPIPE","syscall":"write"}.

Do you have any idea why it might happen ?

yumauri commented 3 years ago

Wow

According to Gotenberg's documentation https://thecodingmachine.github.io/gotenberg/#scalability

For the HTML, URL and Markdown endpoints, the API does only 6 conversions in parallel. Indeed, Google Chrome misbehaves if there are too many concurrent conversions. The more concurrent requests, the more 504 HTTP codes the API will return.

I guess issue is related

yumauri commented 3 years ago

Can you show, how do you batch requests? And can you try to decrease batch size to 5, for example?

MollardMichael commented 3 years ago

I'm using fp-ts sequences to run then in parallel pipe.

This issue only happen on my kubernetes network. I'm looking more into it. I'll tell you if it comes from the lib or some configuration of axios If I find something.

By the way thanks for the rate limiting. I hadn't seen it

yumauri commented 3 years ago

I think you got me wrong :)

I'm not taking any part in Gotenberg's development, this library is a client, which I wrote originally for myself, because there are only two official clients — for Go and PHP.

There is no any rate limiting in this client (maybe I should add it though), this quote is from Gotenberg's documentation.

And I didn't quite understand your mentioning axios... Do you use your own HTTP client, based on axios, with this library? This is possible, but not documented, so I would be quite surprised :) Internally it uses regular Node.js http.request and https.request directly.

MollardMichael commented 3 years ago

I understood correctly the first time. Indeed I made a mistake. I went through your code and I know how to do it now but i'll keep the regular Node.js client.

After some investigation, It turns out that it was a bug in Istio (A service Mesh for Kubernetes).

I'll close this issue. And thank you for the lib. It works great !