Reconnection is not supported

[winnochan]

When connection with elasticsearch server is broken, it won't send any msg to server any more. Can you add this feature?

[mcollina]

@delvedor maybe you can help?

Otherwise it would be good for you to send a PR.

[alexandrevanhoutte]

Is there any update about this feature? Is there a temporary solution to resolve it ?

[mcollina]

@alexandrevanhoutte Would you like to send a Pull Request to address this issue? Remember to add unit tests.

[penumbra23]

We're experiencing the same issue, once the ES instance crashes or any network congestion hits the roof, the connection gets broken. While inspecting the ClientOptions inside @elastic/elasticsearch-js, there's the following field:

resurrectStrategy?: 'ping' | 'optimistic' | 'none'

Maybe setting this one to ping should be sufficient for the problem?

[mcollina]

Maybe? Would you like to send a PR?

[penumbra23]

I can give it a try 😄

[delvedor]

@penumbra23 it won't solve this specific issue out if the box, the the v8 client uses a new type of connection pool.

Try to use ClusterConnectionPool in the client configuration, the ping is the default behavior.

It would be useful to understand why the client can't reconnect. The connection pool will retry connection either way after a while. It might be something related to the underlying http library itself.

[Granjow]

I have a service which, on startup, sometimes logs the following from a pino-elasticsearch error event:

connect ENETUNREACH 10.10.10.100:9200 - Local (0.0.0.0:0)

It will then never try to reconnect again until I restart the service. Before that, I see that elasticsearch retries sending messages several times when setting the environment variable DEBUG=elasticsearch.

  elasticsearch Retrying request, there are still 0 attempts {
  path: '/_bulk',
  method: 'POST',
  querystring: {},
  bulkBody: [
    // (data here)
  ]
} +6ms
  elasticsearch Starting a new request {
  method: 'POST',
  path: '/_bulk',
  body: 'DATA',
  querystring: '',
  headers: {
    'user-agent': 'elastic-transport-js/8.2.0 (linux 5.10.103-v7l+-arm; Node.js v18.13.0)',
    'x-elastic-client-meta': 'es=8.5.0,js=18.13.0,t=8.2.0,hc=18.13.0,h=bp',
    'content-type': 'application/vnd.elasticsearch+x-ndjson; compatible-with=8',
    accept: 'application/vnd.elasticsearch+json; compatible-with=8',
    'content-length': '1175'
  }
} +6ms

After the pino-elasticsearch error, nothing from elasticsearch is logged anymore.

Does this help fixing the issue?

[padenaa]

Is there any plan for reconnection to be implemented as a feature? I'm doing some development that could benefit from it

[mcollina]

My hope is that somebody will eventually send a PR for this. I currently do not have time.

[sheldhur]

The main problem in elasticsearch https://github.com/elastic/elastic-transport-js/issues/53

I can try fix it and make PR but before I want to be sure that it isn't in vain, because my implementation will do the same elasticsearch is trying to do, but only after the client emits error.

https://github.com/elastic/elastic-transport-js/blob/0c9b7c56089c68efaa5a5bc93d85220cbeb06d56/src/pool/ClusterConnectionPool.ts#L160-L192

It'll help me if somebody has a ready solution and you can publish it in this thread.

[sheldhur]

It was a little easier)

[mcollina]

trhx

[Suniron]

How can I set a resurrect strategy? Is there a way?

[sheldhur]

@mcollina @delvedor can you reopen this issue? Sometimes I keep seeing this problem in my production. I have a single node ES and ~50% services can't restore connection after restarting node.

I think my PR closed #140 issue, but not this.

[paulish]

Yes, have the same problem. If at the start of process the elasticsearch node is not available then you have no logs before process restart.

[mcollina]

Would you like to send a PR?

[sheldhur]

@mcollina that client problem https://github.com/elastic/elasticsearch-js/issues/1714 https://github.com/elastic/elastic-transport-js/issues/53

I can share my dirty hack like as a temporary solution which just ignore all connection errors

import { UndiciConnection } from '@elastic/transport';

export class IgnoreErrorConnection extends UndiciConnection {
  constructor(opts) {
    super(opts);
    this.requestAfter = null;
    this.requestAfterInterval = 5 * 60000;
  }

  async request(params, options) {
    const now = new Date();
    if (this.requestAfter && this.requestAfter > now) {
      return this.fakeResponse(params);
    }

    try {
      const response = await super.request(params, options);
      this.requestAfter = null;

      return response;
    } catch (err) {
      switch (err.name) {
        case 'TimeoutError':
        case 'ConnectionError': {
          this.requestAfter = new Date(now.valueOf() + this.requestAfterInterval);
          return this.fakeResponse(params);
        }
        default: {
          throw err;
        }
      }
    }
  }

  fakeResponse(params) {
    const docs = params.body.split('\n').length;
    const result = {
      errors: false,
      took: docs,
      items: Array(docs).fill({ result: 'created', status: 201 })
    };
    const body = JSON.stringify(result);

    return {
      statusCode: 200,
      headers: {
        'content-type': 'application/vnd.elasticsearch+json;compatible-with=8',
        'content-length': body.length,
        'x-elastic-product': 'Elasticsearch'
      },
      body
    };
  }
}

const elastic = pinoElastic({
  ...
  Connection: IgnoreErrorConnection,
  ...
});