Error connecting with ElasticSearch while importing products. (help)

[pmaojo]

I have Vue Storefront API running, Elasticsearch responds to this command: curl -i http://elastic:changeme@localhost:8080/api/catalog/vue_storefront_catalog/_search

And this is the config file:

# Spree config
SPREE_URL=https://mysiteURL.com
SPREE_PATH=api/v2/storefront/
SPREE_IMAGES_HOST=https://mysiteURL.com

# Server config
# 8889 port is same as in docker-compose.yml
SERVER_PORT=8889

# Importer config
PER_PAGE=100
MAX_PAGES=1000

# Elastic Search config
ES_URL=es1:9200
ES_INDEX=vue_storefront_catalog
ES_LOG_LEVEL=error
ES_REQUEST_TIMEOUT=3000
ES_BULK_SIZE=200

# Scheduler config
CRON_SCHEDULE="*/1 * * * *"

This is the log with the error:

./docker-bin/spree2vs.sh yarn import:all
Recreating spark-solutions_spree2vs ... done
yarn run v1.13.0
$ yarn run import:products && yarn run import:categories
$ ./dist/index.js products
{
  "message": "Importing products and removing unused",
  "level": "info",
  "timestamp": "2019-06-11T13:31:51.963Z"
}
{
  "message": "No date provided. Updating all products and setting cursor to 1560259911966.",
  "level": "info",
  "timestamp": "2019-06-11T13:31:51.966Z"
}
{
  "message": "Downloaded page 1 containing 11 resources, processing",
  "level": "info",
  "timestamp": "2019-06-11T13:31:52.826Z"
}
{
  "message": "Pagination finished, total resources <= 100",
  "level": "info",
  "timestamp": "2019-06-11T13:31:52.826Z"
}
{
  "message": "Categories fetched. Importing products.",
  "level": "info",
  "timestamp": "2019-06-11T13:31:52.827Z"
}
{
  "message": "Downloaded page 1 containing 16 resources, processing",
  "level": "info",
  "timestamp": "2019-06-11T13:31:54.343Z"
}
{
  "message": "Pagination finished, total resources <= 100",
  "level": "info",
  "timestamp": "2019-06-11T13:31:54.349Z"
}
{
  "message": "Products' cursor updates requested: 0. Products' content updates requested: 16.",
  "level": "info",
  "timestamp": "2019-06-11T13:31:54.349Z"
}

Elasticsearch ERROR: 2019-06-11T13:31:54Z
  Error: Request error, retrying
  POST http://es1:9200/_bulk?refresh=wait_for => getaddrinfo ENOTFOUND es1 es1:9200
      at Log.error (/app/node_modules/elasticsearch/src/lib/log.js:226:56)
      at checkRespForFailure (/app/node_modules/elasticsearch/src/lib/transport.js:259:18)
      at HttpConnector.<anonymous> (/app/node_modules/elasticsearch/src/lib/connectors/http.js:164:7)
      at ClientRequest.wrapper (/app/node_modules/lodash/lodash.js:4935:19)
      at ClientRequest.emit (events.js:189:13)
      at Socket.socketErrorListener (_http_client.js:392:9)
      at Socket.emit (events.js:189:13)
      at emitErrorNT (internal/streams/destroy.js:82:8)
      at emitErrorAndCloseNT (internal/streams/destroy.js:50:3)
      at process._tickCallback (internal/process/next_tick.js:63:19)

{
  "message": [
    "Could not fully process products.",
    {
      "message": "No Living connections",
      "name": "Error",
      "stack": "Error: No Living connections\n    at sendReqWithConnection (/app/node_modules/elasticsearch/src/lib/transport.js:226:15)\n    at next (/app/node_modules/elasticsearch/src/lib/connection_pool.js:214:7)\n    at process._tickCallback (internal/process/next_tick.js:61:11)"
    }
  ],
  "level": "error",
  "timestamp": "2019-06-11T13:31:54.607Z"
}
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

[tniezg]

Hi @pmaojo, It looks like spree2vs can't connect to Elastic Search. Try changing es1:9200 to the appropriate address of the ES server. spree2vs connects directly to ES and skips connecting to vs-api when importing the catalogue.

[pmaojo]

hi @tniezg I can't guess which is the appropriate address, I have a fresh Vue Storefront API docker image running..

[tniezg]

If you're running vs-api inside Docker Compose and spree2vs inside Docker Compose, you'll have to put them in the same Docker network for them to see each other. spree2vs already joins an auxiliary network called spree_vue_storefront_shared_development_network. Try modifying vs-api's docker-compose.yml file to also join this network. You can use https://github.com/spark-solutions/spree2vuestorefront/blob/master/docker-compose.yml as reference. version: '3.5' of Docker Compose is required. After doing this, spree2vs should see Elastic Search at address es1:9200.

[pmaojo]

Thanks! I will try!

[pmaojo]

That was it. It almost works. This is how docker-compose.yml of vue storefront api looks like

version: '3.5'
services:
  es1:
    container_name: elasticsearch
    build: docker/elasticsearch/
    volumes:
      - ./docker/elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml:ro
      - ./docker/elasticsearch/data:/usr/share/elasticsearch/data
    ports:
      - '9200:9200'
      - '9300:9300'
    networks:
      - net
    environment:
      ES_JAVA_OPTS: "-Xmx512m -Xms512m"
  kibana:
    build: docker/kibana/
    volumes:
      - ./docker/kibana/config/:/usr/share/kibana/config:ro
    ports:
      - '5601:5601'
    depends_on:
      - es1

  redis:
    image: 'redis:4-alpine'
    ports:
      - '6379:6379'
volumes:
  esdat1:
networks:
  net:
    name: spree_vue_storefront_shared_development_network

[pmaojo]

To make it connect, I had to change also .env.docker

# Spree config
SPREE_URL=https://xxxxx.com
SPREE_PATH=api/v2/storefront/
SPREE_IMAGES_HOST=https://xxxxxx.com

# Server config
# 8889 port is same as in docker-compose.yml
SERVER_PORT=8889

# Importer config
PER_PAGE=100
MAX_PAGES=1000

# Elastic Search config
ES_URL=http://elasticsearch:9200
ES_INDEX=vue_storefront_catalog
ES_LOG_LEVEL=error
ES_REQUEST_TIMEOUT=3000
ES_BULK_SIZE=200

# Scheduler config
CRON_SCHEDULE="*/1 * * * *"