elastic / eland

Python Client and Toolkit for DataFrames, Big Data, Machine Learning and ETL in Elasticsearch
https://eland.readthedocs.io
Apache License 2.0
18 stars 98 forks source link

eland_import_hub_model does not work with Elastic cloud's elasticsearch end point #500

Closed johnt-softclouds closed 1 year ago

johnt-softclouds commented 2 years ago

I found the Elasticsearch end point using the admin console. I was able to log into the end point with a Firefox browser using the elastic username and secret password. I then used eland to import a pre-trained NLP model. I used the command below (the private data replaced with X for security reasons):

eland_import_hub_model -u elastic -p XXXXXXXXXX --url https://xxxx.es.us-west-2.aws.found.io --hub-model-id sentence-transformers/msmarco-MiniLM-L-12-v3 --task-type text_embedding --start

The eland command above displayed the following error message:

ValueError: URL must include a 'scheme', 'host', and 'port' component (ie 'https://localhost:9200')

Why doesn't the eland command work with the cloud Elasticsearch end point? My Elastic.co cloud end point for Elasticsearch does not have a port number. Is there a parameter that I can use to disable the port verification?

es-endpoint

johnt-softclouds commented 1 year ago

Solution is to use https://xxxx.es.us-west-2.aws.found.io:9243

davidkyle commented 1 year ago

You can also use the --cloud-id option for elastic cloud, often is it simpler. Your example would be something like:

eland_import_hub_model -u elastic -p XXXXXXXXXX --cloud-id YYYYYYY --hub-model-id sentence-transformers/msmarco-MiniLM-L-12-v3 --task-type text_embedding --start

Copy the cloud ID from the same page in the admin console

Screenshot 2022-11-14 at 14 38 51