search

prestodb / presto

The official home of the Presto distributed SQL query engine for big data
http://prestodb.io
Apache License 2.0
15.74k stars 5.28k forks source link

Automatically update the configuration or load a new connector in Presto #14964

Open HuangDCat opened 3 years ago

HuangDCat commented 3 years ago

If I modify the configuration or add a new connector, do I have to restart the coordinator and worker?

Is there any other way to make the coordinator and worker recognize the configuration or new connector and load it automatically?

Looking forward to the answer, thank you

wenleix commented 3 years ago

You need to restart the cluster if you want to change the configuration.

wenleix commented 3 years ago

Speaking of adding dynamic loading/removing connectors, this has been a very long discussion (see for example https://github.com/prestodb/presto/issues/2445). And we have no plan to add it in near future , as there's no reliable way of removing registered connectors without introducing memory / thread / class loader leaks.

isonusingh commented 3 years ago

As of now, I created a Trino Agent App which is actually a Spring Boot App and providing Rest APIs to manager catalogs on Trino Server. For example, I can add, edit and delete catalog on Trino Server using my agent app. Also, I am restarting the Trino server using my agent app after adding, editing and deleting catalogs.

As of now, I am able to add catalogs for the below connectors: Hive MySQL SQLServer MongoDB Kafka MemSQL Oracle PostgreSQL RedShift Google BigQuery This approach is working fine with a single user but when multiple users will use Agent App that will be a problem. So, I added a locking mechanism in my agent app and now one user can only add/edit/delete catalogs.

For my use-case, Trino should load catalogs dynamically and also I am feeling that this feature is very much required in Trino.

tooptoop4 commented 2 years ago

have u open sourced it @SonuSingh200190 ?