search

prestodb / presto-yarn

57 stars 28 forks source link

Possible race condition between process start and plugin copying #49

Closed tglstory closed 8 years ago

tglstory commented 8 years ago

I tested two scenarios: 1) have the presto-server-0.xxx.tar.gz already include the plugin jars I need 2) have slider/presto-yarn copy over the plugins that I specify on appConfig.json

The contents of the $PRESTO_DATA/plugin/ directory are the same in both cases, but in scenario 2 presto instances fail to fetch the dependencies that are in the plugins directory.

mattsfuller commented 8 years ago

Thanks @tglstory. We will take a look.

mattsfuller commented 8 years ago

@kokosing or @anusudarsan

kokosing commented 8 years ago

I will try to reproduce this failure.

kokosing commented 8 years ago

@tglstory

To diagnose the problem you encountered I need more information. Can you please send me a appConfig.json and resources.json files content? Also I would be nice if you could send me a listing files in application package file (like the one shown here: https://github.com/prestodb/presto-yarn#building-the-project).

From the documentation (https://github.com/prestodb/presto-yarn#presto-installation-directory-structure and https://github.com/prestodb/presto-yarn#-appconfigjson (point 12)) it should work. There is also a product tests which ensures that this functionality works (see https://github.com/prestodb/presto-yarn/blob/master/presto-yarn-test/src/main/java/com/teradata/presto/yarn/test/PrestoClusterTest.java#L147). So I would assume that there is something wrong with the configuration you are using.

tglstory commented 8 years ago

We've just decided to compile the tar.gz with the plugins included, as it was too much hassle to try to make this work when we need this productionized fast. When I attempt this again, I'll re-open this case.