search

trinodb / trino

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)
https://trino.io
Apache License 2.0
10.35k stars 2.98k forks source link

Test connection failed on Ranger UI, NPE during trino server start-up after enabling Ranger-Trino plugin #16404

Closed debimishra89 closed 1 year ago

debimishra89 commented 1 year ago

Hi Team, I am still having issues with Ranger-Trino integration. Test connection succeeds on Ranger Admin UI when ranger-trino plugin is disabled, but Fails when this is enabled. Ranger version-2.3.0 Trino version-388. Trino server log--> 2023-03-07T16:00:10.533+0800 INFO main stdout INFO - This policy engine contains 7 policy evaluators > 2023-03-07T16:00:10.533+0800 INFO main stdout DEBUG - policy evaluation order: 7 policies > 2023-03-07T16:00:10.533+0800 INFO main stdout DEBUG - policy evaluation order: #1 - policy id=30; name=all - catalog, schema, table, column; evalOrder=9848 > 2023-03-07T16:00:10.533+0800 INFO main stdout DEBUG - policy evaluation order: #2 - policy id=27; name=all - catalog, schema, procedure; evalOrder=9894 > 2023-03-07T16:00:10.533+0800 INFO main stdout DEBUG - policy evaluation order: #3 - policy id=26; name=all - catalog, sessionproperty; evalOrder=9922 > 2023-03-07T16:00:10.533+0800 INFO main stdout DEBUG - policy evaluation order: #4 - policy id=24; name=all - catalog; evalOrder=9923 > 2023-03-07T16:00:10.533+0800 INFO main stdout DEBUG - policy evaluation order: #5 - policy id=25; name=all - function; evalOrder=9945 > 2023-03-07T16:00:10.533+0800 INFO main stdout DEBUG - policy evaluation order: #6 - policy id=23; name=all - trinouser; evalOrder=9946 > 2023-03-07T16:00:10.533+0800 INFO main stdout DEBUG - policy evaluation order: #7 - policy id=29; name=all - systemproperty; evalOrder=9947 > 2023-03-07T16:00:10.533+0800 INFO main stdout DEBUG - dataMask policy evaluation order: 0 policies > 2023-03-07T16:00:10.533+0800 INFO main stdout DEBUG - rowFilter policy evaluation order: 0 policies > 2023-03-07T16:00:10.536+0800 INFO main stdout ERROR - setPolicies: policy engine initialization failed! Leaving current policy engine as-is. Exception : java.lang.NullPointerException: null > at org.apache.ranger.plugin.policyengine.RangerPolicyRepository.init(RangerPolicyRepository.java:991) > at org.apache.ranger.plugin.policyengine.RangerPolicyRepository.<init>(RangerPolicyRepository.java:231) > at org.apache.ranger.plugin.policyengine.RangerPolicyRepository.<init>(RangerPolicyRepository.java:182) > at org.apache.ranger.plugin.policyengine.PolicyEngine.<init>(PolicyEngine.java:212) > at org.apache.ranger.plugin.policyengine.RangerPolicyEngineImpl.<init>(RangerPolicyEngineImpl.java:104) > at org.apache.ranger.plugin.service.RangerBasePlugin.setPolicies(RangerBasePlugin.java:364) > at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:263) > at org.apache.ranger.plugin.util.PolicyRefresher.startRefresher(PolicyRefresher.java:145) > at org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:243) > at org.apache.ranger.authorization.trino.authorizer.RangerSystemAccessControl.<init>(RangerSystemAccessControl.java:116) > at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490) > at org.apache.ranger.authorization.trino.authorizer.RangerSystemAccessControl.<init>(RangerSystemAccessControl.java:65) > at org.apache.ranger.authorization.trino.authorizer.RangerSystemAccessControl$$FastClassByGuice$$f94a7f6a.newInstance(<generated>) > at com.google.inject.internal.DefaultConstructionProxyFactory$FastClassProxy.newInstance(DefaultConstructionProxyFactory.java:89) > at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:114) > at com.google.inject.internal.ConstructorInjector.access$000(ConstructorInjector.java:32) > at com.google.inject.internal.ConstructorInjector$1.call(ConstructorInjector.java:98) > at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:112) > at io.airlift.bootstrap.LifeCycleModule.provision(LifeCycleModule.java:54) > at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:120) > at com.google.inject.internal.ProvisionListenerStackCallback.provision(ProvisionListenerStackCallback.java:66) > at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:93) > at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:306) > at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40) > at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:168) > at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:39) > at com.google.inject.internal.InternalInjectorCreator.loadEagerSingletons(InternalInjectorCreator.java:211) > at com.google.inject.internal.InternalInjectorCreator.injectDynamically(InternalInjectorCreator.java:182) > at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:109) > at com.google.inject.Guice.createInjector(Guice.java:87) > at io.airlift.bootstrap.Bootstrap.initialize(Bootstrap.java:262) > at org.apache.ranger.authorization.trino.authorizer.RangerSystemAccessControlFactory.create(RangerSystemAccessControlFactory.java:53) > at io.trino.security.AccessControlManager.createSystemAccessControl(AccessControlManager.java:181) > at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) > at java.base/java.util.Collections$2.tryAdvance(Collections.java:4747) > at java.base/java.util.Collections$2.forEachRemaining(Collections.java:4755) > at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) > at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) > at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) > at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) > at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) > at io.trino.security.AccessControlManager.loadSystemAccessControl(AccessControlManager.java:150) > at io.trino.server.Server.doStart(Server.java:136) > at io.trino.server.Server.lambda$start$0(Server.java:80) > at io.trino.$gen.Trino_388____20230307_080002_1.run(Unknown Source) > at io.trino.server.Server.start(Server.java:80) > at io.trino.server.TrinoServer.main(TrinoServer.java:38)

Slack Message

dprophet commented 1 year ago

I suspect your configs are empty

https://github.com/dprophet/trino/blob/master/docs/src/main/sphinx/connector/ranger.rst#configuration

I have changes I havent pushed upstream. https://github.com/dprophet/trino/blob/master/plugin/trino-ranger/src/main/java/io/trino/plugin/ranger/RangerSystemAccessControlFactory.java#L42 requireNonNull(config, "config is null"); Should also be not empty check

debimishra89 commented 1 year ago

I suspect your configs are empty

https://github.com/dprophet/trino/blob/master/docs/src/main/sphinx/connector/ranger.rst#configuration

I have changes I havent pushed upstream. https://github.com/dprophet/trino/blob/master/plugin/trino-ranger/src/main/java/io/trino/plugin/ranger/RangerSystemAccessControlFactory.java#L42 requireNonNull(config, "config is null"); Should also be not empty check

Thanks for the guidance. It looks like it doesn't recognise these config properties

`2023-03-08T09:52:22.727+0800    INFO    main    Bootstrap       PROPERTY              DEFAULT  RUNTIME                                                     DESCRIPTION
2023-03-08T09:52:22.727+0800    INFO    main    Bootstrap       ranger.hadoop_config  ----     /opt/darc/trino/trino-server-388/etc/trino-ranger-site.xml  Path to hadoop configuration. Defaults to trino-ranger-site.xml in classpath
2023-03-08T09:52:22.727+0800    INFO    main    Bootstrap       ranger.keytab         ----     ----                                                        Keytab for authentication against Ranger
2023-03-08T09:52:22.727+0800    INFO    main    Bootstrap       ranger.principal      ----     ----                                                        Principal for authentication against Ranger with keytab
2023-03-08T09:52:22.727+0800    INFO    main    Bootstrap       ranger.use_ugi        false    true                                                        Use Hadoop User Group Information instead of Trino groups
2023-03-08T09:52:22.727+0800    WARN    main    Bootstrap       UNUSED PROPERTIES
2023-03-08T09:52:22.727+0800    WARN    main    Bootstrap       ranger.audit_resource
2023-03-08T09:52:22.727+0800    WARN    main    Bootstrap       ranger.policy_manager_ssl_resource
2023-03-08T09:52:22.727+0800    WARN    main    Bootstrap       ranger.security_resource
2023-03-08T09:52:22.727+0800    WARN    main    Bootstrap       ranger.service_name
2023-03-08T09:52:22.727+0800    WARN    main    Bootstrap
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$1 (file:/opt/darc/trino/trino_data/plugin/ranger/.guice-4.2.2.jar.20230220115735) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$1
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
2023-03-08T09:52:22.802+0800    ERROR   main    io.trino.server.Server  Unable to create injector, see the following errors:

1) Configuration property 'ranger.audit_resource' was not used

2) Configuration property 'ranger.policy_manager_ssl_resource' was not used

3) Configuration property 'ranger.security_resource' was not used

4) Configuration property 'ranger.service_name' was not used

4 errors
com.google.inject.CreationException: Unable to create injector, see the following errors:

1) Configuration property 'ranger.audit_resource' was not used

2) Configuration property 'ranger.policy_manager_ssl_resource' was not used

3) Configuration property 'ranger.security_resource' was not used

4) Configuration property 'ranger.service_name' was not used

4 errors
        at com.google.inject.internal.Errors.throwCreationExceptionIfErrorsExist(Errors.java:543)
        at com.google.inject.internal.InternalInjectorCreator.initializeStatically(InternalInjectorCreator.java:159)
        at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:106)
        at com.google.inject.Guice.createInjector(Guice.java:87)
        at io.airlift.bootstrap.Bootstrap.initialize(Bootstrap.java:262)
        at org.apache.ranger.authorization.trino.authorizer.RangerSystemAccessControlFactory.create(RangerSystemAccessControlFactory.java:53)
        at io.trino.security.AccessControlManager.createSystemAccessControl(AccessControlManager.java:181)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
        at java.base/java.util.Collections$2.tryAdvance(Collections.java:4747)
        at java.base/java.util.Collections$2.forEachRemaining(Collections.java:4755)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
        at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
        at io.trino.security.AccessControlManager.loadSystemAccessControl(AccessControlManager.java:150)
        at io.trino.server.Server.doStart(Server.java:136)
        at io.trino.server.Server.lambda$start$0(Server.java:80)
        at io.trino.$gen.Trino_388____20230308_015215_1.run(Unknown Source)
        at io.trino.server.Server.start(Server.java:80)
        at io.trino.server.TrinoServer.main(TrinoServer.java:38)

`

debimishra89 commented 1 year ago

@dprophet , If I build trino from your forked repo, this above error should go away? I have built from your forked repo, but no idea how to use it. Any steps ?