Duplicate fails because /_settings is not allowed

avasquez614 commented 3 weeks ago

Duplicates

[X] I have searched the existing issues

Latest version

[X] The issue is in the develop branch
[X] The issue is in the latest released 4.1.x
[ ] The issue is in the latest released 4.0.x
[ ] The issue is in the latest released 3.1.x

Describe the issue

When using Studio with AWS OpenSearch and trying to duplicate a site, it fails because OS blocks /_settings

Steps to reproduce

Steps:

Setup an AWS OpenSearch Domain with version 2.9
In Studio, create a site based in Editorial
Publish the site
Duplicate the site

Duplication fails in Studio, and in the Deployer you can see an error like the one attached

Relevant log output

2024-04-23 20:24:58.643 ERROR 23 --- [http-nio-9191-exec-2] cms.deployer.impl.rest.ExceptionHandlers : Request /api/1/target/duplicate/s3-us-east-1/editorial failed with status 500 INTERNAL_SERVER_ERROR
org.craftercms.deployer.api.exceptions.TargetServiceException: Failed to execute duplicate hooks for target 'editorial2-s3-us-east-1'; nested exception is OpenSearchStatusException[Unable to parse response body]; nested: ResponseException[method [GET], host [...], URI [/_settings?master_timeout=30s&ignore_throttled=false&ignore_unavailable=false&expand_wildcards=open%2Cclosed&allow_no_indices=true], status line [HTTP/1.1 401 Unauthorized]
{"Message":"Your request: '/_settings' is not allowed."}];
    at org.craftercms.deployer.impl.TargetServiceImpl.duplicateTarget(TargetServiceImpl.java:312)
    at org.craftercms.deployer.impl.rest.TargetController.duplicateTarget(TargetController.java:407)
    at org.craftercms.deployer.impl.rest.TargetController$$FastClassBySpringCGLIB$$b2995acb.invoke(<generated>)
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:792)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:762)
    at org.springframework.validation.beanvalidation.MethodValidationInterceptor.invoke(MethodValidationInterceptor.java:123)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:762)
    at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:707)
    at org.craftercms.deployer.impl.rest.TargetController$$EnhancerBySpringCGLIB$$a21e59a6.duplicateTarget(<generated>)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150)
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:903)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:809)
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1072)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:965)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:555)
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:623)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:209)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:168)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:481)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:130)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
    at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:765)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:346)
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:390)
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:928)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1794)
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
    at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
    at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63)
    at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: OpenSearchStatusException[Unable to parse response body]; nested: ResponseException[method [GET], host [...], URI [/_settings?master_timeout=30s&ignore_throttled=false&ignore_unavailable=false&expand_wildcards=open%2Cclosed&allow_no_indices=true], status line [HTTP/1.1 401 Unauthorized]
{"Message":"Your request: '/_settings' is not allowed."}];
    at org.opensearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:2215)
    at org.opensearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1931)
    at org.opensearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1884)
    at org.opensearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1852)
    at org.opensearch.client.IndicesClient.getSettings(IndicesClient.java:727)
    at org.craftercms.search.opensearch.impl.OpenSearchAdminServiceImpl.doGetIndexSettings(OpenSearchAdminServiceImpl.java:322)
    at org.craftercms.search.opensearch.impl.OpenSearchAdminServiceImpl.doDuplicateIndex(OpenSearchAdminServiceImpl.java:371)
    at org.craftercms.search.opensearch.impl.OpenSearchAdminServiceImpl.duplicateIndex(OpenSearchAdminServiceImpl.java:365)
    at org.craftercms.deployer.impl.lifecycle.DuplicateIndexHook.doExecute(DuplicateIndexHook.java:44)
    at org.craftercms.deployer.impl.lifecycle.AbstractLifecycleHook.execute(AbstractLifecycleHook.java:53)
    at org.craftercms.deployer.impl.TargetServiceImpl.executeDuplicateHooks(TargetServiceImpl.java:580)
    at org.craftercms.deployer.impl.TargetServiceImpl.duplicateTarget(TargetServiceImpl.java:310)
    ... 62 more
    Suppressed: ParsingException[Failed to parse object: expecting field with name [error] but found [Message]]
        at org.opensearch.common.xcontent.XContentParserUtils.ensureFieldName(XContentParserUtils.java:66)
        at org.opensearch.OpenSearchException.failureFromXContent(OpenSearchException.java:632)
        at org.opensearch.rest.BytesRestResponse.errorFromXContent(BytesRestResponse.java:201)
        at org.opensearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:2235)
        at org.opensearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:2212)
        ... 73 more
Caused by: org.opensearch.client.ResponseException: method [GET], host [...], URI [/_settings?master_timeout=30s&ignore_throttled=false&ignore_unavailable=false&expand_wildcards=open%2Cclosed&allow_no_indices=true], status line [HTTP/1.1 401 Unauthorized]
{"Message":"Your request: '/_settings' is not allowed."}
    at org.opensearch.client.RestClient.convertResponse(RestClient.java:375)
    at org.opensearch.client.RestClient.performRequest(RestClient.java:345)
    at org.opensearch.client.RestClient.performRequest(RestClient.java:320)
    at org.opensearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1918)
    ... 72 more

Screenshots and/or videos

No response

jmendeza commented 3 weeks ago

This can happen when the sourceSiteName sent to the duplicate target hook is empty (probably due to misconfiguration in the target template). The hook has been updated so it prevents its execution when sourceSiteName is an empty value.

avasquez614 commented 3 weeks ago

Duplicate is still failing for anothe reason. In the duplicate code in Deployer we're calling resolveTargetFromConfigFile before executeDuplicateHooks. These causes the init hooks to be executed before the duplicate hooks. init should be always executed after the create and duplicate hooks (see https://github.com/craftercms/deployer/blob/develop/src/main/java/org/craftercms/deployer/impl/TargetServiceImpl.java#L411). I think we need to modify the loadTarget method to accept a CreateMode enum (which can be either CREATE or DUPLICATE) instead of passing a boolean so we can correctly call the correct create or duplicate hooks before the init hooks.

jmendeza commented 3 weeks ago

Updated so init hooks are executed after create/duplicate hooks

avasquez614 commented 3 weeks ago

Verified in 4.1.4

craftercms / craftercms