search

Australian-Imaging-Service / pipelines

Scripts to generate analysis pipelines that can be run in XNAT's container service
Other
1 stars 3 forks source link

[STORY] Look into issue with large upload failures (>60GB) #310

Open tclose opened 2 weeks ago

tclose commented 2 weeks ago

Metadata

Epic: #271 Feature: Feature Release: Required knowledge: shallow or mid-level or deep

Description

There is an issue when zipped sessions are uploaded that are >25GB in size. Could be due to extraction of files and temporary storage issues

Acceptance Criteria

anzhao commented 4 days ago

Hi Fang, just to provide some further information for you to debug this error. When upload the file over 60GB, I got the following error:

| Jul 5, 2024 @ 20:36:46.804 | time:Jul 5, 2024 @ 20:36:46.804 stream:stdout log:2024-07-05 11:06:46,804 [http-nio-8080-exec-11] ERROR org.nrg.xnat.restlet.resources.UserCacheResource - docker.container_id:9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5 kubernetes.container_name:xnat-web kubernetes.namespace_name:xnatdev kubernetes.pod_name:xnateks-xnat-web-0 kubernetes.pod_id:84e7c6d9-5a56-44a8-a1ad-56b37302fec7 kubernetes.pod_ip:10.86.15.222 kubernetes.host:ip-10-86-15-165.ap-southeast-2.compute.internal _id:FJyTgpABwcjZ3i_noaDl _type: - _index:dev.ais.var.log.containers.xnateks-xnat-web-0_xnatdev_xnat-web-9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5.log _score: -

  | Jul 5, 2024 @ 20:36:46.804 | time:Jul 5, 2024 @ 20:36:46.804 stream:stdout log:org.apache.commons.fileupload.FileUploadBase$IOFileUploadException: Processing of multipart/form-data request failed. java.io.EOFException at org.apache.commons.fileupload.FileUploadBase.parseRequest(FileUploadBase.java:380) at org.restlet.ext.fileupload.RestletFileUpload.parseRequest(RestletFileUpload.java:104) at org.nrg.xnat.restlet.resources.UserCacheResource.handleAttachedUserFileUpload(UserCacheResource.java:414) at org.nrg.xnat.restlet.resources.UserCacheResource.uploadUserFile(UserCacheResource.java:360) at org.nrg.xnat.restlet.resources.UserCacheResource.handlePut(UserCacheResource.java:165) at org.restlet.Finder.handle(Finder.java:359) at org.restlet.Filter.doHandle(Filter.java:150) at org.restlet.Filter.handle(Filter.java:195) at org.restlet.Router.handle(Router.java:504) at org.restlet.Filter.doHandle(Filter.java:150) at org.restlet.Filter.handle(Filter.java:195) at org.restlet.Filter.doHandle(Filter.java:150) at org.restlet.Filter.handle(Filter.java:195) at org.restlet.Router.handle(Router.java:504) at org.restlet.Filter.doHandle(Filter.java:150) at org.restlet.Filter.handle(Filter.java:195) at org.restlet.Filter.doHandle(Filter.java:150) at org.restlet.Filter.handle(Filter.java:195) at org.restlet.Filter.doHandle(Filter.java:150) at com.noelios.restlet.StatusFilter.doHandle(StatusFilter.java:130) at org.restlet.Filter.handle(Filter.java:195) at org.restlet.Filter.doHandle(Filter.java:150) at org.restlet.Filter.handle(Filter.java:195) at com.noelios.restlet.ChainHelper.handle(ChainHelper.java:124) at com.noelios.restlet.application.ApplicationHelper.handle(ApplicationHelper.java:112) at org.restlet.Application.handle(Application.java:341) at org.restlet.Filter.doHandle(Filter.java:150) at org.restlet.Filter.handle(Filter.java:195) at org.restlet.Router.handle(Router.java:504) at org.restlet.Filter.doHandle(Filter.java:150) at org.restlet.Filter.handle(Filter.java:195) at org.restlet.Router.handle(Router.java:504) at org.restlet.Filter.doHandle(Filter.java:150) at com.noelios.restlet.StatusFilter.doHandle(StatusFilter.java:130) at org.restlet.Filter.handle(Filter.java:195) at org.restlet.Filter.doHandle(Filter.java:150) at org.restlet.Filter.handle(Filter.java:195) at com.noelios.restlet.ChainHelper.handle(ChainHelper.java:124) at org.restlet.Component.handle(Component.java:673) at org.restlet.Server.handle(Server.java:331) at com.noelios.restlet.ServerHelper.handle(ServerHelper.java:68) at com.noelios.restlet.http.HttpServerHelper.handle(HttpServerHelper.java:147) at com.noelios.restlet.ext.servlet.ServerServlet.service(ServerServlet.java:881) at javax.servlet.http.HttpServlet.service(HttpServlet.java:623) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:199) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:168) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144) at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:239) at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:215) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:168) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317) at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127) at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.nrg.xnat.security.XnatInitCheckFilter.doFilterInternal(XnatInitCheckFilter.java:41) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.nrg.xnat.security.XnatExpiredPasswordFilter.doFilterInternal(XnatExpiredPasswordFilter.java:193) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.nrg.xnat.security.XnatBasicAuthenticationFilter.doFilterInternal(XnatBasicAuthenticationFilter.java:117) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.web.session.ConcurrentSessionFilter.doFilter(ConcurrentSessionFilter.java:155) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.oauth2.client.filter.OAuth2ClientContextFilter.doFilter(OAuth2ClientContextFilter.java:64) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:66) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.web.access.channel.ChannelProcessingFilter.doFilter(ChannelProcessingFilter.java:157) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214) at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:347) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:263) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:168) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:168) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:481) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:130) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:660) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:346) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:388) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:928) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1791) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52) at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1190) at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63) at java.lang.Thread.run(Thread.java:750)Caused by: org.apache.catalina.connector.ClientAbortException: java.io.EOFException at org.apache.catalina.connector.InputBuffer.realReadBytes(InputBuffer.java:312) at org.apache.catalina.connector.InputBuffer.checkByteBufferEof(InputBuffer.java:615) at org.apache.catalina.connector.InputBuffer.read(InputBuffer.java:358) at org.apache.catalina.connector.CoyoteInputStream.read(CoyoteInputStream.java:132) at org.apache.commons.fileupload.MultipartStream$ItemInputStream.makeAvailable(MultipartStream.java:1027) at org.apache.commons.fileupload.MultipartStream$ItemInputStream.read(MultipartStream.java:931) at java.io.InputStream.read(InputStream.java:101) at org.apache.commons.fileupload.util.Streams.copy(Streams.java:98) at org.apache.commons.fileupload.FileUploadBase.parseRequest(FileUploadBase.java:376) ... 123 common frames omittedCaused by: java.io.EOFException: null at org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper.fillReadBuffer(NioEndpoint.java:1320) at org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper.read(NioEndpoint.java:1231) at org.apache.coyote.http11.Http11InputBuffer.fill(Http11InputBuffer.java:789) at org.apache.coyote.http11.Http11InputBuffer.access$400(Http11InputBuffer.java:41) at org.apache.coyote.http11.Http11InputBuffer$SocketInputBuffer.doRead(Http11InputBuffer.java:1195) at org.apache.coyote.http11.filters.IdentityInputFilter.doRead(IdentityInputFilter.java:97) at org.apache.coyote.http11.Http11InputBuffer.doRead(Http11InputBuffer.java:244) at org.apache.coyote.Request.doRead(Request.java:626) at org.apache.catalina.connector.InputBuffer.realReadBytes(InputBuffer.java:303) ... 131 common frames omitted docker.container_id:9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5 kubernetes.container_name:xnat-web kubernetes.namespace_name:xnatdev kubernetes.pod_name:xnateks-xnat-web-0 kubernetes.pod_id:84e7c6d9-5a56-44a8-a1ad-56b37302fec7 kubernetes.pod_ip:10.86.15.222 kubernetes.host:ip-10-86-15-165.ap-southeast-2.compute.internal _id:FZyTgpABwcjZ3i_noaDl _type: - _index:dev.ais.var.log.containers.xnateks-xnat-web-0_xnatdev_xnat-web-9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5.log _score: -

  | Jul 5, 2024 @ 20:36:45.049 | time:Jul 5, 2024 @ 20:36:45.049 stream:stdout log:2024-07-05 11:06:45,048 - antest 3.25.93.73 PUT http://dev.xnat.ais.sydney.edu.au/data/user/cache/resources/20240705_105544/files/AILDR_SAMPLE_data.zip?XNAT_CSRF=f5e965ad-d876-4763-942a-34a63d19eeb8&x=xhrfe21n7ee "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36" docker.container_id:9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5 kubernetes.container_name:xnat-web kubernetes.namespace_name:xnatdev kubernetes.pod_name:xnateks-xnat-web-0 kubernetes.pod_id:84e7c6d9-5a56-44a8-a1ad-56b37302fec7 kubernetes.pod_ip:10.86.15.222 kubernetes.host:ip-10-86-15-165.ap-southeast-2.compute.internal _id:E5yTgpABwcjZ3i_noaDl _type: - _index:dev.ais.var.log.containers.xnateks-xnat-web-0_xnatdev_xnat-web-9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5.log _score: -

  | Jul 5, 2024 @ 20:34:34.976 | time:Jul 5, 2024 @ 20:34:34.976 stream:stdout log:2024-07-05 11:04:34,976 - antest 3.25.93.73 PUT http://dev.xnat.ais.sydney.edu.au/data/user/cache/resources/20240705_105544/files/AILDR_SAMPLE_data.zip?XNAT_CSRF=f5e965ad-d876-4763-942a-34a63d19eeb8&x=xhrfe21n7ee "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36" docker.container_id:9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5 kubernetes.container_name:xnat-web kubernetes.namespace_name:xnatdev kubernetes.pod_name:xnateks-xnat-web-0 kubernetes.pod_id:84e7c6d9-5a56-44a8-a1ad-56b37302fec7 kubernetes.pod_ip:10.86.15.222 kubernetes.host:ip-10-86-15-165.ap-southeast-2.compute.internal _id:_pyRgpABwcjZ3i_nzJ_t _type: - _index:dev.ais.var.log.containers.xnateks-xnat-web-0_xnatdev_xnat-web-9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5.log _score: -

  | Jul 5, 2024 @ 20:34:17.313 | time:Jul 5, 2024 @ 20:34:17.313 stream:stdout log:2024-07-05 11:04:17,313 - antest 3.25.93.73 GET http://dev.xnat.ais.sydney.edu.au/xapi/siteConfig/buildInfo?x=xhr6derk1gm "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36" docker.container_id:9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5 kubernetes.container_name:xnat-web kubernetes.namespace_name:xnatdev kubernetes.pod_name:xnateks-xnat-web-0 kubernetes.pod_id:84e7c6d9-5a56-44a8-a1ad-56b37302fec7 kubernetes.pod_ip:10.86.15.222 kubernetes.host:ip-10-86-15-165.ap-southeast-2.compute.internal _id:gueRgpABKvgvenOxV8WI _type: - _index:dev.ais.var.log.containers.xnateks-xnat-web-0_xnatdev_xnat-web-9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5.log _score: -

  | Jul 5, 2024 @ 20:33:49.293 | time:Jul 5, 2024 @ 20:33:49.293 stream:stderr log:05-Jul-2024 11:03:49.293 INFO [http-nio-8080-exec-5] org.apache.coyote.http11.Http11Processor.service Error parsing HTTP request header docker.container_id:9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5 kubernetes.container_name:xnat-web kubernetes.namespace_name:xnatdev kubernetes.pod_name:xnateks-xnat-web-0 kubernetes.pod_id:84e7c6d9-5a56-44a8-a1ad-56b37302fec7 kubernetes.pod_ip:10.86.15.222 kubernetes.host:ip-10-86-15-165.ap-southeast-2.compute.internal _id:f-eQgpABKvgvenOx4sUs _type: - _index:dev.ais.var.log.containers.xnateks-xnat-web-0_xnatdev_xnat-web-9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5.log _score: -

  | Jul 5, 2024 @ 20:33:49.293 | time:Jul 5, 2024 @ 20:33:49.293 stream:stderr log: Note: further occurrences of HTTP request parsing errors will be logged at DEBUG level. docker.container_id:9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5 kubernetes.container_name:xnat-web kubernetes.namespace_name:xnatdev kubernetes.pod_name:xnateks-xnat-web-0 kubernetes.pod_id:84e7c6d9-5a56-44a8-a1ad-56b37302fec7 kubernetes.pod_ip:10.86.15.222 kubernetes.host:ip-10-86-15-165.ap-southeast-2.compute.internal _id:gOeQgpABKvgvenOx4sUs _type: - _index:dev.ais.var.log.containers.xnateks-xnat-web-0_xnatdev_xnat-web-9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5.log _score: -

  | Jul 5, 2024 @ 20:33:49.293 | time:Jul 5, 2024 @ 20:33:49.293 stream:stderr log: java.lang.IllegalArgumentException: Invalid character found in the request target [/wp-json/lp/v1/load_content_via_ajax/?callback={"class"%3a"LP_Debug","method"%3a"var_dump"}&args="2iopsQ6KU1cOgEeO3JU52wzgujo" ]. The valid characters are defined in RFC 7230 and RFC 3986 at org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:482) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:261) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:928) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1791) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52) at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1190) at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63) at java.lang.Thread.run(Thread.java:750) docker.container_id:9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5 kubernetes.container_name:xnat-web kubernetes.namespace_name:xnatdev kubernetes.pod_name:xnateks-xnat-web-0 kubernetes.pod_id:84e7c6d9-5a56-44a8-a1ad-56b37302fec7 kubernetes.pod_ip:10.86.15.222 kubernetes.host:ip-10-86-15-165.ap-southeast-2.compute.internal _id:geeQgpABKvgvenOx4sUs _type: - _index:dev.ais.var.log.containers.xnateks-xnat-web-0_xnatdev_xnat-web-9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5.log _score: -

  | Jul 5, 2024 @ 20:33:08.985 | time:Jul 5, 2024 @ 20:33:08.985 stream:stdout log:2024-07-05 11:03:08,985 - antest 3.25.93.73 GET http://dev.xnat.ais.sydney.edu.au/xapi/siteConfig/buildInfo?x=xhrkr1n7p8h "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36" docker.container_id:9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5 kubernetes.container_name:xnat-web kubernetes.namespace_name:xnatdev kubernetes.pod_name:xnateks-xnat-web-0 kubernetes.pod_id:84e7c6d9-5a56-44a8-a1ad-56b37302fec7 kubernetes.pod_ip:10.86.15.222 kubernetes.host:ip-10-86-15-165.ap-southeast-2.compute.internal _id:1ZyQgpABwcjZ3i_nbJ_R _type: - _index:dev.ais.var.log.containers.xnateks-xnat-web-0_xnatdev_xnat-web-9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5.log _score: -

  | Jul 5, 2024 @ 20:32:07.975 | time:Jul 5, 2024 @ 20:32:07.975 stream:stdout log:2024-07-05 11:02:07,975 - antest 3.25.93.73 GET http://dev.xnat.ais.sydney.edu.au/xapi/siteConfig/buildInfo?x=xhrna8w9vt3 "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36" docker.container_id:9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5 kubernetes.container_name:xnat-web kubernetes.namespace_name:xnatdev kubernetes.pod_name:xnateks-xnat-web-0 kubernetes.pod_id:84e7c6d9-5a56-44a8-a1ad-56b37302fec7 kubernetes.pod_ip:10.86.15.222 kubernetes.host:ip-10-86-15-165.ap-southeast-2.compute.internal _id:c-ePgpABKvgvenOxgsVA _type: - _index:dev.ais.var.log.containers.xnateks-xnat-web-0_xnatdev_xnat-web-9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5.log _score: -

  | Jul 5, 2024 @ 20:31:06.984 | time:Jul 5, 2024 @ 20:31:06.984 stream:stdout log:2024-07-05 11:01:06,984 - antest 3.25.93.73 GET http://dev.xnat.ais.sydney.edu.au/xapi/siteConfig/buildInfo?x=xhr3e3ngyzy "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36" docker.container_id:9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5 kubernetes.container_name:xnat-web kubernetes.namespace_name:xnatdev kubernetes.pod_name:xnateks-xnat-web-0 kubernetes.pod_id:84e7c6d9-5a56-44a8-a1ad-56b37302fec7 kubernetes.pod_ip:10.86.15.222 kubernetes.host:ip-10-86-15-165.ap-southeast-2.compute.internal _id:0pyOgpABwcjZ3i_nl5-0 _type: - _index:dev.ais.var.log.containers.xnateks-xnat-web-0_xnatdev_xnat-web-9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5.log _score: -

  | Jul 5, 2024 @ 20:30:05.991 | time:Jul 5, 2024 @ 20:30:05.991 stream:stdout log:2024-07-05 11:00:05,990 - antest 3.25.93.73 GET http://dev.xnat.ais.sydney.edu.au/xapi/siteConfig/buildInfo?x=xhrbf8xv0kr "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36" docker.container_id:9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5 kubernetes.container_name:xnat-web kubernetes.namespace_name:xnatdev kubernetes.pod_name:xnateks-xnat-web-0 kubernetes.pod_id:84e7c6d9-5a56-44a8-a1ad-56b37302fec7 kubernetes.pod_ip:10.86.15.222 kubernetes.host:ip-10-86-15-165.ap-southeast-2.compute.internal _id:z5yNgpABwcjZ3i_nrZ8o _type: - _index:dev.ais.var.log.containers.xnateks-xnat-web-0_xnatdev_xnat-web-9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5.log _score: -

  | Jul 5, 2024 @ 20:29:04.984 | time:Jul 5, 2024 @ 20:29:04.984 stream:stdout log:2024-07-05 10:59:04,984 - antest 3.25.93.73 GET http://dev.xnat.ais.sydney.edu.au/xapi/siteConfig/buildInfo?x=xhrquy613uc "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36" docker.container_id:9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5 kubernetes.container_name:xnat-web kubernetes.namespace_name:xnatdev kubernetes.pod_name:xnateks-xnat-web-0 kubernetes.pod_id:84e7c6d9-5a56-44a8-a1ad-56b37302fec7 kubernetes.pod_ip:10.86.15.222 kubernetes.host:ip-10-86-15-165.ap-southeast-2.compute.internal _id:iOeMgpABKvgvenOxwq2W _type: - _index:dev.ais.var.log.containers.xnateks-xnat-web-0_xnatdev_xnat-web-9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5.log _score: -

  | Jul 5, 2024 @ 20:28:03.982 | time:Jul 5, 2024 @ 20:28:03.982 stream:stdout log:2024-07-05 10:58:03,982 - antest 3.25.93.73 GET http://dev.xnat.ais.sydney.edu.au/xapi/siteConfig/buildInfo?x=xhrafpa8ngx "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36" docker.container_id:9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5 kubernetes.container_name:xnat-web kubernetes.namespace_name:xnatdev kubernetes.pod_name:xnateks-xnat-web-0 kubernetes.pod_id:84e7c6d9-5a56-44a8-a1ad-56b37302fec7 kubernetes.pod_ip:10.86.15.222 kubernetes.host:ip-10-86-15-165.ap-southeast-2.compute.internal _id:heeLgpABKvgvenOx2K0E _type: - _index:dev.ais.var.log.containers.xnateks-xnat-web-0_xnatdev_xnat-web-9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5.log _score: -

  | Jul 5, 2024 @ 20:27:02.985 | time:Jul 5, 2024 @ 20:27:02.985 stream:stdout log:2024-07-05 10:57:02,985 - antest 3.25.93.73 GET http://dev.xnat.ais.sydney.edu.au/xapi/siteConfig/buildInfo?x=xhrw53iwu96 "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36" docker.container_id:9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5 kubernetes.container_name:xnat-web kubernetes.namespace_name:xnatdev kubernetes.pod_name:xnateks-xnat-web-0 kubernetes.pod_id:84e7c6d9-5a56-44a8-a1ad-56b37302fec7 kubernetes.pod_ip:10.86.15.222 kubernetes.host:ip-10-86-15-165.ap-southeast-2.compute.internal _id:t5yKgpABwcjZ3i_n7ZV3 _type: - _index:dev.ais.var.log.containers.xnateks-xnat-web-0_xnatdev_xnat-web-9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5.log _score: -

  | Jul 5, 2024 @ 20:25:44.987 | time:Jul 5, 2024 @ 20:25:44.987 stream:stdout log:2024-07-05 10:55:44,986 - antest 3.25.93.73 GET http://dev.xnat.ais.sydney.edu.au/xapi/xnatTask/checkNodeConfigurationStatus?_=1720176944804 "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36" docker.container_id:9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5 kubernetes.container_name:xnat-web kubernetes.namespace_name:xnatdev kubernetes.pod_name:xnateks-xnat-web-0 kubernetes.pod_id:84e7c6d9-5a56-44a8-a1ad-56b37302fec7 kubernetes.pod_ip:10.86.15.222 kubernetes.host:ip-10-86-15-165.ap-southeast-2.compute.internal _id:cueJgpABKvgvenOxja2A _type: - _index:dev.ais.var.log.containers.xnateks-xnat-web-0_xnatdev_xnat-web-9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5.log _score: -

  | Jul 5, 2024 @ 20:25:44.984 | time:Jul 5, 2024 @ 20:25:44.984 stream:stdout log:2024-07-05 10:55:44,984 - antest 3.25.93.73 GET http://dev.xnat.ais.sydney.edu.au/xapi/siteConfig/buildInfo "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36" docker.container_id:9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5 kubernetes.container_name:xnat-web kubernetes.namespace_name:xnatdev kubernetes.pod_name:xnateks-xnat-web-0 kubernetes.pod_id:84e7c6d9-5a56-44a8-a1ad-56b37302fec7 kubernetes.pod_ip:10.86.15.222 kubernetes.host:ip-10-86-15-165.ap-southeast-2.compute.internal _id:ceeJgpABKvgvenOxja2A _type: - _index:dev.ais.var.log.containers.xnateks-xnat-web-0_xnatdev_xnat-web-9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5.log _score: -

  | Jul 5, 2024 @ 20:25:44.941 | time:Jul 5, 2024 @ 20:25:44.941 stream:stdout log:2024-07-05 10:55:44,941 - antest 3.25.93.73 GET http://dev.xnat.ais.sydney.edu.au/REST/projects?XNAT_CSRF=f5e965ad-d876-4763-942a-34a63d19eeb8&format=json&timestamp=1720176944933&owner=true&member=true&prearc_code=true&rnd=1720176944933 "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36" docker.container_id:9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5 kubernetes.container_name:xnat-web kubernetes.namespace_name:xnatdev kubernetes.pod_name:xnateks-xnat-web-0 kubernetes.pod_id:84e7c6d9-5a56-44a8-a1ad-56b37302fec7 kubernetes.pod_ip:10.86.15.222 kubernetes.host:ip-10-86-15-165.ap-southeast-2.compute.internal _id:b-eJgpABKvgvenOxja2A _type: - _index:dev.ais.var.log.containers.xnateks-xnat-web-0_xnatdev_xnat-web-9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5.log _score: -

  | Jul 5, 2024 @ 20:25:44.941 | time:Jul 5, 2024 @ 20:25:44.941 stream:stdout log:2024-07-05 10:55:44,941 - antest 3.25.93.73 GET http://dev.xnat.ais.sydney.edu.au/xapi/siteConfig?format=json "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36" docker.container_id:9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5 kubernetes.container_name:xnat-web kubernetes.namespace_name:xnatdev kubernetes.pod_name:xnateks-xnat-web-0 kubernetes.pod_id:84e7c6d9-5a56-44a8-a1ad-56b37302fec7 kubernetes.pod_ip:10.86.15.222 kubernetes.host:ip-10-86-15-165.ap-southeast-2.compute.internal _id:cOeJgpABKvgvenOxja2A _type: - _index:dev.ais.var.log.containers.xnateks-xnat-web-0_xnatdev_xnat-web-9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5.log _score: -

  | Jul 5, 2024 @ 20:25:44.918 | time:Jul 5, 2024 @ 20:25:44.918 stream:stdout log:2024-07-05 10:55:44,917 - antest 3.25.93.73 GET http://dev.xnat.ais.sydney.edu.au/xapi/siteConfig/buildInfo?x=xhrlgpjnd92 "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36" docker.container_id:9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5 kubernetes.container_name:xnat-web kubernetes.namespace_name:xnatdev kubernetes.pod_name:xnateks-xnat-web-0 kubernetes.pod_id:84e7c6d9-5a56-44a8-a1ad-56b37302fec7 kubernetes.pod_ip:10.86.15.222 kubernetes.host:ip-10-86-15-165.ap-southeast-2.compute.internal _id:bueJgpABKvgvenOxja2A _type: - _index:dev.ais.var.log.containers.xnateks-xnat-web-0_xnatdev_xnat-web-9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5.log _score: -

  | Jul 5, 2024 @ 20:25:44.881 | time:Jul 5, 2024 @ 20:25:44.881 stream:stdout log:2024-07-05 10:55:44,881 - antest 3.25.93.73 GET http://dev.xnat.ais.sydney.edu.au/data/search/saved?format=json&x=xhrukeryjn7 "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36" docker.container_id:9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5 kubernetes.container_name:xnat-web kubernetes.namespace_name:xnatdev kubernetes.pod_name:xnateks-xnat-web-0 kubernetes.pod_id:84e7c6d9-5a56-44a8-a1ad-56b37302fec7 kubernetes.pod_ip:10.86.15.222 kubernetes.host:ip-10-86-15-165.ap-southeast-2.compute.internal _id:beeJgpABKvgvenOxja2A _type: - _index:dev.ais.var.log.containers.xnateks-xnat-web-0_xnatdev_xnat-web-9201e009473447b9252c116eb98419c40c941e62f59f0e5f853c1d45ce3b46b5.log _score: -

Hope the above helps for you to debug this error. Thanks, An