Documentation for ConsentInterceptor is a bit misleading

[jvitrifork]

The documentation on ConsentInterceptor (https://hapifhir.io/hapi-fhir/docs/security/consent_interceptor.html) is a bit misleading. The documentation states:

** It can reject a resource from being disclosed to the user by examining it while calculating search results. This calculation is performed very early in the process of building search results, in order to ensure that in many cases the user is unaware that results have been removed.

If the dataset consists of 2 male patients and 1 female patient and an implementation of IConsentService filters away eg. all female patients, the bundle result set will state bundle.total as 3, but the bundle will only contain 2 entries. Hence, the user is sort of aware that something was filtered away.

Furthermore, given the dataset above, if the search is conducted with _count=1the result has been observed that the first two pages each contain one male, and the last page is left empty, which shouldn't have been there in the first place.

[jamesagnew]

I've added a test that verifies that this works as documented. If you're seeing other results, can you add to this test to demonstrate your use case?

[jvitrifork]

Full dataset

Bundle.total is equal to 3 - yet only to entries are present. I'm running on 5.0.2

males.txt

[jvitrifork]

The example implementation

 registerInterceptor(new ConsentInterceptor(new IConsentService() {
            @Override
            public ConsentOutcome startOperation(RequestDetails theRequestDetails, IConsentContextServices theContextServices) {
                return ConsentOutcome.PROCEED;
            }

            @Override
            public ConsentOutcome canSeeResource(RequestDetails theRequestDetails, IBaseResource theResource, IConsentContextServices theContextServices) {
                if(theResource.getMeta().getProfile().stream().map(IPrimitiveType::getValueAsString).anyMatch(s -> s.equals("http://ehealth.sundhed.dk/fhir/StructureDefinition/ehealth-patient")) && theRequestDetails.getRestOperationType() != RestOperationTypeEnum.CREATE)
                {
                    Patient patient = (Patient) theResource;
                    return patient.getGender() == Enumerations.AdministrativeGender.FEMALE ? ConsentOutcome.REJECT : ConsentOutcome.PROCEED;

                }
                return ConsentOutcome.AUTHORIZED;
            }

            @Override
            public ConsentOutcome willSeeResource(RequestDetails theRequestDetails, IBaseResource theResource, IConsentContextServices theContextServices) {
                if(theResource.getMeta().getProfile().stream().map(IPrimitiveType::getValueAsString).anyMatch(s -> s.equals("http://ehealth.sundhed.dk/fhir/StructureDefinition/ehealth-patient")) && theRequestDetails.getRestOperationType() != RestOperationTypeEnum.CREATE)
                {
                    Patient patient = (Patient) theResource;
                    if(patient.getGender() == Enumerations.AdministrativeGender.MALE)
                        return new ConsentOutcome(ConsentOperationStatusEnum.PROCEED, patient.setAddress(List.of()));
                    else
                        return new ConsentOutcome(ConsentOperationStatusEnum.REJECT);
                }
                return ConsentOutcome.AUTHORIZED;
            }

            @Override
            public void completeOperationSuccess(RequestDetails theRequestDetails, IConsentContextServices theContextServices) {

            }

            @Override
            public void completeOperationFailure(RequestDetails theRequestDetails, BaseServerResponseException theException, IConsentContextServices theContextServices) {

            }
        }));

[jvitrifork]

so the value of total exposes that there are more entries than the user was allowed to see

[jvitrifork]

traversing the results using http://localhost:8080/fhir/Patient?_count=1&_format=json:

{
  "resourceType": "Bundle",
  "id": "d9d11909-d1dd-4b9f-aec8-c93ebc35b3ff",
  "meta": {
    "lastUpdated": "2020-08-04T15:34:12.904+02:00"
  },
  "type": "searchset",
  "total": 3,
  "link": [ {
    "relation": "self",
    "url": "http://localhost:8080/fhir/Patient?_count=1&_format=json"
  }, {
    "relation": "next",
    "url": "http://localhost:8080/fhir?_getpages=d9d11909-d1dd-4b9f-aec8-c93ebc35b3ff&_getpagesoffset=1&_count=1&_format=json&_pretty=true&_bundletype=searchset"
  } ],
  "entry": [ {
    "fullUrl": "http://localhost:8080/fhir/Patient/101",
    "resource": {
      "resourceType": "Patient",
      "id": "101",
      "meta": {
        "extension": [ {
          "url": "http://hapifhir.io/fhir/StructureDefinition/resource-meta-source",
          "valueUri": "#F1PD9DHLjhjjLpiP"
        } ],
        "versionId": "1",
        "lastUpdated": "2020-08-04T15:27:10.476+02:00",
        "profile": [ "http://ehealth.sundhed.dk/fhir/StructureDefinition/ehealth-patient" ]
      },
      "text": {
        "status": "generated",
        "div": "<div xmlns=\"http://www.w3.org/1999/xhtml\"><div class=\"hapiHeaderText\">Torsteinn <b>LIND </b></div><table class=\"hapiPropertyTable\"><tbody><tr><td>Identifier</td><td>0506504005</td></tr><tr><td>Address</td><td><span>Nordre Ringgade 3 </span><br/><span>Frederiksberg C </span><span>DK </span></td></tr><tr><td>Date of birth</td><td><span>04 June 1950</span></td></tr></tbody></table></div>"
      },
      "identifier": [ {
        "use": "official",
        "system": "urn:oid:1.2.208.176.1.2",
        "value": "0506504005"
      } ],
      "name": [ {
        "use": "official",
        "family": "Lind",
        "given": [ "Torsteinn" ]
      } ],
      "gender": "male",
      "birthDate": "1950-06-04",
      "generalPractitioner": [ {
        "identifier": {
          "system": "urn:oid:1.2.208.176.1.4",
          "value": "134066"
        }
      } ]
    },
    "search": {
      "mode": "match"
    }
  } ]
}

{
  "resourceType": "Bundle",
  "id": "23d75247-2fb3-4444-baf6-711f71e0ef77",
  "meta": {
    "lastUpdated": "2020-08-04T15:34:04.946+02:00"
  },
  "type": "searchset",
  "total": 3,
  "link": [ {
    "relation": "self",
    "url": "http://localhost:8080/fhir?_getpages=23d75247-2fb3-4444-baf6-711f71e0ef77&_pretty=true&_count=1&_getpagesoffset=1&_bundletype=searchset&_format=json"
  }, {
    "relation": "next",
    "url": "http://localhost:8080/fhir?_getpages=23d75247-2fb3-4444-baf6-711f71e0ef77&_getpagesoffset=2&_count=1&_format=json&_pretty=true&_bundletype=searchset"
  }, {
    "relation": "previous",
    "url": "http://localhost:8080/fhir?_getpages=23d75247-2fb3-4444-baf6-711f71e0ef77&_getpagesoffset=0&_count=1&_format=json&_pretty=true&_bundletype=searchset"
  } ],
  "entry": [ {
    "fullUrl": "http://localhost:8080/fhir/Patient/102",
    "resource": {
      "resourceType": "Patient",
      "id": "102",
      "meta": {
        "extension": [ {
          "url": "http://hapifhir.io/fhir/StructureDefinition/resource-meta-source",
          "valueUri": "#XvqZGNXZYEOmgh0A"
        } ],
        "versionId": "1",
        "lastUpdated": "2020-08-04T15:27:11.682+02:00",
        "profile": [ "http://ehealth.sundhed.dk/fhir/StructureDefinition/ehealth-patient" ]
      },
      "text": {
        "status": "generated",
        "div": "<div xmlns=\"http://www.w3.org/1999/xhtml\"><div class=\"hapiHeaderText\">Torsteinn <b>LIND </b></div><table class=\"hapiPropertyTable\"><tbody><tr><td>Identifier</td><td>0506504005</td></tr><tr><td>Address</td><td><span>Nordre Ringgade 3 </span><br/><span>Frederiksberg C </span><span>DK </span></td></tr><tr><td>Date of birth</td><td><span>04 June 1950</span></td></tr></tbody></table></div>"
      },
      "identifier": [ {
        "use": "official",
        "system": "urn:oid:1.2.208.176.1.2",
        "value": "0506504005"
      } ],
      "name": [ {
        "use": "official",
        "family": "Lind",
        "given": [ "Torsteinn" ]
      } ],
      "gender": "male",
      "birthDate": "1950-06-04",
      "generalPractitioner": [ {
        "identifier": {
          "system": "urn:oid:1.2.208.176.1.4",
          "value": "134066"
        }
      } ]
    },
    "search": {
      "mode": "match"
    }
  } ]
}

{
  "resourceType": "Bundle",
  "id": "23d75247-2fb3-4444-baf6-711f71e0ef77",
  "meta": {
    "lastUpdated": "2020-08-04T15:34:04.946+02:00"
  },
  "type": "searchset",
  "total": 3,
  "link": [ {
    "relation": "self",
    "url": "http://localhost:8080/fhir?_getpages=23d75247-2fb3-4444-baf6-711f71e0ef77&_pretty=true&_count=1&_getpagesoffset=2&_bundletype=searchset&_format=json"
  } ]
}

[jamesagnew]

It's not obvious to me at first glance how that is different from the test case I committed. If you can adapt your test case to fit into that unit test that would be appreciated

[jvitrifork]

I'll see what I can do - and there has been no changes to the calculations on Bundle.total in 5.1.0 - (as I'm running on 5.0.2)?

[jamesagnew]

None that I'm aware of

On Tue, Aug 4, 2020 at 9:53 AM Jens Kristian Villadsen < notifications@github.com> wrote:

I'll see what I can do - and there has been no changes to the calculations on Bundle.total in 5.1.0 - (as I'm running on 5.0.2)?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jamesagnew/hapi-fhir/issues/2012#issuecomment-668609977, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA2N7HJZOAXROU7PM7CO3ODR7AHHLANCNFSM4PULO3TQ .

[jvitrifork]

Hmmm ... tried doing a clean checkout and IDEA complains a lot about org.shadehapi.elasticsearch missing.

On another note, my issue seems to be a lot related to what you stated here: https://github.com/jamesagnew/hapi-fhir/wiki/Consent-Service-Considerations

So

Block/filter Bundle.total and avoid returning search bundles that are obviously missing entries

-was an issue a year ago?

[jvitrifork]

maven is all dandy - IDEA not so much

[jvitrifork]

[tadgh]

Hey, sorry about this. Here's a workaround to get intellij working:

1. Right-click on the hapi-fhir-elasticsearch-6 module and click:Maven -> Ignore Projects
2. When prompted whether to remove the module from the project as well, click yes.
3. Right-click on the top-level hapi-fhir project and click: Maven -> reimport

Any previous build errors relating to the org.shadehapi.elasticsearch should now disappear.

[jvitrifork]

@tadgh well - that made IDEA stop complaining ... Now I'm left with unit test errors:

`

java.lang.IllegalStateException: Failed to load ApplicationContext

at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:132)
at org.springframework.test.context.support.DefaultTestContext.getApplicationContext(DefaultTestContext.java:123)
at org.springframework.test.context.support.DependencyInjectionTestExecutionListener.injectDependencies(DependencyInjectionTestExecutionListener.java:118)
at org.springframework.test.context.support.DependencyInjectionTestExecutionListener.prepareTestInstance(DependencyInjectionTestExecutionListener.java:83)
at org.springframework.test.context.TestContextManager.prepareTestInstance(TestContextManager.java:244)
at org.springframework.test.context.junit.jupiter.SpringExtension.postProcessTestInstance(SpringExtension.java:98)
at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$invokeTestInstancePostProcessors$5(ClassBasedTestDescriptor.java:341)
at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.executeAndMaskThrowable(ClassBasedTestDescriptor.java:346)
at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$invokeTestInstancePostProcessors$6(ClassBasedTestDescriptor.java:341)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.forEachRemaining(StreamSpliterators.java:312)
at java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:735)
at java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:734)
at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:658)
at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.invokeTestInstancePostProcessors(ClassBasedTestDescriptor.java:340)
at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.instantiateAndPostProcessTestInstance(ClassBasedTestDescriptor.java:263)
at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$testInstancesProvider$2(ClassBasedTestDescriptor.java:256)
at java.base/java.util.Optional.orElseGet(Optional.java:369)
at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$testInstancesProvider$3(ClassBasedTestDescriptor.java:255)
at org.junit.jupiter.engine.execution.TestInstancesProvider.getTestInstances(TestInstancesProvider.java:29)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$prepare$0(TestMethodTestDescriptor.java:108)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.prepare(TestMethodTestDescriptor.java:107)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.prepare(TestMethodTestDescriptor.java:71)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$prepare$1(NodeTestTask.java:107)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.prepare(NodeTestTask.java:107)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:75)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:248)
at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$5(DefaultLauncher.java:211)
at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:226)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:199)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:132)
at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:71)
at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:220)
at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:53)

Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'entityManagerFactory' defined in ca.uhn.fhir.jpa.config.TestR4ConfigWithElasticsearchClient: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean]: Factory method 'entityManagerFactory' threw exception; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jpaProperties' defined in ca.uhn.fhir.jpa.config.TestR4ConfigWithElasticsearchClient: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [java.util.Properties]: Factory method 'jpaProperties' threw exception; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'embeddedElasticSearch' defined in ca.uhn.fhir.jpa.config.TestR4ConfigWithElasticsearchClient: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [pl.allegro.tech.embeddedelasticsearch.EmbeddedElastic]: Factory method 'embeddedElasticSearch' threw exception; nested exception is pl.allegro.tech.embeddedelasticsearch.EmbeddedElasticsearchStartupException: Failed to start elasticsearch. Check previous logs for details at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:656) at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:484) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1338) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1177) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:557) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:517) at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:323) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:321) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1108) at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:868) at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:550) at org.springframework.test.context.support.AbstractGenericContextLoader.loadContext(AbstractGenericContextLoader.java:128) at org.springframework.test.context.support.AbstractGenericContextLoader.loadContext(AbstractGenericContextLoader.java:60) at org.springframework.test.context.support.AbstractDelegatingSmartContextLoader.delegateLoading(AbstractDelegatingSmartContextLoader.java:275) at org.springframework.test.context.support.AbstractDelegatingSmartContextLoader.loadContext(AbstractDelegatingSmartContextLoader.java:243) at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContextInternal(DefaultCacheAwareContextLoaderDelegate.java:99) at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:124) ... 63 more Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean]: Factory method 'entityManagerFactory' threw exception; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jpaProperties' defined in ca.uhn.fhir.jpa.config.TestR4ConfigWithElasticsearchClient: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [java.util.Properties]: Factory method 'jpaProperties' threw exception; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'embeddedElasticSearch' defined in ca.uhn.fhir.jpa.config.TestR4ConfigWithElasticsearchClient: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [pl.allegro.tech.embeddedelasticsearch.EmbeddedElastic]: Factory method 'embeddedElasticSearch' threw exception; nested exception is pl.allegro.tech.embeddedelasticsearch.EmbeddedElasticsearchStartupException: Failed to start elasticsearch. Check previous logs for details at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:185) at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:651) ... 81 more Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jpaProperties' defined in ca.uhn.fhir.jpa.config.TestR4ConfigWithElasticsearchClient: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [java.util.Properties]: Factory method 'jpaProperties' threw exception; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'embeddedElasticSearch' defined in ca.uhn.fhir.jpa.config.TestR4ConfigWithElasticsearchClient: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [pl.allegro.tech.embeddedelasticsearch.EmbeddedElastic]: Factory method 'embeddedElasticSearch' threw exception; nested exception is pl.allegro.tech.embeddedelasticsearch.EmbeddedElasticsearchStartupException: Failed to start elasticsearch. Check previous logs for details at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:656) at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:636) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1338) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1177) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:557) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:517) at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:323) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:321) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.resolveBeanReference(ConfigurationClassEnhancer.java:394) at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:366) at ca.uhn.fhir.jpa.config.TestR4ConfigWithElasticsearchClient$$EnhancerBySpringCGLIB$$88830f72.jpaProperties() at ca.uhn.fhir.jpa.config.TestR4Config.entityManagerFactory(TestR4Config.java:152) at ca.uhn.fhir.jpa.config.TestR4ConfigWithElasticsearchClient$$EnhancerBySpringCGLIB$$88830f72.CGLIB$entityManagerFactory$8() at ca.uhn.fhir.jpa.config.TestR4ConfigWithElasticsearchClient$$EnhancerBySpringCGLIB$$88830f72$$FastClassBySpringCGLIB$$bb157f92.invoke() at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244) at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:363) at ca.uhn.fhir.jpa.config.TestR4ConfigWithElasticsearchClient$$EnhancerBySpringCGLIB$$88830f72.entityManagerFactory() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154) ... 82 more Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [java.util.Properties]: Factory method 'jpaProperties' threw exception; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'embeddedElasticSearch' defined in ca.uhn.fhir.jpa.config.TestR4ConfigWithElasticsearchClient: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [pl.allegro.tech.embeddedelasticsearch.EmbeddedElastic]: Factory method 'embeddedElasticSearch' threw exception; nested exception is pl.allegro.tech.embeddedelasticsearch.EmbeddedElasticsearchStartupException: Failed to start elasticsearch. Check previous logs for details at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:185) at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:651) ... 105 more Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'embeddedElasticSearch' defined in ca.uhn.fhir.jpa.config.TestR4ConfigWithElasticsearchClient: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [pl.allegro.tech.embeddedelasticsearch.EmbeddedElastic]: Factory method 'embeddedElasticSearch' threw exception; nested exception is pl.allegro.tech.embeddedelasticsearch.EmbeddedElasticsearchStartupException: Failed to start elasticsearch. Check previous logs for details at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:656) at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:484) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1338) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1177) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:557) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:517) at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:323) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:321) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.resolveBeanReference(ConfigurationClassEnhancer.java:394) at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:366) at ca.uhn.fhir.jpa.config.TestR4ConfigWithElasticsearchClient$$EnhancerBySpringCGLIB$$88830f72.embeddedElasticSearch() at ca.uhn.fhir.jpa.config.TestR4ConfigWithElasticSearch.jpaProperties(TestR4ConfigWithElasticSearch.java:36) at ca.uhn.fhir.jpa.config.TestR4ConfigWithElasticsearchClient$$EnhancerBySpringCGLIB$$88830f72.CGLIB$jpaProperties$3() at ca.uhn.fhir.jpa.config.TestR4ConfigWithElasticsearchClient$$EnhancerBySpringCGLIB$$88830f72$$FastClassBySpringCGLIB$$bb157f92.invoke() at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244) at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:363) at ca.uhn.fhir.jpa.config.TestR4ConfigWithElasticsearchClient$$EnhancerBySpringCGLIB$$88830f72.jpaProperties() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154) ... 106 more Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [pl.allegro.tech.embeddedelasticsearch.EmbeddedElastic]: Factory method 'embeddedElasticSearch' threw exception; nested exception is pl.allegro.tech.embeddedelasticsearch.EmbeddedElasticsearchStartupException: Failed to start elasticsearch. Check previous logs for details at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:185) at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:651) ... 129 more Caused by: pl.allegro.tech.embeddedelasticsearch.EmbeddedElasticsearchStartupException: Failed to start elasticsearch. Check previous logs for details at pl.allegro.tech.embeddedelasticsearch.ElasticServer.waitForElasticToStart(ElasticServer.java:127) at pl.allegro.tech.embeddedelasticsearch.ElasticServer.start(ElasticServer.java:50) at pl.allegro.tech.embeddedelasticsearch.EmbeddedElastic.startElastic(EmbeddedElastic.java:82) at pl.allegro.tech.embeddedelasticsearch.EmbeddedElastic.start(EmbeddedElastic.java:63) at ca.uhn.fhir.jpa.config.TestR4ConfigWithElasticSearch.embeddedElasticSearch(TestR4ConfigWithElasticSearch.java:64) at ca.uhn.fhir.jpa.config.TestR4ConfigWithElasticsearchClient$$EnhancerBySpringCGLIB$$88830f72.CGLIB$embeddedElasticSearch$4() at ca.uhn.fhir.jpa.config.TestR4ConfigWithElasticsearchClient$$EnhancerBySpringCGLIB$$88830f72$$FastClassBySpringCGLIB$$bb157f92.invoke() at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244) at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:363) at ca.uhn.fhir.jpa.config.TestR4ConfigWithElasticsearchClient$$EnhancerBySpringCGLIB$$88830f72.embeddedElasticSearch() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154) ... 130 more

`

[jvitrifork]

for hapi-fhir-jpaserver-base that is

[tadgh]

Are you running a new test, or just the existing test suite that uses that Config file? If the latter, the tests are working for me locally. If the former, is it possible there's something higher in the logs showing why ES is failing to start?

[jvitrifork]

error was on my side .... wait ... no it wasnt

[jvitrifork]

I just right clicked on the java folder under the test folder and choose run 'All tests'. The first 68 passes. It starts failing from the FhirResourceDaoR4SearchLastNAsyncIT test cases

[tadgh]

Trying to replicate.. standby. Just to be sure though, you haven't modified any code here, you're just trying to run the raw tests in master?

[tadgh]

I do get some ES failures when trying to run the full suite, but these don't manifest in maven, leading me to think its the way that IJ is forking the tests. In any case, you should be OK if you just run the tests in the area you actually care about. Sorry about that!

[jvitrifork]

Haven't modified as much as a bit

[jvitrifork]

I'll just focus on the Consent stuff that I planned to work on then. Thx for the help so far ;)

[jvitrifork]

@jamesagnew here you are https://github.com/jamesagnew/hapi-fhir/pull/2018

[jamesagnew]

I see the issue- You have AUTHORIZED as the default status where other tests in there have PROCEED as the default, and that's making it skip cleaning the Bundle itself (which it shouldn't).

Fix coming up.

[jamesagnew]

Fix has been merged. Thanks again for reporting and for the test case!

[gjergjsheldija]

hi everybody, sorry to open this thread again. In the documentation about the ConsentInterceptor it states then "... and create audit trail events.". Since the total was removed with this ticket, all those who were using this interceptor to implement the AuditEvent, will have wrong results in their queries. A possible solution could be to replace the total with real total, with something like :

if (theResource.getResponseResource() instanceof IBaseBundle) {
                BundleUtil.setTotal(theRequestDetails.getFhirContext(), (IBaseBundle) theResource.getResponseResource(), ((Bundle) (((IBaseBundle) theResource.getResponseResource()))).getEntry().size());
}

but that would cause the issues that Bundle needs hapi-fhir-structure-dstu2 which will introduce a circular reference. So not an easy solution. I can gladly create a PR for that, but I need some guidance about the circular reference.

Best Gjergj

[psavva]

And a year onwards, I'm also sorry to open this thread again.

We are facing an issue where we have now implemented our consentInterceptor, and our implementation makes use of _total, in order to be able to do paging for search results.

Without being able to bring back the totals, our current implementation needs to be modified to allow us to get total somehow.

The question is how to retrieve total filtered records when the ConsentInterceptor is implemented?

@jamesagnew @jvitrifork @gjergjsheldija Any suggestions?

[michaelabuckley]

The standard way to do paging is to fetch the next page until there are no more next links. The current code does not support reporting a total when a consent interceptor is active.