Dynamic IDPs or MetadataProviders (i.e, from database)?

Can you give any guidance on how I could modify the configuration here so that the IDP metadata is drive by something more dynamic? For example, pulling from a database. I've tried several approaches, overriding various beans, etc,. but I seem to come up against one brick wall after another.

It seems to me like this would be a very common pattern, so I'm really surprised I haven't been able to find any good resources on this. Maybe I'm just missing something obvious?

By IDP metadata are you asking about multiple IDP metadata or just single IDP metadata? Depending on how you see the config I’d say it’s dynamic. Usually you don’t ship your code with config and you place your IDP config in your deployment to be picked up by your artifact. You change IDP config by modifying the deployment. So there’s no need to pull that from a database. For multiple IDPs it can also be done through config by iterating over multiple config values. If you just need to load the config from dB you can implement a dB properties source, there are probably some out there

@ulisesbocchio thank you for responding. What I've got is a situation in which our application will have very dynamic and frequently-changing IDP connections to maintain. This needs to be able to happen without restarting the application (and therefor going through the initialization steps).

It isn't a factor of reading from a database vs. filesystem or whatever; what i'm trying to accomplish is the ability for a support person to configure a new IDP on-the-fly. We will have hundreds or even thousands of these configurations that will be created/removed frequently throughout the day (hence, no ability to restart the application to re-init). I do personally think the situation that is driving this need for thousands of changing configurations is silly, but that's more of a business argument and not a technical one.

I've tried to define my own MetadataManager as a bean, but when I do I get an error about the KeyManger not being set. So this leads to me believe that I'm simply missing something in the way the bean should be declared or Bean-ified.

The documentation says that you can simply define a "MetadataManager" bean to override the manager behavior, but like I said that just leads to a KeyManager error.

Even something as simply as this (and this is not even a custom class at all) causes the KeyManager error:

    @Bean
    public MetadataManager metadataManager() throws MetadataProviderException {
        return new CachingMetadataManager(new ArrayList<>());
    }

So like i said, I feel like I'm missing something on how to declare these overriding beans. I don't need any details about pulling from a database or other sources from you -- I'm looking for how to effectively override the various pieces needed to make it possible for me to make it more "dynamic."

Thanks again for your time!

Hello @ledjon Did you solved your problem? I am struggling same problem. We developing system with Azure AD, our customers needs to configure their own Azure account IdP MetaData on the go. If you have any suggestions please help me!

@Jaha96 yes I did manage to solve this. Basically I wasn't able to define the MetadataManager as a bean (see my previous post) so I ended up just injecting it manually rather than depending on Bean management. I probably wont get a complete set of code examples here, so let me know if you continue to run into issues and I'll see what I can do to dig out more logic I've written here.

Here is how I'm confuring the saml part of the http security (most of this is standard from the docs):

         http.apply(saml)
            .serviceProvider()
                .metadataGenerator()
                .entityId(LocalSamlConfig.LOCAL_SAML_ENTITY_ID)
                .entityBaseURL(entityBaseUrl)
                .includeDiscoveryExtension(false)
                .bindingsSLO("redirect")
            .and()
                .sso()
                .successHandler(new SendToSuccessUrlPostAuthSuccessHandler(canvasAuthService))
            .and()
                .metadataManager(new LocalMetadataManagerAdapter(samlAuthProviderService))
                .extendedMetadata()
                .idpDiscoveryEnabled(false)
            .and()
                .keyManager()
                .privateKeyDERLocation("classpath:/saml/localhost.key.der")
                .publicKeyPEMLocation("classpath:/saml/localhost.cert")
            .and()
                .http()
                    .authorizeRequests()
                    .requestMatchers(saml.endpointsMatcher())
                    .permitAll()
        ;

The important part here is the .metadataManager(new LocalMetadataManagerAdapter(samlAuthProviderService)) which is what we're trying to solve for here. The object "samlAuthProviderService" is a Bean-managed object and it contains the logic to actually retrieve the metadata from the database, so there's not a lot that is specially about it. But here is what my LocalMetadataManagerAdapter roughly looks like:

@Slf4j
public class LocalMetadataManagerAdapter extends CachingMetadataManager {

    private final SamlAuthProviderService samlAuthProviderService;

    public LocalMetadataManagerAdapter(SamlAuthProviderService samlAuthProviderService) throws MetadataProviderException {
        super(null);
        this.samlAuthProviderService = samlAuthProviderService;
    }

    @Override
    public boolean isRefreshRequired() {
        return false;
    }

    @Override
    public EntityDescriptor getEntityDescriptor(String entityID) throws MetadataProviderException {
        // we don't really want to use our default at all, so we're going to throw an error
        // this string value is defined in the "classpath:/saml/idp-metadata.xml" file:
        // which is then referenced in application.properties as saml.sso.idp.metadata-location=classpath:/saml/idp-metadata.xml
        if("defaultidpmetadata".equals(entityID)) {
            throw exNotFound("Unable to process requests for default idp. Please select idp with ?idp=x parameter.");
        }

        EntityDescriptor staticEntity = super.getEntityDescriptor(entityID);

        if(staticEntity != null)
            return staticEntity;

        // we need to inject one, and try again:
        injectProviderMetadata(entityID);

        return super.getEntityDescriptor(entityID);
    }

    @SneakyThrows
    private void injectProviderMetadata(String entityID) {
        String xml =
            samlAuthProviderService.getMetadataForConnection(entityID)
                .orElseThrow(() -> exRuntime("Unable to find metadata for entity: " + entityID));

        addMetadataProvider(new LocalMetadataProvider(entityID, xml));

        // this will force a refresh/re-wrap of the new entity
        super.refreshMetadata();
    }
}

The important part here is the override of getEntityDescriptor() which will get called to get the metadata object at runtime. I'm also disabling refreshes by overriding isRefreshRequired() to return false. You can determine if this makes sense for your use case or not. It has been a little while since I completed this implementation so I'm a little fuzzy on the reasons for making all of the choices that I did.

The referenced LocalMetadataProvider is just a wrapper class to store/return the xml string when required:

public class LocalMetadataProvider extends AbstractReloadingMetadataProvider {

    private final String Id;
    private final String xmlData;

    public LocalMetadataProvider(String id, String xmlData) {
        this.Id = id;
        this.xmlData = xmlData;

        setParserPool(LocalBeanUtil.getBeanOrThrow(ParserPool.class));
    }

    @Override
    protected String getMetadataIdentifier() {
        return this.Id;
    }

    @Override
    protected byte[] fetchMetadata() throws MetadataProviderException {
        return xmlData.getBytes();
    }
}

There is a little bit of a hack when calling setParserPool that basically looks for the 'ParserPool' bean in the spring context and sets it. This could probably be handled more gracefully by passing that bean reference another way.

I hope this helps and sets you on the right path here!

@ledjon Thank you very much for your valuable implementation,

I did configuration according to your implementation but I encountered some problems.

Can you share me LocalBeanUtil.getBeanOrThrow() this Util class?
Did you set any configurations on application.properties
I did not figured it out how to we change entityId's?

When I run my sample app I am encountered following error. I also uploaded my sample source code to github and invited you as collaborator. I appreciate you spending your time


Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2020-08-13 13:27:39.404 ERROR 3052 --- [  restartedMain] o.s.boot.SpringApplication               : Application run failed

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'springSecurityFilterChain' defined in class path resource [org/springframework/security/config/annotation/web/configuration/WebSecurityConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.servlet.Filter]: Factory method 'springSecurityFilterChain' threw exception; nested exception is org.opensaml.util.resource.ResourceException: Wrapper resource does not exist: class path resource [idp-metadata.xml] at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:655) ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:483) ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1336) ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1176) ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:556) ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:516) ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:324) ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:226) ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:322) ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:311) ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:897) ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:879) ~[spring-context-5.2.8.RELEASE.jar:5.2.8.RELEASE] at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:551) ~[spring-context-5.2.8.RELEASE.jar:5.2.8.RELEASE] at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:143) ~[spring-boot-2.3.2.RELEASE.jar:2.3.2.RELEASE] at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:758) [spring-boot-2.3.2.RELEASE.jar:2.3.2.RELEASE] at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:750) [spring-boot-2.3.2.RELEASE.jar:2.3.2.RELEASE] at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397) [spring-boot-2.3.2.RELEASE.jar:2.3.2.RELEASE] at org.springframework.boot.SpringApplication.run(SpringApplication.java:315) [spring-boot-2.3.2.RELEASE.jar:2.3.2.RELEASE] at org.springframework.boot.SpringApplication.run(SpringApplication.java:1237) [spring-boot-2.3.2.RELEASE.jar:2.3.2.RELEASE] at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226) [spring-boot-2.3.2.RELEASE.jar:2.3.2.RELEASE] at com.riskmonster.development.SamlDynamicMetadataApplication.main(SamlDynamicMetadataApplication.java:13) [classes/:na] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_241] at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_241] at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_241] at java.lang.reflect.Method.invoke(Unknown Source) ~[na:1.8.0_241] at org.springframework.boot.devtools.restart.RestartLauncher.run(RestartLauncher.java:49) [spring-boot-devtools-2.3.2.RELEASE.jar:2.3.2.RELEASE] Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.servlet.Filter]: Factory method 'springSecurityFilterChain' threw exception; nested exception is org.opensaml.util.resource.ResourceException: Wrapper resource does not exist: class path resource [idp-metadata.xml] at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:185) ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:650) ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] ... 27 common frames omitted Caused by: org.opensaml.util.resource.ResourceException: Wrapper resource does not exist: class path resource [idp-metadata.xml] at com.github.ulisesbocchio.spring.boot.security.saml.resource.SpringResourceWrapperOpenSAMLResource.(SpringResourceWrapperOpenSAMLResource.java:24) ~[spring-boot-security-saml-1.17.jar:na] at com.github.ulisesbocchio.spring.boot.security.saml.configurer.builder.MetadataManagerConfigurer.createDefaultMetadataProvider(MetadataManagerConfigurer.java:160) ~[spring-boot-security-saml-1.17.jar:na] at com.github.ulisesbocchio.spring.boot.security.saml.configurer.builder.MetadataManagerConfigurer.configure(MetadataManagerConfigurer.java:131) ~[spring-boot-security-saml-1.17.jar:na] at com.github.ulisesbocchio.spring.boot.security.saml.configurer.builder.MetadataManagerConfigurer.configure(MetadataManagerConfigurer.java:61) ~[spring-boot-security-saml-1.17.jar:na] at org.springframework.security.config.annotation.AbstractConfiguredSecurityBuilder.configure(AbstractConfiguredSecurityBuilder.java:383) ~[spring-security-config-5.3.3.RELEASE.jar:5.3.3.RELEASE] at org.springframework.security.config.annotation.AbstractConfiguredSecurityBuilder.doBuild(AbstractConfiguredSecurityBuilder.java:329) ~[spring-security-config-5.3.3.RELEASE.jar:5.3.3.RELEASE] at org.springframework.security.config.annotation.AbstractSecurityBuilder.build(AbstractSecurityBuilder.java:41) ~[spring-security-config-5.3.3.RELEASE.jar:5.3.3.RELEASE] at com.github.ulisesbocchio.spring.boot.security.saml.bean.SAMLConfigurerBean.init(SAMLConfigurerBean.java:225) ~[spring-boot-security-saml-1.17.jar:na] at com.github.ulisesbocchio.spring.boot.security.saml.bean.SAMLConfigurerBean.init(SAMLConfigurerBean.java:137) ~[spring-boot-security-saml-1.17.jar:na] at org.springframework.security.config.annotation.AbstractConfiguredSecurityBuilder.init(AbstractConfiguredSecurityBuilder.java:370) ~[spring-security-config-5.3.3.RELEASE.jar:5.3.3.RELEASE] at org.springframework.security.config.annotation.AbstractConfiguredSecurityBuilder.doBuild(AbstractConfiguredSecurityBuilder.java:324) ~[spring-security-config-5.3.3.RELEASE.jar:5.3.3.RELEASE] at org.springframework.security.config.annotation.AbstractSecurityBuilder.build(AbstractSecurityBuilder.java:41) ~[spring-security-config-5.3.3.RELEASE.jar:5.3.3.RELEASE] at org.springframework.security.config.annotation.web.builders.WebSecurity.performBuild(WebSecurity.java:292) ~[spring-security-config-5.3.3.RELEASE.jar:5.3.3.RELEASE] at org.springframework.security.config.annotation.web.builders.WebSecurity.performBuild(WebSecurity.java:79) ~[spring-security-config-5.3.3.RELEASE.jar:5.3.3.RELEASE] at org.springframework.security.config.annotation.AbstractConfiguredSecurityBuilder.doBuild(AbstractConfiguredSecurityBuilder.java:333) ~[spring-security-config-5.3.3.RELEASE.jar:5.3.3.RELEASE] at org.springframework.security.config.annotation.AbstractSecurityBuilder.build(AbstractSecurityBuilder.java:41) ~[spring-security-config-5.3.3.RELEASE.jar:5.3.3.RELEASE] at org.springframework.security.config.annotation.web.configuration.WebSecurityConfiguration.springSecurityFilterChain(WebSecurityConfiguration.java:104) ~[spring-security-config-5.3.3.RELEASE.jar:5.3.3.RELEASE] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_241] at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_241] at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_241] at java.lang.reflect.Method.invoke(Unknown Source) ~[na:1.8.0_241] at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154) ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] ... 28 common frames omitted`

@Jaha96 here's the logic for the BeanUtil class:

@Service
public class LocalBeanUtil implements ApplicationContextAware {

    private static ApplicationContext context;

    @Override
    public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
        context = applicationContext;
    }

    public static <T> T getBeanOrThrow(Class<T> beanClass) {
        return context.getBean(beanClass);
    }
}

It is important that the above class be a @Service as this will cause spring to call the setApplicationContext at startup time (which is then stored/retrieved from a static variable). This is what I mean by a little bit hacky.

The error you're getting on startup is simply because you don't have a file named idp-metadata.xml in the resources/ folder. This can be a dummy IDP metadata file (you can google for a sample). Basically the startup logic of the saml library will not work correctly until there is some sort of idp metadata file to read on startup.

@ledjon Thanks for responding! It did worked. I will continuously upload my sample source code to above github for future readers. But I have one question how do we change or select IdP metadata. Is there any way to read following entityID from parameter or any ways to change this entityID? public EntityDescriptor getEntityDescriptor(String entityID)

@Jaha96 My memory is a little fuzzy, but I think entityID value is defined by either the default idp-metadata.xml file or the http security configuration section (based on my first post, that would be .entityId(LocalSamlConfig.LOCAL_SAML_ENTITY_ID)). That's how the static values get defined, at least. The way you and I care about (dynamic values) depends on the workflow.

Basically there are two workflows for saml authentication:

IDP-initiated -- this is where somebody is logged into their IDP and they click on some button to do a single signon login to your product. The IDP would be something like okta or onelogin.
SP-initiated -- in this case your application is starting the login workflow (which will eventually send the user over to the IDP to do the actual authentication).

In the case of #1, the user's browser will POST and xml saml assertion to your servers. This xml document will contain an entityID value, and the MetadataManager's getEntityDescriptor() will be called, passing in that entityID value. This gives us a chance to dynamically inject the "idp metadata" (an xml file or xml string) into the cache. So you're given an opporutnity to "hook" into the lookup workflow to find or create the xml string that represents that entityID from the service provider side.

In the case of #2, you would typically redirect the user to /saml/login?idp=X where X is the entityID value you want to get passed to getEntityDescriptor(). This url path (/saml/login) and query parameter (?idp=X) are both defined in the saml library configuration and can be overwritten via config or overrides if you want to change the paths/behaviors. See https://docs.spring.io/autorepo/docs/spring-security-saml/1.0.x-SNAPSHOT/reference/htmlsingle/#configuration-discovery for more info on this workflow.

Do that help with what you're trying to figure out? It is all quite complicated and I'm not even 100% sure I'm correct about what I've said above :p.

The best way to figure this stuff out is to set lots of breakpoints in your code and try IDP/SP-initiated workflows and see what gets called and in what order. Don't be afraid to debug into the saml libraries we're using here either. Most of what I discovered was done by debugging the logic from this project as well as the official spring-saml project.

ulisesbocchio / spring-boot-security-saml

Dynamic IDPs or MetadataProviders (i.e, from database)? #90