Source Filtering

[dclemenzi]

The following setting:

ElasticDataStoreFactory.SOURCE_FILTERING_ENABLED

Enables source filtering but it appears to add all of the attributes which defeats the purpose.

        if (dataStore.isSourceFilteringEnabled()) {
            // add source includes
            setSourceIncludes(searchRequest);
        }
...

    private void setSourceIncludes(final ElasticRequest searchRequest) throws IOException {
        final ElasticDataStore dataStore = getDataStore();
        final List<ElasticAttribute> attributes = dataStore.getElasticAttributes(entry.getName());
        for (final ElasticAttribute attribute : attributes) {
            if (attribute.isUse() && attribute.isStored()) {
                searchRequest.addField(attribute.getName());
            } else if (attribute.isUse()) {
                searchRequest.addSourceInclude(attribute.getName());
            }
        }
    }

Is there a currently a means to only include attributes specified in query.getProperties().

[dclemenzi]

If not I am happy to add it. Wanted to make sure I am not missing something first.

[dclemenzi]

Don't want to break what is already there so I added another feature flag to do what we need for bandwidth performance.

        if (dataStore.isSourceFilteringEnabled()) {
            if (dataStore.isSourceFilteringProperties()) {
                for (String property : query.getPropertyNames()) {
                    searchRequest.addSourceInclude(property);
                }
            } else {
                // add source includes
                setSourceIncludes(searchRequest);
            }
        }

If there is another means to enable this without this code change please let us know. Thanks.

[sjudeng]

No I think the original implementation is wrong to always include all fields from the feature type. When source filtering is enabled and query.getProperties() != Query.ALL_PROPERTIES then it should limit to attributes in query.getProperties() as you indicate. Good catch.

Can you implement this under the existing isSourceFilteringEnabled configuration (no need to add another config)?

[dclemenzi]

My concern would be breakage with existing code that is expecting to get all fields back. Since your good with it I will move the implementation under the existing feature flag and submit a pull request.

Have run into a few things but overall this plugin has saved us a ton of work and is very well done. Thank you so much for all your effort and quick responses!!!

[sjudeng]

Fixed in #76