node and client bean not created in a java-based spring-configuration

[bsadeh]

using an xml-based application-context works just fine. however, when creating a java-based application-context (via the @Configuration annotation), the esNode and esClient are not initialized correctly; they are null. reason being is that the afterPropertiesSet is not being called yet.

the hack around it for now, is to invoke factory.afterPropertiesSet() directly before factory.getObject(). (but that does not feel right)

[dadoonet]

Hi Benny,

I just wrote a small test case to start to play with @Configuration annotation. Have a look at the ConfigurationTest test case and the AppConfig class that start the Spring context.

I added also a small annotation-context.xml file to define the AppConfig Bean :

<bean class="fr.pilato.spring.elasticsearch.annotation.AppConfig"/>

Let me know if it helps Cheers

[bsadeh]

hi David, thanks for the effort!

I looked at the code and a few things came to mind:

1. in annotation-context.xml, you don't need the following: because < context:component-scan base-package= "fr.pilato.spring.elasticsearch.annotation" /> is already doing it for you. furthermore, AppConfig is not a bean; it's actually a full ApplicationContext on its own.
2. in order to take full advantage of the index configuration facilities you are providing via the factory, the following is a more complete solution: @Bean public Client esClient() throws Exception { ElasticsearchClientFactoryBean factory = new ElasticsearchClientFactoryBean(); factory.setNode(esNode()); factory.setClasspathRoot(elasticsearchClasspathRoot); factory.setAutoscan(elasticsearchAutoscan); factory.setForceTemplate(elasticsearchForceTemplate); factory.setForceMapping(elasticsearchForceMapping); factory.setMergeMapping(elasticsearchMergeMapping); if (!elasticsearchTemplates.isEmpty()) factory.setTemplates(elasticsearchTemplates.split(",")); if (!elasticsearchMappings.isEmpty()) factory.setMappings(elasticsearchMappings.split(",")); if (!elasticsearchAliases.isEmpty()) factory.setAliases(elasticsearchAliases.split(",")); // factory.readIndexSettings("???"); factory.afterPropertiesSet(); return factory.getObject(); } (notice the // factory.readIndexSettings("???"); not sure what to do there ...)
3. I couldn't get the transport client using the same method as above
4. I was worried about using the afterPropertiesSet framework method directly. I thought spring should have somehow call the method transparently (but couldn't figure where/how). what do you think.

let me know if I can explain any of this better. best, Benny

On Tue, Jun 5, 2012 at 2:06 PM, David Pilato < reply@reply.github.com

wrote:

Hi Benny,

I just wrote a small test case to start to play with @Configuration annotation. Have a look at the ConfigurationTest test case and the AppConfig class that start the Spring context.

I added also a small annotation-context.xml file to define the AppConfig Bean :

<bean class="fr.pilato.spring.elasticsearch.annotation.AppConfig"/>

Let me know if it helps Cheers

---

Reply to this email directly or view it on GitHub:

https://github.com/dadoonet/spring-elasticsearch/issues/8#issuecomment-6136724

[dadoonet]

in annotation-context.xml, you don't need the following: because ...

Yes. You're right... But it was late last night and I was feeling asleep ;-)

in order to take full advantage of the index configuration facilities you are providing via the factory, the following is a more complete solution:

Yes. I have seen that in the code you sent me, but I was trying to start with very simple configuration.

I couldn't get the transport client using the same method as above

Ok. I will work on it to see what's going on.

I was worried about using the afterPropertiesSet framework method directly. I thought spring should have somehow call the method transparently (but couldn't figure where/how). what do you think.

So do I. But I don't see by now another workaround.

I will let you know in a few days.

[bsadeh]

np David. I hope this helps ;-)

I couldn't get the transport client using the same method as above

Ok. I will work on it to see what's going on.

I'm actually at quite a loss with trying top configure the transport client to work in a production environment where the client is to connect to cluster residing at ec2 (and the client can be in or off ec2, or in ec2 but on a different zone).

can you please provide me an example of how to configure such a transport client? keep in mind that the jvm for that client should not have any node configured in it.

David, I would appreciate a quick response as I am trying to push my lovely new ES implementation to production. many thanks, Benny

[dadoonet]

Just a quick answer: You should try to add the elasticsearch-cloud-aws (https://github.com/elasticsearch/elasticsearch-cloud-aws) in your pom.xml file. Something like this:

<dependency>
  <groupId>org.elasticsearch</groupId>
  <artifactId>elasticsearch-cloud-aws</artifactId>
  <version>1.6.0</version>
</dependency>

Then, add a /es.properties file in your classpath.

cloud.aws.access_key=AKVAIQBF2RECL7FJWGJQ
cloud.aws.secret_key=vExyMThREXeRMm/b/LRzEB8jWwvzQeXgjqMX+6br

discovery.type=ec2

It could (should) work ;-)

HTH David.

[bsadeh]

that was super quick DAvid, thanks!

I'll try and report ... ;-)

On Wed, Jun 6, 2012 at 12:18 PM, David Pilato < reply@reply.github.com

wrote:

Just a quick answer: You should try to add the elasticsearch-cloud-aws ( https://github.com/elasticsearch/elasticsearch-cloud-aws) in your pom.xml file. Something like this:

<dependency>
 <groupId>org.elasticsearch</groupId>
 <artifactId>elasticsearch-cloud-aws</artifactId>
 <version>1.6.0</version>
</dependency>

Then, add a /es.properties file in your classpath.

cloud.aws.access_key=AKVAIQBF2RECL7FJWGJQ
cloud.aws.secret_key=vExyMThREXeRMm/b/LRzEB8jWwvzQeXgjqMX+6br

discovery.type=ec2

It could (should) work ;-)

HTH David.

---

Reply to this email directly or view it on GitHub:

https://github.com/dadoonet/spring-elasticsearch/issues/8#issuecomment-6159915

[bsadeh]

hi David, I think you misunderstood me: I was asking about configuring the transport client only (my ec2 cluster is up and running fine).

so I think in the settings file for the client I need: cluster.name=some.cluster.name client.transport.sniff=true

anything else? like: discovery.zen.ping.multicast.enabled=?

if you have a real-life example, it'll be awesome!

also, I resorted to using your elasticsearch-spring module as originally intended, via xml configuration, and found a way to sneak it in. however, although it looks from the code that a transport client can be configured just like the node client, I failed to accomplish doing the same as with a node client. so the following example fails to instantiate the bean for me:

<bean id="esClient" class="fr.pilato.spring.elasticsearch.ElasticsearchTransportClientFactoryBean">

  <property name="autoscan" value="true" />
  <property name="classpathRoot" value="/path/to/elasticsearch/config"

/>

  <property name="forceMapping" value="true" />
  <property name="mergeMapping" value="true" />
  <property name="mergeSettings" value="true" />
  <property name="templates">
     <list>
        <value>some_template</value>
     </list>
  </property>
  <property name="mappings">
     <list>
        <value>existing.index/type.with.mapping</value>
        <value>non.existing.index/type.without.mapping</value>
     </list>
  </property>
  <property name="aliases">
     <list>
        <value>the.great.alias:type.with.mapping</value>
        <value>the.great.alias:type.without.mapping</value>
     </list>
  </property>

am I missing something? I would love this to work, as I think it is one of the best features of your module,! allowing to push mappings etc. to the cluster is just so fantastic ... if only I could get it to work ...

On Wed, Jun 6, 2012 at 12:18 PM, David Pilato < reply@reply.github.com

wrote:

Just a quick answer: You should try to add the elasticsearch-cloud-aws ( https://github.com/elasticsearch/elasticsearch-cloud-aws) in your pom.xml file. Something like this:

<dependency>
 <groupId>org.elasticsearch</groupId>
 <artifactId>elasticsearch-cloud-aws</artifactId>
 <version>1.6.0</version>
</dependency>

Then, add a /es.properties file in your classpath.

cloud.aws.access_key=AKVAIQBF2RECL7FJWGJQ
cloud.aws.secret_key=vExyMThREXeRMm/b/LRzEB8jWwvzQeXgjqMX+6br

discovery.type=ec2

It could (should) work ;-)

HTH David.

---

Reply to this email directly or view it on GitHub:

https://github.com/dadoonet/spring-elasticsearch/issues/8#issuecomment-6159915

[dadoonet]

also, I resorted to using your elasticsearch-spring module as originally intended, via xml configuration, and found a way to sneak it in. however, although it looks from the code that a transport client can be configured just like the node client, I failed to accomplish doing the same as with a node client. so the following example fails to instantiate the bean for me:

Any stacktrace ?

BTW, for the AWS problem, if your AWS server hostname is : myawsserver.mydomain.com and if you have opened 9300 port (see your AWS network settings), then, you should be able to connect to it using the TransportClient.

<elasticsearch:client 
  id="testTransportClient" 
  esNodes="myawsserver.mydomain.com:9300"
  settingsFile="a.properties.file" />

or

<bean id="esClient"
  class="fr.pilato.spring.elasticsearch.ElasticsearchTransportClientFactoryBean">
      <property name="settingsFile" value="a.properties.file" />
      <property name="esNodes" value="myawsserver.mydomain.com:9300" />
</bean>

Does it help ?

[bsadeh]

On Mon, Jun 11, 2012 at 1:25 AM, David Pilato < reply@reply.github.com

wrote:

BTW, for the AWS problem, if your AWS server hostname is : myawsserver.mydomain.com and if you have opened 9300 port (see your AWS network settings), then, you should be able to connect to it using the TransportClient.

<elasticsearch:client
 id="testTransportClient"
 esNodes="myawsserver.mydomain.com:9300"
 settingsFile="a.properties.file" />

or

<bean id="esClient"

 class="fr.pilato.spring.elasticsearch.ElasticsearchTransportClientFactoryBean">
     <property name="settingsFile" value="a.properties.file" />
      <property name="esNodes" value="myawsserver.mydomain.com:9300" />
</bean>

Does it help ?

yes, I'm actually using: <elasticsearch:client id="esClient" settingsFile="elasticsearch.properties" esNodes="${elasticsearch.host}:9300,${elasticsearch.host}:9301" />

as in the past 9301 seemed to have the one needed to connect. and beside the cluster name, I also add: client.transport.sniff=true to the settingsFile of the transport client.

[dadoonet]

If I understood well, I can close this issue. If not, feel free to reopen it and add more information.

[bsadeh]

well, I just gave up on using it via the a @Configuration class. can't tell you if it is resolved or not ;-)

On Mon, Jun 11, 2012 at 12:04 PM, David Pilato < reply@reply.github.com

wrote:

If I understood well, I can close this issue. If not, feel free to reopen it and add more information.

---

Reply to this email directly or view it on GitHub:

https://github.com/dadoonet/spring-elasticsearch/issues/8#issuecomment-6253017

[marek-obuchowicz]

Did anyone involved in this issue figure out how to connect to Elasticsearch cluster in AWS (using spring-data elasticsearch) and NOT specifying esNodes? If we specify the nodes, we don't need the cloud-aws plugin at all...

[dadoonet]

@marek-obuchowicz This project is not the spring-data one. Are you sure you are commenting in the right repo?

[marek-obuchowicz]

@dadoonet yes, i am aware of that, however it was the only place in the web, where I found someone touching the topic of using spring, elasticsearch TransportClient and cloud-aws. I have been wondering if you can have any tips for my question - if it's possible to use spring+elasticsearch with cloud-aws node discovery, not specifying esNodes (which is, in end, common with spring-data - the client is initialized in the same way)