DreamFactory custom headers

[jrounds]

Hi,

Any support for custom headers in the R package Elastic?

I am working with an elasticsearch db that is behind a DreamFactory wall. As such if I query "by-hand" I have to supply a custom header with an API key and session key. E.G.:

header = list()
header[["X-DreamFactory-Api-Key"]] = "<snip>2fac261809e3d89f"
header$session_token = content(r)$session_token  # acquired from a REST API
r = POST("https:/fake.url.com/api/v2/sentiment_emotion_analytics",do.call(add_headers, header), body=body,encode="json")
content(r)

and content will have a filter result based on the JSON of body which is standard elasticsearch. This "sentiment_emotion_analytics" maps straight to an elasticsearch index on an intranet at port 9200. We just need it behind DreamFactory for security reasons.

Is there anyway to add a header to elastic::connect that will get utilized in all the underlying calls?

[jrounds]

Well examining the code I found out how to pass header to the POST in the R package Elastic, but the problem is the connect(). Since connect expects to get a connection to the "front-end" of a elasticsearch and not just an index URL it won't connect.

[sckott]

Thanks for your message. I'll have a look.

So the base URL in your case is https:/fake.url.com/api/v2/sentiment_emotion_analytics ? And that maps directly to ES?

You should be able to pass in any auth stuff (headers/session tokens/etc.) via the ... in the functions.

[jrounds]

Naw it is worse than that (and I don't blame you if it is so far out of what you intended it can't be addressed)

The base url is https:/fake.url.com/api/v2/sentiment_emotion_analytics but that maps directly via DreamFactory Proxy to http://<fake cloud intranet url for elasticsearch>:9200/sentiment_emotion_analytics/_search

Basically the whole elastic::connect idea is a little screwed up because I can't tell it a base url for the elastic search as it is behind a Proxy server and not exposed.

On Fri, Apr 8, 2016 at 12:52 PM, Scott Chamberlain <notifications@github.com

wrote:

Thanks for your message. I'll have a look.

So the base URL in your case is https:/ fake.url.com/api/v2/sentiment_emotion_analytics ? And that maps directly to ES?

— You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub https://github.com/ropensci/elastic/issues/128#issuecomment-207576428

[jrounds]

I got a message out to a partner to see if we can expose the 9200 head to GET so that connect works, and after that just re-arranging URLs might make your package work for me again.

On Fri, Apr 8, 2016 at 2:07 PM, jeremiah rounds roundsjeremiah@gmail.com wrote:

Naw it is worse than that (and I don't blame you if it is so far out of what you intended it can't be addressed)

The base url is https:/fake.url.com/api/v2/sentiment_emotion_analytics but that maps directly via DreamFactory Proxy to http://<fake cloud intranet url for elasticsearch>:9200/sentiment_emotion_analytics/_search

Basically the whole elastic::connect idea is a little screwed up because I can't tell it a base url for the elastic search as it is behind a Proxy server and not exposed.

On Fri, Apr 8, 2016 at 12:52 PM, Scott Chamberlain < notifications@github.com> wrote:

Thanks for your message. I'll have a look.

So the base URL in your case is https:/ fake.url.com/api/v2/sentiment_emotion_analytics ? And that maps directly to ES?

— You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub https://github.com/ropensci/elastic/issues/128#issuecomment-207576428

[sckott]

I will try to get this sorted - it's definitely not out of the question - I'll have a look at DreamFactory to see if I can get a test setup going to test this out

[sckott]

@jrounds any tutorial on setting up elasticsearch with dreamfactory. All I could find was this http://community.dreamfactory.com/t/integrating-elasticsearch/680

[jrounds]

I am not sure. I hate to bother you with a non-standard use case, but sort of a generalization of this is what if you want to use R package elastic for a RESTful (in the sense of connectionless) API directly to a _search URL of an index? It isn't so mcuh about the dreamfactory as it is that idiom. Turns out I started the conversation with "how to I get a header" for a connection to elasticsearch into the POST, but that wasn't the real biting issue. I asked the guy maintaining this and he said he doesn't want to expose the root URL to the internet.

On Fri, Apr 8, 2016 at 2:43 PM, Scott Chamberlain notifications@github.com wrote:

@jrounds https://github.com/jrounds any tutorial on setting up elasticsearch with dreamfactory. All I could find was this http://community.dreamfactory.com/t/integrating-elasticsearch/680

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/ropensci/elastic/issues/128#issuecomment-207619556

[jrounds]

BTW I been poking around on DreamFactory it should be fairly easy to do unless the admin customized stuff, but I certainly can't imagine you would do this unless you have some interest:

You set up : 1) Service -- This is map between urls Type: "Remote Web Service" Go to "Config" set the base url to ElasticSearch URL (index or head or whatever)

2) You setup a role. After naming it go to Access. Add a new service. Add an access type. Need GET and POST at least and make it for the Service you set up in 1. 3) Set up a User. Give it access to the role you created in 2. 4) Set up an App. Give it access to the role you created in 2.

To connect by hand you first need to handshake the user with R code:

body = list() body[["X-DreamFactory-Api-Key"]] = "FAKEf2fac261809e3d89f" body$email = "fake_user@fake.com" body$password = "fake_password" body$duration = 0 r = POST("fake_url.com/api/v2/user/session", body = body, encode="json") content(r)

r now has the ability to complete the connection, and actually that is the snippet at the top (that is all there is). The url in my very first snippet is the serivce set up in (1) and dreamfactory proxies it to the ElasticSearch cluster and caches.

On Fri, Apr 8, 2016 at 3:15 PM, jeremiah rounds roundsjeremiah@gmail.com wrote:

I am not sure. I hate to bother you with a non-standard use case, but sort of a generalization of this is what if you want to use R package elastic for a RESTful (in the sense of connectionless) API directly to a _search URL of an index? It isn't so mcuh about the dreamfactory as it is that idiom. Turns out I started the conversation with "how to I get a header" for a connection to elasticsearch into the POST, but that wasn't the real biting issue. I asked the guy maintaining this and he said he doesn't want to expose the root URL to the internet.

On Fri, Apr 8, 2016 at 2:43 PM, Scott Chamberlain < notifications@github.com> wrote:

@jrounds https://github.com/jrounds any tutorial on setting up elasticsearch with dreamfactory. All I could find was this http://community.dreamfactory.com/t/integrating-elasticsearch/680

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/ropensci/elastic/issues/128#issuecomment-207619556

[sckott]

I do want to be able to test this - so I do need to use Dreamfactory or something similar - thanks for the instructions

Opened a separate issue for headers #129

[sckott]

@jrounds I've added ability to set headers in connect() for all requests - let me know if you have any problems with that

still working on the actual issue here

[sckott]

closing - do reopen if you're still having issues