garvincasimir / Elasticsearch-Azure-PAAS

Visual Studio Project which creates an Elasticsearch cluster on Microsoft Azure using worker roles
MIT License
17 stars 8 forks source link

Can this be run in web jobs? #21

Closed jlkalberer closed 8 years ago

jlkalberer commented 8 years ago

I'm pretty unfamiliar with how this actually works. I have a really low traffic site but I need ES and was hoping I could just add it via a web job.

garvincasimir commented 8 years ago

It might be possible in a continuous web job if are using a single instance in your app service plan.

jlkalberer commented 8 years ago

Ok, thanks. I tried getting this set up but I was missing some Azure SDK dlls. I'll post back here if I can get this working.

garvincasimir commented 8 years ago

I updated the code recently but have not updated the nuget package. how are you testing it?

jlkalberer commented 8 years ago

Nuget. I'll compile locally and try that tonight.

garvincasimir commented 8 years ago

The nuget version works with ES v1.4

jlkalberer commented 8 years ago

Alright, I pulled down latest but I'm having issues compiling. Looks like Worker.Common won't build because NEST was updated and you're still using an old version there?

jlkalberer commented 8 years ago

OK, I'm guessing that the webjob doesn't have appropriate permissions or something. With the nuget package I fail like this:

[10/12/2016 22:37:11 > c6c8fb: ERR ] Unhandled Exception: System.InvalidOperationException: role discovery data is unavailable
[10/12/2016 22:37:11 > c6c8fb: ERR ]    at Microsoft.WindowsAzure.ServiceRuntime.RoleEnvironment.get_CurrentRoleInstance()
[10/12/2016 22:37:11 > c6c8fb: ERR ]    at ElasticsearchWorker.Core.ElasticsearchServiceSettings.FromStorage(CloudStorageAccount account)
[10/12/2016 22:37:11 > c6c8fb: ERR ]    at Keg.ElasticSearch.Program.Main()
jlkalberer commented 8 years ago

Ok.. I see that I would need to set up the config myself instead of doing it with the helper. I'm thinking I'll just bite the bullet and create a cloud service. It will be better for deployments if I move my site under that anyways.

garvincasimir commented 8 years ago

Sorry I should have provided more detail in my response. This will not work in a Webjob without customization. And even with customization you will probably only get one instance.

  1. You need to implement your own IElasticsearchServiceSettings.cs. (AppSettingsESserviceSettings.cs perhaps?)
  2. I am assuming the JVM is already installed on these instances but we probably don't have access to the registry to discover the path (needs verification).
  3. I don't know that you can mount azure file storage so you might be stuck with local storage (needs verification)
  4. The discovery plugin and service is specifically designed to work with Web and Worker Roles so you would need to remove them. I don't know of any available api that would allow discovery for multiple instances in App Services.
jlkalberer commented 8 years ago

Yeah, makes sense. I gave up on it because if I ever needed to scale I'd have to do this all over again :)

Anyways, I'm running into the issue now about setting JAVA_PATH in the cloud emulator. Any ideas what that's about?

garvincasimir commented 8 years ago

You need to upload the java installer to your storage account and put the url in the appropriate setting

jlkalberer commented 8 years ago

I used a web url and that is downloading and starting the installer. The actual error message is complaining about setting JAVA_HOME.

On Wed, Oct 12, 2016 at 5:38 PM Garvin Casimir notifications@github.com wrote:

You need to upload the java installer to your storage account and put the url in the appropriate setting

— You are receiving this because you modified the open/close state.

Reply to this email directly, view it on GitHub https://github.com/garvincasimir/Elasticsearch-Azure-PAAS/issues/21#issuecomment-253379157, or mute the thread https://github.com/notifications/unsubscribe-auth/ABDlxFjD21zlZjbuEB5I2dQcl4eUevOfks5qzX2ZgaJpZM4KUX8i .

garvincasimir commented 8 years ago

What is the actual error message? Are you running visual studio as an administrator?

jlkalberer commented 8 years ago

Ahhh... thank you so much. I haven't worked with cloud services in years and forgot about the "run as administrator" issues.

On Wed, Oct 12, 2016 at 6:25 PM Garvin Casimir notifications@github.com wrote:

What is the actual error message? Are you running visual studio as an administrator?

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/garvincasimir/Elasticsearch-Azure-PAAS/issues/21#issuecomment-253385139, or mute the thread https://github.com/notifications/unsubscribe-auth/ABDlxKnSo8FQzi1l9XNHkwNSvGbrHYsBks5qzYh3gaJpZM4KUX8i .

garvincasimir commented 8 years ago

No problem.

garvincasimir commented 8 years ago

How did it go?

jlkalberer commented 8 years ago

Yeah, that worked. I'm still working on it with the emulator for now (trying to get a good analyzer working) but other than that there are no problems. It would be awesome if you pushed the latest package to nuget :)

garvincasimir commented 8 years ago

I could not push the package until I confirmed everything was working. I think it is safe to tag the release now. Thanks for testing it.

jlkalberer commented 8 years ago

Whoah... Wait a minute. I used the old version on nuget.

I couldn't test because this repo wouldn't build on my machine.. but that may have been an SDK issue.

On Sun, Oct 16, 2016 at 5:54 PM Garvin Casimir notifications@github.com wrote:

I could not push the package until I confirmed everything was working. I think it is safe to tag the release now. Thanks for testing it.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/garvincasimir/Elasticsearch-Azure-PAAS/issues/21#issuecomment-254089857, or mute the thread https://github.com/notifications/unsubscribe-auth/ABDlxIIMe443e0wQRJF2wxvvWrS0Q8Woks5q0sdIgaJpZM4KUX8i .

garvincasimir commented 8 years ago

Oh I guess I need to do those tests then.

jlkalberer commented 8 years ago

Yeah.. I just pulled again and couldn't get it to build. I don't know if there is an issue with building the nuget packages or what but the main dll won't build.

garvincasimir commented 8 years ago

If you updated the packages before building, the Elasticsearch .net client has breaking chances.

garvincasimir commented 7 years ago

Nuget package updated to Elasticsearch v5