Azure / doAzureParallel

A R package that allows users to submit parallel workloads in Azure
MIT License
107 stars 51 forks source link

connectivity problems to Azure Batch #123

Closed dirkpetersen closed 7 years ago

dirkpetersen commented 7 years ago

This worked yesterday with the same creds, is there a problem ?

setCredentials(".azure/batch-credentials.json") [1] "Your azure credentials have been set."

setVerbose(TRUE) cluster <- makeCluster("cluster.json") [1] "POST\n\n\n1166\n\napplication/json;odata=minimalmetadata\n\n\n\n\n\n\nocp-date:Tue, 26 Sep 2017 17:00:02 GMT\n/fhbatxpetersen/pools\napi-version:2017-05-01.5.0" [1] "http://fhbatxpetersen.westus2.batch.azure.com/pools" [1] "Auth String: SharedKey fhbatxpetersen:2XNA6kV9g5l/P4kMRyl6CTnXcfP9aq+"

Headers: * User-Agent: rAzureBatch/0.4.0;doAzureParallel/0.4.0 * Content-Length: 1166 * Content-Type: application/json;odata=minimalmetadata * ocp-date: Tue, 26 Sep 2017 17:00:02 GMT * Authorization: SharedKey fhbatxpetersen:2XNA6kV9g5l/P4kMRyl6CTnXc Error in curl::curl_fetch_memory(url, handle = handle) : Timeout was reached: Connection timed out after 10000 milliseconds
paselem commented 7 years ago

Hi @dirkpetersen,

First, and very importantly - you just shared your secrets in your post:

SharedKey fhbatxpetersen:2XNA6kV9g5l/P4kMRyl6CTnXc

I highly recommend going to the azure portal and regenerating your keys and updating your local secrets file.

I am not aware of any issues with Batch at the moment. Are you waiting for low-priority VMs? How many VMs are you asking for?

dirkpetersen commented 7 years ago

I did shorten my key, perhaps it would be good if the debug function did not display the entire key, perhaps replace the latter half with xxxxxxxxxxxxxxxxxx? That would allow careless users just to do a cut and paste ?

I am just using the default cluster.json from v0.4

{ "name": "myPoolName", "vmSize": "Standard_D2_v2", "maxTasksPerNode": 1, "poolSize": { "dedicatedNodes": { "min": 3, "max": 3 }, "lowPriorityNodes": { "min": 3, "max": 3 }, "autoscaleFormula": "QUEUE" }, "rPackages": { "cran": [], "github": [], "githubAuthenticationToken": "" }, "commandLine": [] }

dirkpetersen commented 7 years ago

fixed it, I had http in the batch url instead of https. What odd is that http:// worked two days ago.

paselem commented 7 years ago

That is actually very strange. Our service shouldn't accept http endpoints, only https. At a minimum we should show a better error than a Timeout.