search

Azure / doAzureParallel

A R package that allows users to submit parallel workloads in Azure
MIT License
107 stars 50 forks source link

All previous versions of doAzureParallel unable to orchestrate jobs #341

Closed sorenvind closed 5 years ago

sorenvind commented 5 years ago

The merge of PR336 (https://github.com/Azure/doAzureParallel/pull/336) is the root cause of the bug I attempted to fix in https://github.com/Azure/doAzureParallel/pull/340. However, the bug fix did not succeed.

By merging PR336 to master, the file at https://raw.githubusercontent.com/Azure/doAzureParallel/master/inst/startup/cluster_setup.sh was changed. The file is referenced in old versions of doAzureParallel (see e.g. https://github.com/Azure/doAzureParallel/blob/v0.6.3/R/cluster.R#L145). Since the set up of docker was removed by this PR, all pools/jobs orchestrated using previous versions of doAzureParallel will fail until this PR is reverted or docker is installed again by the cluster_setup.sh file. It is unclear to me if the version on master currently works.

This change has broken the daily runs in our production system relying on Batch for two days now. What can I do to help fix this?

brnleehng commented 5 years ago

Hi @sorenvind

I will be reverting the cluster_setup.sh file. Once the cluster_setup.sh was using the batch-insights app installation.

342

I'll be adding test coverage for backwards compatible as a feature to prevent this situation happening again.

Thanks, Brian