search

Azure-Samples / azure-batch-samples

Azure Batch and HPC Code Samples
Other
261 stars 487 forks source link

Why do I need to manually upload all the DLLs? #265

Closed Cloudmersive closed 5 years ago

Cloudmersive commented 5 years ago

https://github.com/Azure-Samples/azure-batch-samples/blob/49b490ed7d90b2b16240dddb25b27e951b1de17b/CSharp/TopNWords/Job.cs#L82

See above - This is terrible. The system should DO THIS FOR YOU - how am I supposed to use stuff like this?

Cloudmersive commented 5 years ago

Also, I cannot believe everything is done via command line parameters. How is this even a framework? This feels like a giant hack.

Cloudmersive commented 5 years ago

The way this kind of framework is supposed to work is:

  1. You write one app
  2. You click Run
  3. It deploys itself everywhere and uses objects and event callbacks to invoke various tasks (not command line parameters!) and automatically joins the results back into an object/result
  4. Displays the result
  5. Short spin up time (not 5 minutes!)
matthchr commented 5 years ago

Hey @Cloudmersive - Batch isn't really designed with specific C# integration in mind. It doesn't understand C# dependency trees or know how to automatically upload your dependencies for you.

Also, I cannot believe everything is done via command line parameters. How is this even a framework? This feels like a giant hack.

Command line parameters are used because they are the common denominator between process execution in Windows/Linux. Batch supports running applications which may not have been authored by the person submitting the work - a canonical example is rendering, where the process being launched is the renderer (often a proprietary 3rd party app). Similarly for things like deep-learning/AI tools like tensorflow are usually configured via the command line with configuration files. Batch is targeting these sorts of HPC (or HPC-like) workflows.

What you're looking for may be something more like Azure Data Factory or Azure Functions.