[SDK] Getting Started ("Wish List")

[lostintangent]

While we're leading with the Azure CLI as the primary experience for managing your Azure account, there are plenty of meaningful scenarios where a developer would like to automate some of this process from a Node.js script, and shelling out to a CLI is not as simple/natural as using an NPM library would be. That said, we need to ensure that the JavaScript management SDK for Azure is idiomatic to use, has a low barrier-of-entry (especially for users that aren't Azure experts), and is supported with amazing docs and samples.

The following illustrates a straw man proposal for getting started with the Azure SDK for JavaScript, with the goal of simplifying the overall E2E and reducing the amount of friction and concepts needed to onboard to Azure.

North Start (Basic Auth/Management Scenario):

const azure = require("azure");

// Assuming you have your Azure credentials stored as env vars...
const azureClient = await azure.login();
const storageClient = azureClient.createStorageClient();

// Manage your Azure account!

How do we get there?

1. Bundle the ms-rest-azure package as part of the azure NPM library, so that the azure library is completely self-contained, as opposed to requiring multiple dependencies just to get started with a hello world example.

2. Provide a new login method, which is smart enough to detect the type of authentication it will perform, based on the presence (or absence) of SP environment variables.

3. Add support for Promises to the API, as opposed to requiring the use of callbacks as the async primitive. This is a common customer request, and enables the use of async/await for users that are leveraging it (#2033)

4. Enable all of the identity/tenant/subscription arguments to be set as environment variables. This way the code is kept free from config, and can easily be run in different environments.

5. Remove the ARM infix from the create methods, and instead, as an infix to the "legacy" methods if needed. We should be leading with ARM as the "official" way for new users, without expecting them to care what it is (until they need to)

6. Remove the Management infix from the create methods, since the vast majority of the SDK is management/control-plane clients. I'd love to discuss whether we can qualify the handful of control-plane APIs instead, since I feel like it would be easier to message the SDK as a management API.

7. Determine how to keep the typings file for the SDK up to date, so that the authoring experience is awesome, functional and accurate within VS Code (#2061)

8. Update the documentation to ensure it has simple tutorials and meaningful samples for our target audience/scenarios (e.g. provisioning a MEAN app via App Service), and that the pointers to specific service APIs (e.g. web apps) take you directly to the expected content.

9. Provide support for automating Azure resources from a browser, as opposed to requiring the use of Node.js.

10. Improve performance of loading the module (#2053)

11. Fix interop with the yarn, to ensure that users can be successful regardless which NPM client they prefer (#1935)

12. Begin to GA the individual service modules, so that developers can depend on this SDK with confidence.

Further Opportunities

Everything above would lead to a great developer experience, so the following items should be interpreted as nothing more than "thought experiments", and potential areas to get customer feedback around how we could simplify the API even further:

1. Could we simplify the authentication experience even further by removing the need to call any login* methods explicitly, and instead rely on the construction of each management client to ensure that a client session has been established?

2. Could the azure module expose classes for each service as opposed to a create* factory method? That would enable importing just the APIs you want from the Azure SDK and creating them in a very idiomatic/ES6-friendly way

3. Could we remove the Client suffix from the factory/constructors entirely, so that service APIs were named simply based on their identity?

The above three items would effectively change the overall proposal to look something like the following:

// Assuming your SP credentials were
// stored in environment variables
const { Storage } = require("azure");
const client = new Storage();

// Manage your storage account!

[fiveisprime]

Here are some thoughts that I had going in my doc, some of this follows along with your intent. Also, it's worth noting that ms-rest-azure is part of the roll-up SDK module, which makes it all the more confusing. :)

The Azure roll-up SDK needs to provide much more than just a one-stop shop for all of the Node.js SDK modules. It needs to offer convenience and usability for people that may be new to using the SDK to manage their resources. There should be no reason to require dependent modules such as ms-rest-azure for required tasks such as authentication.

The individual Node.js SDK modules need to be maintained in a more proactive way, each module should be getting enough attention that the roll-up module is simply a combination of docs from each of the client modules and a wrapper that simplifies usage for multiple different services at once.

See example above

Here are some of my "how do we get there?" items

1) Reorganize the repository and create new repositories for sub-modules 1) Get all dependencies up to date (there are some errors that put some users off) 1) GA the individual modules and list their status in the README 1) simplify the AutoRest generation and merging process, ensure typings are correct 1) Correct issues with documentation and typings at an individual module level 1) Close out some PRs and issues for the roll-up SDK module 1) Do not deploy unnecessary directories

[fiveisprime]

Thinking through some use cases for the SDK at an application level and I can easily come up with quite a few that I want to capture for potential how-to guides

• Inter-service communication via service bus
• Storage management (processing and storing uploaded files)
• Storing/retrieving keys in Key Vault
• Running cron-style jobs with Scheduler
• Managing state in Azure Redis Cache
• Sending push notifications to mobile devices using Notification Hub
• Running batch process jobs (super great example of offloading work that Node.js isn't so good at)

[lostintangent]

@fiveisprime That looks awesome! There's almost like a general item to "Address quality debt" that needs to happen, and your list includes much of the details around it.

[amarzavery]

How do we get there?

1. Bundle the ms-rest-azure package as part of the azure NPM library, so that the azure library is completely self-contained, as opposed to requiring multiple dependencies just to get started with a hello world example.

Sure. We will take this up. However, majority of our customers download the required packages for individual services rather than downloading the rollup. The only metric we have is the download numbers of the rollup package. This is not indicative of active-usage. (https://www.npmjs.com/package/azure) -> ~18K/month

2. Add support for Promises to the API, as opposed to requiring the use of callbacks as the async primitive. This is a common customer request, and enables the use of async/await (#2033)

In progress https://github.com/Azure/azure-sdk-for-node/pull/2085

3. Enable all of the identity/tenant/subscription arguments to be set as environment variables. This way the code is kept free from config, and can easily be run in different environments.

This is currently documented over here.

4. Remove the ARM infix from the create methods, and instead, as an infix to the "legacy" methods if needed. We should be leading with ARM as the "official" way for new users, without expecting them to care what it is (until they need to)

I agree that this should be done. However, this will be a breaking change. If we are ok with this breaking change one time then we can go forward with this approach.

5. Determine how to keep the typings file for the SDK up to date, so that the authoring experience is awesome and accurate within VS Code (#2061)

We are working on this one. https://github.com/Azure/azure-sdk-for-node/issues/2083

6. Update the documentation to ensure it has simple tutorials and meaningful samples for our target audience/scenarios (e.g. provisioning a MEAN app via App Service), and that the pointers to specific service APIs (e.g. web apps) take you directly to the expected content.

Not sure about app service samples. However, we have samples for other important scenarios that customers can download and start executing

7. Provide support for automating Azure resources from a browser, as opposed to requiring the use of Node.js.

Awesome idea. However, this will require generating isomorphic javascript libraries. We are working on this item as well.

8. Improve performance of loading the module (#2053)

Will fix it as a part of fixing the rollup azure package

9. Fix interop with the yarn, to ensure that users can be successful regardless which NPM client they prefer (#1935)

Agreed. Need to figure this out.

[lostintangent]

@amarzavery Awesome! Thanks for the update 👍

[lostintangent]

@amarzavery FYI, I added #2 and #6 to the above proposal and would love to hear your thoughts :)

[lostintangent]

@fiveisprime @amarzavery I also added a new Further Opportunities section to track some additional random thoughts I had. They aren't necessarily high-pri, but I wanted to make sure I tracked them somewhere.

[amarzavery]

I have one question about the import statements. Does node.js v6.x and current browsers support ES6 import syntax ? import { Storage } from "azure";

[lostintangent]

Ah no, that's my bad :)

But you could require() the module and "extract" just the members you want via the distructuring syntax. The "{ Storage }" part was what I was most interesting in trying to capture.

I'll update the code samples to use Node module syntax, instead of ES6 modules syntax.

[amarzavery]

So, I did some research and came across some really good articles that provide good insights into the current scheme of things around ES6 import/export.

ES6 import/export is not supported in node version 4.x (The sdk currently supports node version 4.x as it is an active LTS version). I am not sure what should be our story moving forward.

This article and this one, provide good insights into the problems faced by the node.js team and their proposal moving forward.

node.js used CommonJS approach and is synchronous whereas browser's use the AMD approach to load modules and is async. ES6 spec around loading modules also states that module loading should be async. However, this is not possible in node.js with the current scheme of things.

Typescript/Babel/Traceur help us get there [One more reason to generate Typescript].

So for now, I think we are good with require('') statements and moving forward we shall see how to add support for ES6 style import/export.

What do you think?

[lostintangent]

@amarzavery Awesome findings! Yep, that sounds like a great plan. Thanks for looking into that.

[amarzavery]

• Was looking at the download metric for different versions of node.js per day from this site.
• This graph was made sometime in Jan 2017.
• Looking at the graph, it looks like by the end of this year 6.x will be trumping other versions.
  • This should help us determine whether we want to continue supporting 4.x or move our minimal support to 6.x?
  • This article on hackernoon provides good insights into the reasons for moving to 6.x.
  • Two big reasons that peaked my interest were:
    • Ninety-seven percent of ES6 features are now supported by Node.js v6.
    • Node.js v6 Equipped with npm v3 (which flattens the dependency tree that helps windows users avoid 256 char limitation on file paths
• node.js 7.x (current release) is scheduled to die in June 2017 which means in the later half of 2017 4.x will be even less relevant.
• Since we are in the business of auto-generating sdks it takes some time to prototype based on the new ES* features and then make the generation pipeline stable.
• By the time we release something tangible for the sdk, 7.x would be close to getting ready. Hence it makes sense to stay at the top of the edge right now and make 6.x as the minimum supported version.

Hope this helps us in making the right decision. Please let me know your thoughts.

[lostintangent]

@amarzavery Once again, thanks so much for doing this research. It's unbelievably useful to see this data! :)

Thinking from the perspective of investments in the next 2-6 months, I've definitely been thinking of v6.9+ as being the primary target, for exactly the same reasons as you mentioned: ES6 support and the fact that it's currently the active LTS version (soon to be replace by v8 in April). Node v4 goes out of "active LTS" in April as well, so it's rapidly going to be viewed as a maintenance release, that doesn't support many language features that are extremely appreciated (e.g. arrow functions).

That said, I didn't meant to imply that the Azure SDK should break compat with older versions by directly depending on newer language features and Node versions. I was just using newer language constructs such as async/await to illustrate the "north star" that could be enabled by making the specified investments (e.g. if we add Promise support, we get async/await for free for users that can use it).