Automatically add new libraries to the package index

[weshaggard]

For libraries being published out of our engineering system automatically pull them into our package index. For the track 2 libraries (ones that are in the azure namespace).

[weshaggard]

As well as detecting new client libraries we should detect new mgmt libaries as well. See https://github.com/Azure/azure-sdk/pull/1772/files#r486027538

[nickzhums]

Regarding tagging Track 2 "mgmt" libraries, currently the criteria i can think of For Java, all libraries who share the group id "com.azure.resourcemanager"

For .NET, all libraries that contains "Azure.ResourceManager" in the namespace https://www.nuget.org/packages/Azure.ResourceManager.Compute/1.0.0-preview.1

For Python, all libraries that contains "azure-mgmt-" in the package name https://pypi.org/project/azure-mgmt-compute/

[weshaggard]

For Python, all libraries that contains "azure-mgmt-" in the package name

That will include all mgmt both track 1 and track 2 libraries. @lmazuel @johanste do you guys know of any good way to identify which libraries are track 2 for both data-plane and mgmt? I want to be able to automatically add them to the "new" section of our package index but I don't know how to easily identify them. Can I look at the setup.py and see if they depend on azure-core?

@ramya-rao-a @bterlson I have a similar question for JS. What is the best way to determine if a JS package is track 2 or not? Since we reused package names we cannot rely on those.

[johanste]

Looking for a reference to azure-core would be a reasonable start (in theory, someone can include hand-written code that takes a dependency on azure-core as well, so it is not bullet-proof).

[ramya-rao-a]

There are no Track 2 JS management packages at the moment. There are no plans to add them any time soon atleast in this semester. When we do, we are not sure whether there will be a naming change or not. But, what @johanste said can apply to JS as well. You can look for @azure/core-http dependency in the package.json file which will tell you that it is a Track 2 library

[weshaggard]

Thanks @johanste @ramya-rao-a I'll start with that general approach and see how far it gets me. This is me just trying to help automate some of the manual release work so if it isn't 100% we can adjust

@johanste @lmazuel for track 2 mgmt would I be looking for azure-mgmt-core (https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/compute/azure-mgmt-compute/setup.py#L85) instead of azure-core?

@ramya-rao-a for JS wouldn't I also need to look for @azure/core-amqp for things like eventhub ? Or would something like core-tracing be a better proxy?

[johanste]

I guess there is no direct dependency on azure-core. But there should also be a master list that includes all libraries that are supposed to use the new code generator. It may be worthwhile using that list rather than parsing the results...

[lmazuel]

@weshaggard I agree that azure-core/azure-mgmt-core is the simplest way right now, though once we figure out the automation story we are working on, it will be faster to just check the version of autorest used (>v5 == track2 in the case of Python)

[nickzhums]

@weshaggard Sorry you are right, "azure-mgmt-xxx" will include both Track 1 and Track 2 for Python. Using dependency is a good idea. Thanks all !

[ramya-rao-a]

Or would something like core-tracing be a better proxy?

@weshaggard That is a good point. Yes, checking @azure/core-tracing would work

[nickzhums]

@weshaggard is this expected to be available soon ? :) we are expected to onboard a lot of RPs to track 2 by the end of this semester

[weshaggard]

@nickzhums unfortunately I've not been able to get around to this yet. If you can point me at the set of packages after you ship them I can make sure these get setup and perhaps use them for testing some logic to automatically identify them.

[nickzhums]

@weshaggard thanks, will do

[weshaggard]

We are now automatically adding new libraries to the CSV file based on the release items we use for ship tracking and that get created with the Prepare-Release scripts.