Asset location updater not functional

[mmitche]

It looks like the asset location updater is not functional right now. It should update package locations if they show up on nuget.org, and clean old versions from existing feeds, but this is not happening.

[mmitche]

@premun

[premun]

cc @dotnet/prodconsvcs

[premun]

@mmitche do you have more details about a package you saw this with? I can see some 401s from the feed cleaner service (maybe the DevDiv AzDO org MI access is not set up correctly or so)

[mmitche]

Try darc get-asset --name "Microsoft.NETCOre.App.Ref" --version 8.0.10 --max-age 10000

Any recently released packages should have nuget.org as a location.

[dkurepa]

I did some digging on this on Friday, and I think it's not been working for a while now, since we did the AzDo token changes. It's currently failing when it tries to get all feeds in the dnceng org, because it's using this URI https://feeds.dev.azure.com/default/_apis/packaging/feeds. The default part is wrong there, it should be feeds.dev.azure.com/dnceng/_apis/packaging/feeds, and this correlates to what we used to have in the config https://github.com/dotnet/arcade-services/blob/76ac42be5a50b336ef737cc45c0a87c38c4cdbd3/src/Maestro/FeedCleanerService/.config/settings.json#L30.

Also whatever changes we make to Maestro, we should also make to PCS

[premun]

@mmitche we found out why it wasn't running and fixed it. We're running it right now.

There seems to be 770k packages altogether in our 41k dnceng feeds. It's been running for 8 hours and so far cleared 170k packages and 500 feeds.

However, we have some questions:

1. We found out a comment in the code "querying packages after cleaning to see if we can remove an empty feed" but the feed got never removed actually. So we added that functionality.
2. When we compare released packages we only consider these release feeds:

           {
               "Account": "dnceng",
               "Project": "public",
               "Name": "dotnet3"
           },
           {
               "Account": "dnceng",
               "Project": "public",
               "Name": "dotnet3.1"
           },
           {
               "Account": "dnceng",
               "Project": "public",
               "Name": "dotnet5"
           },
           {
               "Account": "dnceng",
               "Project": "public",
               "Name": "dotnet-tools"
           }

   Is that okay or should we add 6/7/8/9? Or did something change after 5? I guess it might have just never gotten updated?

[dkurepa]

Regarding the feeds in the configuration, I did a bit of an investigation and saw that the newer feeds, dotnet7 for example don't appear to be used. I suspect that at some point we started tracking packages in nuget.org and mirroring all of those to dotnet-public. My theory is that this happened in dotnet6+ so that's why we never had those feeds in the config

[premun]

Okay, closing this as the feed does now what it's supposed to