avoid making requests to wrong mirror

[jku]

so far the plan has been to use TUF client mirror config like this

mirrors = {
    'pypi': {
        'url_prefix': 'http://localhost:8000',
        'metadata_path': 'tuf/', 
        'targets_path': 'simple/',
        'confined_target_dirs': ['simple/']
    },
    'pythonhosted': {
        'url_prefix': 'https://files.pythonhosted.org/packages/',
        'metadata_path': 'None', # should be actual None but TUF won't accept that yet
        'targets_path': '',
        'confined_target_dirs': ['']
    }
}

There may be issues with this as the tuf client may try to download from wrong mirror -- this only results in one extra request and 404, but I still want to avoid it.

Using confined_target_dirs in the pypi mirror does prevent TUF updater from trying to download distribution files from pypi mirror (metadata downloads might theoretically still happen from pythonhosted but in practice the pypi is first in lookup order). However, warehouse may end up storing the index files in project directories (see warehouse issue 8487), and I don't see how I could then use confined_target_dirs (as it does not match files in sub directories)...

If that happens we'll probably need to re-configure mirrors on every download: Whenever we download index files (or refresh metadata), use only pypi mirror. When we download distribution files use only pythonhosted mirror. We still need to use only a single updater because the metadata is and should be shared.

[jku]

Not 100% fixed yet but:

• code now switches mirror configs everytime it downloads something: this prevents the case where index files could be requested from pythonhosted.org
• https://github.com/theupdateframework/tuf/pull/1153 will allow not setting targets_path or metadata_path in mirror config: this prevents the case where metadata is requested from pythonhosted.org (or distribution files from pypi.org)

Those should cover all problematic cases

[jku]

upstream issue is fixed, closing this once there's a release and I can update vendored sources