pip fork to experiment with PEP-458 implementation https://www.python.org/dev/peps/pep-0458/: See branch tuf-v2 (and tuf-mvp and tuf-mvp-vendored for earlier work)
so far the plan has been to use TUF client mirror config like this
mirrors = {
'pypi': {
'url_prefix': 'http://localhost:8000',
'metadata_path': 'tuf/',
'targets_path': 'simple/',
'confined_target_dirs': ['simple/']
},
'pythonhosted': {
'url_prefix': 'https://files.pythonhosted.org/packages/',
'metadata_path': 'None', # should be actual None but TUF won't accept that yet
'targets_path': '',
'confined_target_dirs': ['']
}
}
There may be issues with this as the tuf client may try to download from wrong mirror -- this only results in one extra request and 404, but I still want to avoid it.
Using confined_target_dirs in the pypi mirror does prevent TUF updater from trying to download distribution files from pypi mirror (metadata downloads might theoretically still happen from pythonhosted but in practice the pypi is first in lookup order). However, warehouse may end up storing the index files in project directories (see warehouse issue 8487), and I don't see how I could then use confined_target_dirs (as it does not match files in sub directories)...
If that happens we'll probably need to re-configure mirrors on every download: Whenever we download index files (or refresh metadata), use only pypi mirror. When we download distribution files use only pythonhosted mirror. We still need to use only a single updater because the metadata is and should be shared.
code now switches mirror configs everytime it downloads something: this prevents the case where index files could be requested from pythonhosted.org
https://github.com/theupdateframework/tuf/pull/1153 will allow not setting targets_path or metadata_path in mirror config: this prevents the case where metadata is requested from pythonhosted.org (or distribution files from pypi.org)
so far the plan has been to use TUF client mirror config like this
There may be issues with this as the tuf client may try to download from wrong mirror -- this only results in one extra request and 404, but I still want to avoid it.
Using confined_target_dirs in the pypi mirror does prevent TUF updater from trying to download distribution files from pypi mirror (metadata downloads might theoretically still happen from pythonhosted but in practice the pypi is first in lookup order). However, warehouse may end up storing the index files in project directories (see warehouse issue 8487), and I don't see how I could then use confined_target_dirs (as it does not match files in sub directories)...
If that happens we'll probably need to re-configure mirrors on every download: Whenever we download index files (or refresh metadata), use only pypi mirror. When we download distribution files use only pythonhosted mirror. We still need to use only a single updater because the metadata is and should be shared.