pypa / bandersnatch

A PyPI mirror client according to PEP 381 http://www.python.org/dev/peps/pep-0381/
Academic Free License v3.0
455 stars 141 forks source link

Can I edit the file "todo"? I encountered a lot of "no longer exists on PyPI" #1673

Closed dajianderichang closed 8 months ago

dajianderichang commented 8 months ago

I have synchronized 13 terabytes of data, and I restarted the bandersnatch service due to configuration changes. A large number of "no longer exists on PyPI" messages are displayed in the startup log. The synchronization of new packages starts about 50 minutes later.

I would like to remove these "no longer exists on PyPI" packages from todo, or is there a better way that doesn't require waiting 50 minutes.

Thanks.

Logs:

2024-02-23 00:45:52,492 INFO: 0wneg no longer exists on PyPI (package.py:66) 2024-02-23 00:45:52,492 INFO: Fetching metadata for package: 11dl-gpu (serial 5644939) (package.py:58) 2024-02-23 00:45:52,821 INFO: 11dl-gpu no longer exists on PyPI (package.py:66) 2024-02-23 00:45:52,821 INFO: Fetching metadata for package: 123456 (serial 19838552) (package.py:58) 2024-02-23 00:45:53,129 INFO: 123456 no longer exists on PyPI (package.py:66) 2024-02-23 00:45:53,129 INFO: Fetching metadata for package: 1234exec (serial 15400513) (package.py:58) 2024-02-23 00:45:53,819 INFO: 1234exec no longer exists on PyPI (package.py:66) 2024-02-23 00:45:53,819 INFO: Fetching metadata for package: 168learn (serial 15341239) (package.py:58) 2024-02-23 00:45:54,150 INFO: 168learn no longer exists on PyPI (package.py:66)

dajianderichang commented 8 months ago

And just worried that the wrong operation will erase the 13TB data

cooperlees commented 8 months ago

Bandersnatch doesn't have the ability to delete any of your downloaded artifacts unless your run a bandersnatch verify --delete

So deleting / editing the TODO file should be fine. It's also fine to just delete the file as we should rerun from the serial you last successfully synced from.