search

pypi / warehouse

The Python Package Index
https://pypi.org
Apache License 2.0
3.6k stars 963 forks source link

xmlrpc serial #3603

Closed cztomsik closed 6 years ago

cztomsik commented 6 years ago

Hey, I've noticed you are going to deprecate xmlrpc entirely, is there any other way to obtain delta of package serials? In the sense of being able which packages has been changed. I'm in the process of adding pypi to http://searchlibs.com and I thought it would be much better to download only changed packages metadata rather than doing dumb never-ending polling.

Also, I have one implementation-specific question - is it cheaper (in terms of performance) for you if I ask for list_packages_with_serial() everytime and diff it against previous result or is it cheaper to store last serial and do changelog_since_serial() and then figure out changed names?

And another related question - I've noticed you are actively developing bandersnatch which also depends on this api, so what is going to happen with mirroring, are you going to drop it?

di commented 6 years ago

@cztomsik Thanks for the report. Please be assured that we won't deprecate XML-RPC until we have added other APIs that fulfill the needs of current XML-RPC users.

I'm in the process of adding pypi to http://searchlibs.com and I thought it would be much better to download only changed packages metadata rather than doing dumb never-ending polling.

I'm not 100% sure what you're trying to do here. PyPI release metadata is now immutable and will not change for a given release. The only thing that would change is a project adding/removing a release, adding/removing files for a release, and removing the project entirely.

Also, I have one implementation-specific question - is it cheaper (in terms of performance) for you if I ask for list_packages_with_serial() everytime and diff it against previous result or is it cheaper to store last serial and do changelog_since_serial() and then figure out changed names?

I think performance/bandwidth-wise, changelog_since_serial would be cheaper if you are syncing frequently.

And another related question - I've noticed you are actively developing bandersnatch which also depends on this api, so what is going to happen with mirroring, are you going to drop it?

Nope, we actively depend on bandersnatch to server as a mirror for PyPI itself, so we are invested in keeping it up-to-date.

cztomsik commented 6 years ago

Cool, thx for your response. And it's not very frequent, I just need to know if there were any new releases.

di commented 6 years ago

Is it possible the RSS feeds would work for you? http://warehouse.readthedocs.io/api-reference/feeds/

cztomsik commented 6 years ago

I don't think so, I need all new releases since the previous visit.

But anyway I'm totally fine with the current api, I was mostly curious if it's going to stay.

brainwane commented 6 years ago

Thanks @cztomsik! I think we've answered all your questions about this so I'm going to close this. But please check out our other APIs/feeds issues and consider subscribing to ones that affect you, and if you think we should update our API docs to reflect stuff you're not seeing there, please speak up? Thanks.