The XML-RPC API code in vanity seems overly complicated, yet is at least sorting the results correctly whereas the JSON implementation is not. Here's an overview of the process:
vanity() calls normalize(package) then downloads_total(package)
downloads_total calls release_data(package)
If JSON, release_data does this::
if protocol:
for package in packages:
data = get_jsonparsed_data(PYPI_JSON % package)
for release in data['releases']:
urls = data['releases'][release]
yield urls, data['info']
return
If not, release_data calls package_releases which does this:
def package_releases(packages):
"""
"""
mcall = xmlrpc.MultiCall(PYPI_XML)
called_packages = deque()
for package in packages:
mcall.package_releases(package, True)
called_packages.append(package)
if len(called_packages) == 100:
result = mcall()
mcall = xmlrpc.MultiCall(PYPI_XML)
for releases in result:
yield called_packages.popleft(), releases
result = mcall()
for releases in result:
yield called_packages.popleft(), releases
So perhaps the challenge is to deque the JSON results, but I failed in trying to get that to work as expected. The "real" issue is a convoluted data model and possibly differing results from different protocols (as far as I can tell.)
The XML-RPC API code in vanity seems overly complicated, yet is at least sorting the results correctly whereas the JSON implementation is not. Here's an overview of the process:
vanity()
callsnormalize(package)
thendownloads_total(package)
downloads_total
callsrelease_data(package)
If JSON,
release_data
does this::If not,
release_data
callspackage_releases
which does this:So perhaps the challenge is to
deque
the JSON results, but I failed in trying to get that to work as expected. The "real" issue is a convoluted data model and possibly differing results from different protocols (as far as I can tell.)