Capture package metadata from PyPI

[ericholscher]

The package_info field isn't currently being updated properly. We should be capturing more info here.

[agjohnson]

Any more info here? This isn't sounding familiar

[ericholscher]

The data we sync from Pypi has more info about the package, which we could use for sorting/filtering/searching.

[agjohnson]

This would be great to pull in so that we can display package release time, package short description, and author information.

[agjohnson]

Useful metadata (from either libraries or pypi):

• Package author
• Package release time
• Package short description
• Package popularity for sorting (downloads or something from libraries)

[ericholscher]

Looks like we have this info for some packages. Believe we just need to re-run the syncing code and it should be there.

[ericholscher]

In [16]: rels = Release.objects.filter(package_info__author__isnull=False)

In [18]: rels[0].package_info
Out[18]:
{'_pypi_hidden': False,
 '_pypi_ordering': 24,
 'author': 'Andreas Mueller',
 'author_email': 'amueller@ais.uni-bonn.de',
 'bugtrack_url': '',
 'cheesecake_code_kwalitee_id': None,
 'cheesecake_documentation_id': None,
 'cheesecake_installability_id': None,
 'classifiers': ['Intended Audience :: Developers',
  'Intended Audience :: Science/Research',
  'License :: OSI Approved',
  'Operating System :: MacOS',
  'Operating System :: Microsoft :: Windows',
  'Operating System :: POSIX',
  'Operating System :: Unix',
  'Programming Language :: C',
  'Programming Language :: Python',
  'Programming Language :: Python :: 2',
  'Programming Language :: Python :: 2.6',
  'Programming Language :: Python :: 2.7',
  'Programming Language :: Python :: 3',
  'Programming Language :: Python :: 3.4',
  'Programming Language :: Python :: 3.5',
  'Topic :: Scientific/Engineering',
  'Topic :: Software Development'],
 'description': '.. -*- mode: rst -*-\r\n\r\n|Travis|_ |AppVeyor|_ |Coveralls|_ |CircleCI|_ |Python27|_ |Python35|_ |PyPi|_ |DOI|_\r\n\r\n.. |Travis| image:: https://api.travis-ci.org/scikit-learn/scikit-learn.svg?branch=master\r\n.. _Travis: https://travis-ci.org/scikit-learn/scikit-learn\r\n\r\n.. |AppVeyor| image:: https://ci.appveyor.com/api/projects/status/github/scikit-learn/scikit-learn?branch=master&svg=true\r\n.. _AppVeyor: https://ci.appveyor.com/project/sklearn-ci/scikit-learn/history\r\n\r\n.. |Coveralls| image:: https://coveralls.io/repos/scikit-learn/scikit-learn/badge.svg?branch=master&service=github\r\n.. _Coveralls: https://coveralls.io/r/scikit-learn/scikit-learn\r\n\r\n.. |CircleCI| image:: https://circleci.com/gh/scikit-learn/scikit-learn/tree/master.svg?style=shield&circle-token=:circle-token\r\n.. _CircleCI: https://circleci.com/gh/scikit-learn/scikit-learn\r\n\r\n.. |Python27| image:: https://img.shields.io/badge/python-2.7-blue.svg\r\n.. _Python27: https://badge.fury.io/py/scikit-learn\r\n\r\n.. |Python35| image:: https://img.shields.io/badge/python-3.5-blue.svg\r\n.. _Python35: https://badge.fury.io/py/scikit-learn\r\n\r\n.. |PyPi| image:: https://badge.fury.io/py/scikit-learn.svg\r\n.. _PyPi: https://badge.fury.io/py/scikit-learn\r\n\r\n.. |DOI| image:: https://zenodo.org/badge/21369/scikit-learn/scikit-learn.svg\r\n.. _DOI: https://zenodo.org/badge/latestdoi/21369/scikit-learn/scikit-learn\r\n\r\nscikit-learn\r\n============\r\n\r\nscikit-learn is a Python module for machine learning built on top of\r\nSciPy and distributed under the 3-Clause BSD license.\r\n\r\nThe project was started in 2007 by David Cournapeau as a Google Summer\r\nof Code project, and since then many volunteers have contributed. See\r\nthe `AUTHORS.rst <AUTHORS.rst>`_ file for a complete list of contributors.\r\n\r\nIt is currently maintained by a team of volunteers.\r\n\r\nWebsite: http://scikit-learn.org\r\n\r\nInstallation\r\n------------\r\n\r\nDependencies\r\n~~~~~~~~~~~~\r\n\r\nscikit-learn requires:\r\n\r\n- Python (>= 2.6 or >= 3.3)\r\n- NumPy (>= 1.6.1)\r\n- SciPy (>= 0.9)\r\n\r\nscikit-learn also uses CBLAS, the C interface to the Basic Linear Algebra\r\nSubprograms library. scikit-learn comes with a reference implementation, but\r\nthe system CBLAS will be detected by the build system and used if present.\r\nCBLAS exists in many implementations; see `Linear algebra libraries\r\n<http://scikit-learn.org/stable/modules/computational_performance.html#linear-algebra-libraries>`_\r\nfor known issues.\r\n\r\nUser installation\r\n~~~~~~~~~~~~~~~~~\r\n\r\nIf you already have a working installation of numpy and scipy,\r\nthe easiest way to install scikit-learn is using ``pip`` ::\r\n\r\n    pip install -U scikit-learn\r\n\r\nor ``conda``::\r\n\r\n    conda install scikit-learn\r\n\r\nThe documentation includes more detailed `installation instructions <http://scikit-learn.org/stable/install.html>`_.\r\n\r\n\r\nDevelopment\r\n-----------\r\n\r\nWe welcome new contributors of all experience levels. The scikit-learn\r\ncommunity goals are to be helpful, welcoming, and effective. The\r\n`Development Guide <http://scikit-learn.org/stable/developers/index.html>`_\r\nhas detailed information about contributing code, documentation, tests, and\r\nmore. We\'ve included some basic information in this README.\r\n\r\nImportant links\r\n~~~~~~~~~~~~~~~\r\n\r\n- Official source code repo: https://github.com/scikit-learn/scikit-learn\r\n- Download releases: https://pypi.python.org/pypi/scikit-learn\r\n- Issue tracker: https://github.com/scikit-learn/scikit-learn/issues\r\n\r\nSource code\r\n~~~~~~~~~~~\r\n\r\nYou can check the latest sources with the command::\r\n\r\n    git clone https://github.com/scikit-learn/scikit-learn.git\r\n\r\nSetting up a development environment\r\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n\r\nQuick tutorial on how to go about setting up your environment to\r\ncontribute to scikit-learn: https://github.com/scikit-learn/scikit-learn/blob/master/CONTRIBUTING.md\r\n\r\nTesting\r\n~~~~~~~\r\n\r\nAfter installation, you can launch the test suite from outside the\r\nsource directory (you will need to have the ``nose`` package installed)::\r\n\r\n    nosetests -v sklearn\r\n\r\nUnder Windows, it is recommended to use the following command (adjust the path\r\nto the ``python.exe`` program) as using the ``nosetests.exe`` program can badly\r\ninteract with tests that use ``multiprocessing``::\r\n\r\n    C:\\Python34\\python.exe -c "import nose; nose.main()" -v sklearn\r\n\r\nSee the web page http://scikit-learn.org/stable/install.html#testing\r\nfor more information.\r\n\r\n    Random number generation can be controlled during testing by setting\r\n    the ``SKLEARN_SEED`` environment variable.\r\n\r\nSubmitting a Pull Request\r\n~~~~~~~~~~~~~~~~~~~~~~~~~\r\n\r\nBefore opening a Pull Request, have a look at the\r\nfull Contributing page to make sure your code complies\r\nwith our guidelines: http://scikit-learn.org/stable/developers/index.html\r\n\r\n\r\nProject History\r\n---------------\r\n\r\nThe project was started in 2007 by David Cournapeau as a Google Summer\r\nof Code project, and since then many volunteers have contributed. See\r\nthe AUTHORS.rst file for a complete list of contributors.\r\n\r\nThe project is currently maintained by a team of volunteers.\r\n\r\n**Note**: `scikit-learn` was previously referred to as `scikits.learn`.\r\n\r\n\r\nHelp and Support\r\n----------------\r\n\r\nDocumentation\r\n~~~~~~~~~~~~~\r\n\r\n- HTML documentation (stable release): http://scikit-learn.org\r\n- HTML documentation (development version): http://scikit-learn.org/dev/\r\n- FAQ: http://scikit-learn.org/stable/faq.html\r\n\r\nCommunication\r\n~~~~~~~~~~~~~\r\n\r\n- Mailing list: https://mail.python.org/mailman/listinfo/scikit-learn\r\n- IRC channel: ``#scikit-learn`` at ``irc.freenode.net``\r\n- Stack Overflow: http://stackoverflow.com/questions/tagged/scikit-learn\r\n- Website: http://scikit-learn.org\r\n\r\nCitation\r\n~~~~~~~~\r\n\r\nIf you use scikit-learn in a scientific publication, we would appreciate citations: http://scikit-learn.org/stable/about.html#citing-scikit-learn',
 'docs_url': None,
 'download_url': 'https://pypi.org/project/scikit-learn/#files',
 'downloads': {'last_day': 0, 'last_month': 0, 'last_week': 0},
 'home_page': 'http://scikit-learn.org',
 'keywords': '',
 'license': 'new BSD',
 'maintainer': '',
 'maintainer_email': '',
 'name': 'scikit-learn',
 'package_url': 'http://pypi.python.org/pypi/scikit-learn',
 'platform': '',
 'release_url': 'http://pypi.python.org/pypi/scikit-learn/0.18.2',
 'requires_dist': ["scipy (>=0.9); extra == 'alldeps'",
  "numpy (>=1.6.1); extra == 'alldeps'"],
 'requires_python': '',
 'summary': 'A set of python modules for machine learning and data mining',
 'version': '0.18.2'}