pydotorg / backlog

a parallel playground for Python community services
0 stars 0 forks source link

Teach Sphinx to provide a list of modules as open data #31

Open techtonik opened 9 years ago

techtonik commented 9 years ago

Why?

  1. Solve issue #8
  2. Provide info about all Python versions
  3. Make data available from the web for data mining, and 3rd party services / tools
  4. Provide a stable data channel for future versions

Current solution:

https://stackoverflow.com/questions/6463918/how-can-i-get-a-list-of-all-the-python-standard-library-modules

which assumes that you need to download and parse Sphinx object inventory (https://docs.python.org/2.7/objects.inv), which is a binary file, or download and execute Python for every version and each platform (some modules are not available on some platforms).

Ideally, the Sphinx should give out the open data about stdlib structure, such as : https://docs.python.org/2.7/_data/1.0/modules.json https://docs.python.org/2.7/_data/1.0/modules.csv list all module names, sorted by name

Then you can easily load this data into web app or use it as a table for analysis. http://www.w3.org/2013/csvw/wiki/Main_Page

techtonik commented 9 years ago

Missed the answer that recommends to parse https://docs.python.org/2/py-modindex.html

techtonik commented 9 years ago

Actually, HTML markup may change in future - https://github.com/jackmaney/python-stdlib-list/issues/1

techtonik commented 9 years ago

Posted to sphinx-dev https://groups.google.com/forum/#!topic/sphinx-dev/Wa_-aPAReC8