API & Documentation Focus Week

[aschampion]

Goals

1. To simplify the process of building tools that consume, process, export or import data in CATMAID through the HTTP API by providing a clean, clear, consistent interface.
2. To reduce unnecessary coupling between the HTTP API and client and the resulting brittleness and maintenance overhead.
3. To lower the barrier to use of the CATMAID API and contribution to CATMAID for developers others than the core contributors.
4. To better operate as a module of a pipeline or stack, rather than dictate it.

   Primary Outcomes

   • [x] A documented partial-REST HTTP API that provides generic, resource-oriented interfaces for core data, with specialized methods where speed or special behavior is necessary for the CATMAID client.
   • [ ] Review of HTTP API use in the CATMAID client to encourage reuse and make clear for each specialized interface what problem it is solving.
   • [x] Swagger documentation for all HTTP API endpoints.
   • Migration documentation from the old HTTP API to the new API.
   • ~~As a gesture of good faith to third-party consumers of the CATMAID HTTP API, update RCatmaid for the new API and set up tests for it.~~
   • [x] Create an API changelog, parallel to the feature changelog, that documents all additions, changes and deprecations to the HTTP API, grouped by release.
   • [ ] Provide an easy method to generate all documentation for CATMAID, including general documentation, Python documentation, HTTP API documentation, and client Javascript documentation.
   • [x] Provide or link version-appropriate documentation from the CATMAID client and admin interfaces.

     Secondary Outcomes

   • [ ] Migrate error handling to HTTP semantics and responses where appropriate.
   • [x] Better organization of API tests.
   • [x] A pylint rc settings file run as part of continuous integration, but not yet required to pass.
   • [x] Automatically generate and publish all documentation for CATMAID on each release.
   • [x] Move catmaid.org hosting to GitHub pages Read the Docs.

     Inspiration

   • http://connectomes.utah.edu/index.html
   • http://www.knossostool.org/documentation/ (For landing page and user docs.)

[aschampion]

Day 1 planning and goals (nevermind that it's Wednesday):

• 3 resources chosen for focused cleanup and testing of documentation and API goals:
  1. Projects (as a simple CRUD resource)
  2. Annotations (as one of the most used and most confusing resources)
  3. Skeletons (as the top used resource for export, with many easy improvements to clarity and consistency)
• Django Rest Framework (DRF) and DRF-Swagger testing
  • [X] Measure overhead, must be on the order of a few ms (tested, it is < 1ms)
  • [x] Confirm that it correctly pulls python docstrings from methods to Swagger
  • [x] Confirm that it generates good Swagger docs for individual, function-based routes that aren't integrated into Django classes
  • [x] Confirm that it sets cache-busting headers for GET responses correctly (Django does with @never_cache)
• [X] Remove VNC application
• [ ] Reviewer whitelist endpoint should handle DVID/FlyTEM project IDs and return 404 or another status, which the client should handle, so that duplicate endpoints for middlewares are not necessary

Agreed standards:

• Project ID as leading prefix is fine
• Plural resource names both for listings and individual access
• Hyphens, not underscores, for URLs (W3C and Google recommendation)
• Endpoints should accept JSON arrays in single POST parameter for ease of use, but may also accept POST arrays, will follow up
• Functional (i.e., non-resource identifying) parameters be passed as GET or POST parameters, not in URL

[aschampion]

Notes on API endpoints from Wednesday meeting