API documentation - improve automation and content

nanometrenat commented 4 years ago

Issue

API documentation is at https://goadmin.ifrc.org/docs/

This needs to be improved and updated, to be more useful to API users. Scratchpad thoughts in Teams wiki, to be added here when actions agreed.

nanometrenat commented 4 years ago

Also to figure out how to add the non-v2 API endpoints to the doco (either same doco page or a different one)

gulfaraz commented 4 years ago

@batpad @nanometrenat Please review the README and confirm if the basic documentation guidelines are sufficient.

nanometrenat commented 4 years ago

@gulfaraz thanks for that - two further related qs for you:

is there somewhere we can also include a generic section at the top (below the heading and above the first endpoint) or is that impossible? There are quite a few aspects that are common across all.
is it feasible to do some kind of full check to see if it's possible to tell which params have and haven't actually been implemented? e.g. through #759 and #790 we found that "search" is only implemented in one although it is listed as an option on all - so not sure why it is appearing on the docs when it's not really there!

Also if it's in the functioning part of the code if I were to do the doco updates I would need advice on update process (in terms of branches and PRs etc. so as not to affect anyone else's work - or perhaps best I just feed the updates to you for action instead?

thanks + cheers

gulfaraz commented 4 years ago

is there somewhere we can also include a generic section at the top (below the heading and above the first endpoint) or is that impossible? There are quite a few aspects that are common across all.

A common string is possible using the description attribute in main/urls.py. It would be ideal for a paragraph or two for describing the whole set of APIs. For anything more complex, I would recommend using a plugin (or a separate static page) which allows extensive customization.

is it feasible to do some kind of full check to see if it's possible to tell which params have and haven't actually been implemented? e.g. through #759 and #790 we found that "search" is only implemented in one although it is listed as an option on all - so not sure why it is appearing on the docs when it's not really there!

From main/settings.py, it looks like search filter is enabled by default for all endpoints. We should disable this filter as default to remove the corresponding documentation. I would check with the dev who enabled this as default before removing anything.

Also if it's in the functioning part of the code if I were to do the doco updates I would need advice on update process (in terms of branches and PRs etc. so as not to affect anyone else's work - or perhaps best I just feed the updates to you for action instead?

These doc changes are integrated with the functioning parts of the code. I can walk you through the branch + PR process over a call.

nanometrenat commented 4 years ago

Thanks @gulfaraz for all the investigation and updates. Next step is content creation. I will give it a go when I next have a moment (after installing GIT) else backstop is to send through the content for dev.

batpad commented 4 years ago

@gulfaraz in the meanwhile, could you make a PR for the documentation update? That will be useful regardless.

nanometrenat commented 4 years ago

@JonathanGarro FYI

nanometrenat commented 3 years ago

@ypyelab this is the API doco ticket - not sure if the instructions at https://github.com/IFRCGo/go-api#documentation still apply given that https://goadmin.ifrc.org/docs/ is in a different format to previously - but for sure worth making sure all the parameters etc. have specific details re what's expected/format/how to use (i.e. data dictionary type stuff, separate to the bigger picture use case documentation by JG). Here is some prelim work of that nature by BRC (from last year so you would need to check if anything changed since the changes for ticket https://github.com/IFRCGo/go-frontend/issues/1529)

batpad commented 1 year ago

Bumping this.

@thenav56 @szabozoltan69 @k9845 it does seem like the auto docs generation is weirdly broken and is a bit hard to reason about right now. Can we take some time to review the tooling around our documentation generation and bring things up-to-date?

We should also auto-generate an openapi.yml schema definition file. Currently, one is supposed to be able to manually do this with something like python manage.py generateschema . Right now, this generates a very small subset of the API schema, along-with a bunch of errors:

# python manage.py generateschema
/usr/local/lib/python3.8/site-packages/django_filters/rest_framework/backends.py:127: UserWarning: <class 'api.drf_views.CountryKeyFigureViewset'> is not compatible with schema generation
  warnings.warn(
/usr/local/lib/python3.8/site-packages/django_filters/rest_framework/backends.py:127: UserWarning: <class 'api.drf_views.CountrySnippetViewset'> is not compatible with schema generation
  warnings.warn(
/usr/local/lib/python3.8/site-packages/django_filters/rest_framework/backends.py:127: UserWarning: <class 'api.drf_views.EventViewset'> is not compatible with schema generation
  warnings.warn(
/usr/local/lib/python3.8/site-packages/django_filters/rest_framework/backends.py:127: UserWarning: <class 'api.drf_views.FieldReportViewset'> is not compatible with schema generation
  warnings.warn(
/usr/local/lib/python3.8/site-packages/django_filters/rest_framework/backends.py:127: UserWarning: <class 'api.drf_views.EventSnippetViewset'> is not compatible with schema generation
  warnings.warn(
/usr/local/lib/python3.8/site-packages/django_filters/rest_framework/backends.py:127: UserWarning: <class 'per.drf_views.LatestCountryOverviewViewset'> is not compatible with schema generation
  warnings.warn(
/usr/local/lib/python3.8/site-packages/django_filters/rest_framework/backends.py:127: UserWarning: <class 'per.drf_views.FormViewset'> is not compatible with schema generation
  warnings.warn(
/usr/local/lib/python3.8/site-packages/django_filters/rest_framework/backends.py:127: UserWarning: <class 'per.drf_views.FormCountryViewset'> is not compatible with schema generation
  warnings.warn(
/usr/local/lib/python3.8/site-packages/django_filters/rest_framework/backends.py:127: UserWarning: <class 'per.drf_views.FormDataViewset'> is not compatible with schema generation
  warnings.warn(
/usr/local/lib/python3.8/site-packages/django_filters/rest_framework/backends.py:127: UserWarning: <class 'per.drf_views.PERDocsViewset'> is not compatible with schema generation
  warnings.warn(
/usr/local/lib/python3.8/site-packages/django_filters/rest_framework/backends.py:127: UserWarning: <class 'per.drf_views.OverviewStrictViewset'> is not compatible with schema generation
  warnings.warn(
/usr/local/lib/python3.8/site-packages/django_filters/rest_framework/backends.py:127: UserWarning: <class 'deployments.drf_views.PersonnelViewset'> is not compatible with schema generation
  warnings.warn(
/usr/local/lib/python3.8/site-packages/django_filters/rest_framework/backends.py:127: UserWarning: <class 'per.drf_views.FormPermissionViewset'> is not compatible with schema generation
  warnings.warn(
/usr/local/lib/python3.8/site-packages/django_filters/rest_framework/backends.py:127: UserWarning: <class 'api.drf_views.ProfileViewset'> is not compatible with schema generation
  warnings.warn(
/usr/local/lib/python3.8/site-packages/django_filters/rest_framework/backends.py:127: UserWarning: <class 'deployments.drf_views.ProjectViewset'> is not compatible with schema generation
  warnings.warn(
/usr/local/lib/python3.8/site-packages/django_filters/rest_framework/backends.py:127: UserWarning: <class 'api.drf_views.RegionKeyFigureViewset'> is not compatible with schema generation
  warnings.warn(
/usr/local/lib/python3.8/site-packages/django_filters/rest_framework/backends.py:127: UserWarning: <class 'api.drf_views.RegionSnippetViewset'> is not compatible with schema generation
  warnings.warn(
/usr/local/lib/python3.8/site-packages/django_filters/rest_framework/backends.py:127: UserWarning: <class 'api.drf_views.SituationReportViewset'> is not compatible with schema generation
  warnings.warn(
/usr/local/lib/python3.8/site-packages/django_filters/rest_framework/backends.py:127: UserWarning: <class 'notifications.drf_views.SubscriptionViewset'> is not compatible with schema generation
  warnings.warn(
/usr/local/lib/python3.8/site-packages/django_filters/rest_framework/backends.py:127: UserWarning: <class 'api.drf_views.UserViewset'> is not compatible with schema generation
  warnings.warn(
/usr/local/lib/python3.8/site-packages/django_filters/rest_framework/backends.py:127: UserWarning: <class 'dref.views.DrefViewSet'> is not compatible with schema generation
  warnings.warn(
/usr/local/lib/python3.8/site-packages/django_filters/rest_framework/backends.py:127: UserWarning: <class 'dref.views.DrefOperationalUpdateViewSet'> is not compatible with schema generation
  warnings.warn(
/usr/local/lib/python3.8/site-packages/django_filters/rest_framework/backends.py:127: UserWarning: <class 'dref.views.DrefFinalReportViewSet'> is not compatible with schema generation
  warnings.warn(
/usr/local/lib/python3.8/site-packages/django_filters/rest_framework/backends.py:127: UserWarning: <class 'dref.views.CompletedDrefOperationsViewSet'> is not compatible with schema generation
  warnings.warn(
/usr/local/lib/python3.8/site-packages/django_filters/rest_framework/backends.py:127: UserWarning: <class 'dref.views.ActiveDrefOperationsViewSet'> is not compatible with schema generation
  warnings.warn(
info:
  description: ''
  title: ''
  version: ''
openapi: 3.0.0
paths:
  /change_password:
    post:
      operationId: change_password_create
      tags:
      - change_password
  /createperoverview:
    post:
      operationId: createperoverview_create
      tags:
      - createperoverview
  /deleteperoverview:
    post:
      operationId: deleteperoverview_create
      tags:
      - deleteperoverview
  /get_auth_token:
    post:
      operationId: get_auth_token_create
      tags:
      - get_auth_token
  /recover_password:
    post:
      operationId: recover_password_create
      tags:
      - recover_password
  /register:
    post:
      operationId: register_create
      tags:
      - register
  /resend_validation:
    post:
      operationId: resend_validation_create
      tags:
      - resend_validation
  /sendperworkplan:
    post:
      operationId: sendperworkplan_create
      tags:
      - sendperworkplan
  /show_username:
    post:
      operationId: show_username_create
      tags:
      - show_username
  /updatemultipleperforms:
    post:
      operationId: updatemultipleperforms_create
      tags:
      - updatemultipleperforms
  /updateperform:
    post:
      operationId: updateperform_create
      tags:
      - updateperform
  /updateperoverview:
    post:
      operationId: updateperoverview_create
      tags:
      - updateperoverview
  /validate_user:
    get:
      operationId: validate_user_list
      tags:
      - validate_user
  /verify_email:
    get:
      operationId: verify_email_list
      tags:
      - verify_email
servers:
- url: ''

I'm not sure if this is related to the overall weirdness around the automatic docs generation, but it seems likely. We should:

Revisit the libraries we are using for automatic doc generation
Test that everything that is supposed to work works:
- Updating doc strings on functions as per https://github.com/IFRCGo/go-api#documentation should work / update the docs on /docs/
- Generating OpenAPI YAML should work for all endpoints

If things do not work, consider other tooling, etc. so that updating docstrings works to update docs, and we can update instructions in the README.

Once this is done, we can start the process of incorporating the excellent documentation @ypyelab has made into the API docs.

nanometrenat commented 9 months ago

@arunissun @tovari @batpad is it still useful having this ticket open, or should we close it given the new world? Cheers

tovari commented 9 months ago

@nanometrenat, we can close this, the api is nicely documented on go-wiki.

nanometrenat commented 9 months ago

Actually, on second thoughts, we need to reopen - as currently the "GO Documentation" link in the GO Footer

still goes to https://goadmin.ifrc.org/docs/

This is fine, because the GO Wiki also directs people to that /docs location to see the Endpoints and parameters.

But we also need to make sure that people who go straight to https://goadmin.ifrc.org/docs/ even know that the GO wiki exists!

The only (tiny) action required by devs please: **Please update the top section of https://goadmin.ifrc.org/docs/ to Replace the words "IFRC-GO_ API Documenation" with the words "Please see the GO Wiki for an overview of API usage." (the link is to https://go-wiki.ifrc.org/en/go-api/api-overview)

The other consideration is how to link to Monty API doco - but that's for another day. @tovari may you please put the above in the to do list? Quick and easy! Thanks

szabozoltan69 commented 9 months ago

What about showing somehow https://goadmin.ifrc.org/api-docs/swagger-ui also? As an interactive API documentation. @nanometrenat

nanometrenat commented 9 months ago

Sounds great! As I've been out of the loop for so long I don't think I'm best placed to advise, my key objectives are:

make sure all users can easily know how to find the information we want them to see (no matter what their entry point is)
if there is any documentation that is out of date or not being actively maintained then we should deprecate it (either keep it visible but marked as out of date, or better yet remove it altogether)

If the Swagger option is deemed to be the best place for people to see endpoints etc. then we should make sure GO footer and GO wiki and GO docs landing page all point to that :) THANK YOU!

arunissun commented 9 months ago

I will re organize the structure of the documentation on Go - wiki. At the moment it has many pages which needs to be combined

szabozoltan69 commented 9 months ago

I modified the description to this: Please see the GO Wiki for an overview of API usage, or the interactive Swagger page.

szabozoltan69 commented 9 months ago

Done via https://github.com/IFRCGo/go-api/commit/76230f654544bb6272aac6b2c64e3b255c2edba6

nanometrenat commented 8 months ago

Closing ticket as I can see this has been implemented in Prod - THANKS!

@tovari the ticket didn't seem to change status when it was released? Shout if needing a hand to figure out a process to keep tickets in sync :)

IFRCGo / go-api

API documentation - improve automation and content #770

Issue