Titles endpoint

[rgaudin]

• List titles on /titles (includes pagination handling)
• Get title details on /title/{name}

Required for #28

[anshulxyz]

@rgaudin

List titles on /titles (includes pagination handling)

Q. What are the properties you want to see in the list? Other than the ident.

[rgaudin]

Thanks for asking ; it's indeed an important question. Here's how I see the /titles REST endpoint:

Note: I know this ticket is about the listings only but I'll document it all here.

GET /titles:

{
  "meta": {
    "count": 2,
    "limit": 10,
    "skip": 0
  },
  "items": [
    {"ident": "wikipedia_fr_all_maxi"},
    {"ident": "wikipedia_fr_all_nopic"}
  ]
}

Rule of thumb is that all list endpoints should use the same wrapping schema with the meta info for pagination. It means that it should also accept skip and limit parameters to control the result list. This is similar to what the zimfarm API does.

Now, this endpoint is gonna be the most used and must be versatile so it serves most if not all of the UI's needs (as well as other needs). It thus needs to allow filtering and payload-control using GET parameters.

Filters:

• ?lang=fra,eng to filter results based on ISO-639-3 lang codes (intersection)
• ?lang=fra|eng to filter results based on ISO-639-3 lang codes (union)
• ?metadata-name=wikipedia_fr_test to filter based on a meta (Name here). Value can include the * character (ex: wikipedia_*_maxi, *_nopic, etc). Illustration_* are not supported obviously.
• ?tag=wikipedia,kids, ?tag=wikipedia|wikibooks to filter results based on Tag (intersection and union) with support for exclusion: tag=wikipedia,!fun|ted,kids (matches Titles with wikipedia or ted that also have the kids tag and not the fun tag) !wikipedia would match all but the wikipedia ones.

Payload-control:

• ?with_metadata=title,scraper,illustration_48x48: includes the requested metadata in response. all is a shortcut for including all metadata (including illustration). all_text is same but only text-ones. illustration is an alias for illustration_48x48.
• ?with_tags: include tags (all) into response.
• ?with_languages: include languages into response.

Complete complex request: GET /titles?lang=fra&metadata-scraper=*1.11&tag=!wikipedia&with_tags&with_languages&with_medata=all

[
  {
    "ident": "wikihow_fr_all_maxi",
    "languages": [
      "fra"
    ],
    "metadata": {
      "Creator": "Wikipedia",
      "Description": "offline version of Wikipedia in Ganda",
      "Flavour": "maxi",
      "Language": "fra",
      "Name": "wikipedia_fr_all_maxi",
      "Publisher": "Kiwix",
      "Tags": "wikihow;_category:wikipedia",
      "Title": "Wikipedia",
      "Scraper": "wikihow2zim 1.11",
      "Illustration_48x48": "xxx"
    },
    "tags": [
      "wikihow",
      "_category:wikipedia"
    ]
  }
]

GET /titles/{ident}

Returns the complete response for the ident

{
  "ident": "wikihow_fr_all_maxi",
  "languages": [
    "fra"
  ],
  "metadata": {
    "Creator": "Wikipedia",
    "Description": "offline version of Wikipedia in Ganda",
    "Flavour": "maxi",
    "Language": "fra",
    "Name": "wikipedia_fr_all_maxi",
    "Publisher": "Kiwix",
    "Tags": "wikihow;_category:wikipedia",
    "Title": "Wikipedia",
    "Scraper": "wikihow2zim 1.11",
    "Illustration_48x48": "xxx"
  },
  "tags": [
    "wikihow",
    "_category:wikipedia"
  ]
}

POST /titles

Create a title using a payload similar to the response above. Fails if ident already exists.

DELETE /titles/{ident}

• removes title completely

PUT /titles/{ident}

replaces the title (including tags and metadata) with the supplied payload (must not include ident).

GET /titles/{ident}/tags

{
  "meta": {
    "count": 2,
    "limit": 10,
    "skip": 0
  },
  "items": [
    "wikipedia",
    "_category:wikipedia"
  ]
}

POST /titles/{ident}/tags

Add Tag to Title using JSON payload

{
  "name": "Scraper",
  "value": "mwoffliner 1.2"
}

PUT /titles/{ident}/tags

replace list of tags with payload

["wikipedia", "_category:wikipedia"]

DELETE /titles/{ident}/tags/{tag}

• removes the tag

GET /titles/{ident}/metadata

{
      "Creator": "Wikipedia",
      "Description": "offline version of Wikipedia in Ganda",
      "Flavour": "maxi",
      "Language": "fra",
      "Name": "wikipedia_fr_all_maxi",
      "Publisher": "Kiwix",
      "Tags": "wikihow;_category:wikipedia",
      "Title": "Wikipedia",
      "Scraper": "wikihow2zim 1.11",
      "Illustration_48x48": "xxx"
}

Payload-control:

• ?with_illustrations to include illustrations or not.

GET /titles/{ident}/metadata/{name}

Single metadata retrieval

{"value": "mwoffliner 1.0"}

PUT /titles/{ident}/metadata/{name}

Updates the metadata value from JSON payload

{"value": "my scraper"}

DELETE /titles/{ident}/metadata/{name}

delete single metadata

GET /titles/{ident}/languages

{
  "meta": {
    "count": 2,
    "limit": 10,
    "skip": 0
  },
  "items": ["fra", "eng"]
}

POST /titles/{ident}/languages

Add a language to Title using

{"value": "ara"}

GET /titles/{ident}/languages/{code}

{
  "code": "fra",
  "english": "French",
  "native": "Français"
}

DELETE /titles/{ident}/languages/{code}

remove language from Title

GET /languages

⚠️ this is outside of /titles but will be necessary for UI (I think).

returns list of all languages.

{
  "meta": {
    "count": 2,
    "limit": 10,
    "skip": 0
  },
  "items": [
    {"code": "fra"},
    {"code": "eng"},
  ]
}

• Supports ?lang=xx filtering
• Supports ?with_english and ?with_native params to control response.

GET /languages/{code}

Single language description

{
  "code": "fra",
  "english": "French",
  "native": "Français"
}