[distribution] prepare a list of actions to publish and promote the API

[severo]

more on this here: https://docs.google.com/document/d/1XgJ8BuPZ2mM_VJ7K3DtwlU_AQSeSgSi8HD40XjE3b34/edit

Work with @huggingface/datasets and @osanseviero + avocado team to push the publication of the API

Tasks:

• [ ] better discovery from the Hub showing how to use the API: https://github.com/huggingface/moon-landing/issues/3722
• [ ] incorporate content into the hub docs https://huggingface.co/docs/hub/index
• [ ] incorporate content into the datasets docs https://huggingface.co/docs/datasets/index. Might be in several places: in the intro, as a separate section (port/API), and as a side note in every relevant API reference function (ie. get_split_names -> the information can also be obtained from a REST API...)

Other ideas

• [ ] prepare a list of announcements:
  • public API
  • why we build it or why we think it's useful: access metadata and data from the Hub datasets through a web API (ie without Python), help the community build exploration tools easily, increasing the offer of tools to connect to the Hub. Currently: allows getting the splits, the first rows, the columns, and their types (features)
  • feedback: what would you want to access through the API? Through GitHub: huggingface/datasets, label "API"? Other channels?
  • roadmap: the future features. Possibly:
  • random access to the rows,
  • statistics on the columns,
  • other preprocessed information (analysis, metrics) about the datasets, eg: detection of biases, distribution of the classes, size of the images, presence or not of faces in image datasets, distribution of images in a class, number of speakers in audio datasets, presence of noise, ie real-world not-clean audio files, in audio datasets.
  • recommendation system, to get a list of other datasets with the same format (same columns with the same type).
  • compute embeddings of the datasets
• [x] announcement materials: code highlighted in carbon, show the postman collection, prepare appealing visuals (eg https://twitter.com/mervenoyann/status/1557753337554034688 or https://twitter.com/mervenoyann/status/1562838963072643072)
• [ ] channels
  • [ ] twitter
  • [ ] linkedin
  • [ ] discord: one specific message per channel (computer vision, audio, etc) (<- too much spam). Better use "announcements"
  • [x] discuss forum -> not for announcements. Might help for feedback
  • [ ] youtube?
  • [x] blog
  • [ ] direct messages to Paperwithcode, Arxiv, Kaggle
• [ ] showcase:
  • [x] observable notebook
  • [ ] HF space (see https://huggingface.co/spaces/julien-c/push-model-from-web for example for a pure browser example)
  • [x] an app that shows all the datasets (bird-eye view), "like" https://lexica.art/ or https://projector.tensorflow.org/?
  • [ ] HF spaces: a dataset selector
  • [ ] pure JS ML using the datasets server to get the data, eg https://mlconsole.com/ / tf.js (even if we only have the first 100 rows for now)
  • [ ] open source a standalone viewer in e.g. Svelte or React or vanilla

Publish the API on specialized sites:

• [x] create a Postman collection (idea). Done, here it is: https://www.postman.com/huggingface/workspace/hugging-face-apis/documentation/23242779-d068584e-96d1-4d92-a703-7cb12cbd8053, in the https://www.postman.com/huggingface team.
  • [ ] try to get a RT from https://twitter.com/getpostman (32K followers)
• [x] create a public API on RapidAPI. Done here: https://rapidapi.com/hugging-face-hugging-face-default/api/hugging-face-datasets-api/, in the https://rapidapi.com/organization/hugging-face org.
  • [ ] try to get a RT from https://twitter.com/rapid_api (89K followers)

ideas by @julien-c @lhoestq @albertvillanova @thomwolf @NimaBoscarino @merveenoyan

[julien-c]

• [ ] open source a standalone viewer in e.g. Svelte or React or vanilla

[severo]

This issue contains a lot of ideas. The timeframe will be something like:

• publish the public API announcement in the next days
• then publish regularly (every two days / every week) related examples to showcase, or try to get feedback, etc.

The idea is to create and feed the momentum

[severo]

• tweets: https://twitter.com/severo_dev/status/1568227077186265089
• linkedin: https://www.linkedin.com/feed/update/urn:li:activity:6973995107293270016/
• discord (#announcements): https://discordapp.com/channels/879548962464493619/897387888663232554/1017789367504347257
• Slack DM to paperwithcode: https://huggingface.slack.com/archives/C01PNHFP214/p1662730930165049

Not sure if it will have any impact, but:

• posted on https://apislist.com/ (not published yet)
• created a Postman collection under the HF org. Everybody at HF can login to the org. https://www.postman.com/huggingface
• created a public API on RapidAPI under the HF org. Everybody at HF can login to the org. https://rapidapi.com/organization/hugging-face

[severo]

• tweets + post linkedin to use the API in Observable / JS: https://observablehq.com/@huggingface/datasets-api-size-of-the-splits, https://twitter.com/severo_dev/status/1569323484383424512, https://www.linkedin.com/feed/update/urn:li:activity:6975093768773328896/

[severo]

• demo, to browse all the Hub image datasets: https://observablehq.com/@huggingface/all-the-images-in-the-hugging-face-hub, https://twitter.com/severo_dev/status/1570069111652188162, https://www.linkedin.com/posts/sylvain--lesage_image-machinelearning-datasets-activity-6975835549395787776-jb-Y/

[severo]

• rework the datasets-server docs: https://github.com/huggingface/datasets-server/pull/566
• integrate links to the API in the datasets docs: https://github.com/huggingface/datasets/pull/4984

[severo]

Moved to the internal tracker