Parent issue for hub & user contributions

[skrawcz]

What is this issue?

It's an issue to help coordinate/reference the work that joins hamilton, hamilton/contrib, and hub.dagworks.io for an initial release.

---

Hub website structure

General

• [x] Algolia search integration
• [x] Documentation links in hamilton.dagworks.io
• [ ] Take a pass at messaging & images
• [ ] Surface all commits for a dataflow

Tags

• [ ] Create canonical list of initial top level tags
• [ ] Surface tags better to help people find things
• [ ] Write a github action that checks for tag conformance

Dataflow page

• [ ] Add JSON structure for encoding example inputs with a submitted dataflow
• [ ] Expose example inputs on dataflow page for cut & paste
• [ ] Expose example configuration better

---

sf-hamilton-contrib package

• [ ] Come up with unit testing story for submitted dataflows. We want to showcase testing & ensuring things work.
• [ ] Build github actions workflow to publish it (can be manually triggered)
• [ ] Add documentation to help people contribute more easily
• [ ] Add documentation to hamilton.dagworks.io about sf-hamilton-contrib

Dataflows wanted

• [ ] LLM related flows
• [ ] ML related flows
• [ ] Common data preprocessing flows
• [ ] Web request processing flows
• [ ] Repackaging some of the Hamilton examples?

---

sf-hamilton package

• [ ] Add documentation on functions we want to expose in dataflows module.
• [ ] Add documentation to hamilton.dagworks.io
• [ ] Add CLI command / someway to bootstrap a new sf-hamilton-contrib contribution

---

General Testing

• [ ] Test end to end flow of:

  1. User creating a PR
  2. Merging it
  3. Docs being updated
  4. User downloads it

• [ ] Test end to end flow of:

  1. User comes to hub
  2. User searches
  3. User gets what they need to get started.

• [ ] Test the page on some OS users:

  1. Get their impressions and feedback.

[skrawcz]

We did most of these. Closing for now.