[EPIC] Improve examples to make them easier to navigate

[alamb]

Is your feature request related to a problem or challenge?

The current number of examples in https://github.com/apache/datafusion/tree/main/datafusion-examples is somewhat overwhelming and hard to navigate

Describe the solution you'd like

I would like fewer consolidated examples. For example a sql.rs example rather than SQL examples being spread out across (at least) 9 different files

• parqet_sql
• parqet_sql_multiple_files
• regexp
• to_char
• to_date
• to_timestamp
• make_date
• avro_sql.rs

Subtasks

• [x] https://github.com/apache/datafusion/issues/11230
• [x] https://github.com/apache/datafusion/issues/11210
• [ ] https://github.com/apache/datafusion/issues/1813

Describe alternatives you've considered

No response

Additional context

Also, I am trying to add some new examples, but the CI in my PR https://github.com/apache/datafusion/pull/11088 is failing due to running out of space

Example failure: https://github.com/apache/datafusion/actions/runs/9715084012/job/26815819949

  = note: /usr/bin/ld: final link failed: No space left on device
          collect2: error: ld returned 1 exit status

I think it is because each example creates its own binary which fills up the disk

Thus I also want to reduce the number of examples so I can add new ones :)

[alamb]

While working with @lewiszlw in https://github.com/apache/datafusion/pull/11173#issuecomment-2198280594 I think I came up with a better idea -- that is to update the user guide so they code examples are run / tested

Then we can move many/all of the examples from rust to the user guide, which will make them easier to find and navigate. 🏃

[alamb]

I played around with what we have and it turns out I think we can test examples in the user guide. I made a PR with docs of how to do it: https://github.com/apache/datafusion/pull/11178

I think with that framework we could add many of the simpler examples directly into the documentation where they would be eaiser to find. We can leave the more complex ones in datafusion-examples perhaps

Thoughts?

[efredine]

As a newcomer I'll say that this seems right to me. Examples that are right in the documentation are much more discoverable.

[alamb]

I plan do to this as a background task, but realistically can't devote enough focus time to it now