search

PrefectHQ / prefect-collection-template

Template to quickly bootstrap a Prefect Collection
Apache License 2.0
68 stars 8 forks source link

Add recipes repo link to readme #90

Closed serinamarie closed 2 years ago

serinamarie commented 2 years ago

@ahuang11 Not sure if diving deeper is the best word-choice here, but it worked well in recipes :)

ahuang11 commented 2 years ago

With this, we should probably add more collection related recipes to prefect-recipes since I think all of the ones on the recipes page at the moment is core?

serinamarie commented 2 years ago

At the top of my head I know we have prefect-snowflake, prefect-airbyte, and prefect-aws. I can make a recipe request for more frequently used collections. Any in particular you were thinking of?

desertaxle commented 2 years ago

At the top of my head I know we have prefect-snowflake, prefect-airbyte, and prefect-aws. I can make a recipe request for more frequently used collections. Any in particular you were thinking of?

Based on a scan of the recipes repo, those recipes are for 1.0. I don't think that we have any recipes that use collections, which is something we should fix! I think the link to the Prefect Recipes repo can be added on a case by case basis once a recipe for the given collection exists.

serinamarie commented 2 years ago

Maybe I'm misunderstanding something. Here are some recipes that utilize the prefect-aws, prefect-airbyte, and prefect-snowflake collections.

Export Airbyte Configuration and Load to S3 bucket using blocks, including Python-based deployment

ELT with Snowflake Using Async and Blocks

desertaxle commented 2 years ago

Ah, I missed those! I think it makes sense to have a link to Prefect Recipes repo in the collection repos that correspond to those recipes. Potentially even link to those specific recipes in addition to a link to the recipes repo landing page.

ahuang11 commented 2 years ago

Are there plans to make a prefect-recipes webpage, combining visual cards like https://orion-docs.prefect.io/recipes/recipes/ and filtering ability like https://www.prefectlabs.com/experts? Then all collections can link to a specific URL with the filter applied corresponding to the collection, e.g. Click [here](https://www.prefectlabs.com/recipes?tag=prefect-snowflake) to see recipes related to the collection

I imagine the single source of truth / backend data can be the prefect-recipes page with addition of tags that indicate whether it's core prefect or a prefect collection.

I feel like if we add a link to those specific recipes to each repo, there will be three places that we need to keep track of, which is likely to become out of sync:

  1. prefect-recipes
  2. Prefect docs
  3. the corresponding Collection
serinamarie commented 2 years ago

I feel like if we add a link to those specific recipes to each repo, there will be three places that we need to keep track of, which is likely to become out of sync:

  1. prefect-recipes
  2. Prefect docs
  3. the corresponding Collection

Indeed, I think we should limit the sources of truth. But I think it's a good idea to discuss at the next recipes meeting, so thanks for that!

ahuang11 commented 2 years ago

Closing this for now, opened https://github.com/PrefectHQ/prefect-recipes/issues/155 to track.