search

iterative / example-repos-dev

Source code and generator scripts for example DVC projects
https://dvc.org/doc
21 stars 13 forks source link

auto-generate CML example repos #100

Closed casperdcl closed 1 year ago

casperdcl commented 2 years ago

Example repos are painful to update everywhere (e.g. iterative/cml.dev#178 touches all example repos). Should auto-generate them from one source similar to how https://github.com/iterative/example-repos-dev works.

casperdcl commented 2 years ago

perhaps best to do this in the existing https://github.com/iterative/example-repos-dev? Could transfer this issue there /CC @iesahin @jorgeorpinel

iesahin commented 2 years ago

Moved and will be tracking here. Do you have any specific requirements for repositories? Would a simple generate.bash script suffice? @casperdcl

iesahin commented 2 years ago

I think you prefer to have a single generate script for all these repositories, right? @casperdcl

casperdcl commented 2 years ago

Yes exactly. Reordered the list a bit to reflect that most should be based on the current https://github.com/iterative/example_cml

iesahin commented 2 years ago

Could you select which of the repository branches you'd like to keep? @casperdcl

I'll rename master branches to main.

casperdcl commented 2 years ago

@iesahin

  1. rename master to main using the GH interface so people get redirected appropriately
  2. we need to keep all branches as they are (i.e. keep the associated PRs open as examples of what PRs look like)
  3. ideally only auto-update one (i.e. the latest) main commit from one unified codebase

I'm open to other suggestions for (3). Basically requirement (2) is quite different from the DVC example repos and I don't see a neat way around it.

iesahin commented 2 years ago

Let me investigate the repositories a bit. AFA I understand you want generate.sh to clone one (private?) repository generate other repositories from that by applying the changes.

I want to use an approach as simple as possible, you'll keep project files in a repository (iterative/example_cml or some base repository) and have a directory structure similar to this: 

   example-cml-generator/
     example-cml/
       main/
       experiment/
       experiment-1/
....
    cml_dvc_case/
       main/
       experiment/
       ....

From these, it will apply cml_dvc_case/main to the base repository and get main branch, then apply experiment to this main branch and get experiment branch, etc.

Please let me know if there are more complex relationships between the repositories.

casperdcl commented 2 years ago

I was thinking:

0x2b3bfa0 commented 2 years ago

Note: as per https://github.com/iterative/cml/issues/890#issuecomment-1106118830, https://gitlab.com/iterative.ai/cml-base-case was broken in every imaginable way.

0x2b3bfa0 commented 1 year ago

Closing in favor of https://github.com/iterative/cml-examples-dev/issues/6

casperdcl commented 1 year ago

@0x2b3bfa0 does this need to be reopened? What's the status?

0x2b3bfa0 commented 1 year ago

@casperdcl, it probably doesnt (?) 🤔🐶

0x2b3bfa0 commented 1 year ago

What's the status?

The new examples are less broken than the old ones, although that doesn't necessarily[^1] imply that they work.

[^1]: Weasel words, read: “they're also broken, but in minor ways by comparison”

casperdcl commented 1 year ago

I'd say this was closed in favour of/migrated to https://github.com/iterative/cml-examples-dev/issues/9 instead :)

0x2b3bfa0 commented 1 year ago

Or maybe https://github.com/iterative/cml-examples-dev/issues/16?