search

google / copybara

Copybara: A tool for transforming and moving code between repositories.
Apache License 2.0
2.13k stars 258 forks source link

best way to handle gradually open-sourcing multiple parts of a repo? #262

Open petergaultney opened 11 months ago

petergaultney commented 11 months ago

I am open-sourcing parts of an internal monorepo, which has a rough structure looking like:

libs/foo libs/bar libs/baz etc.

Today, I want to open-source only libs/foo and libs/bar, as they're in a reasonably stable state and we're ready to share that state with everybody. We also prefer to publish git commits so that internal authors will get public credit for their contributions (and so that, eventually, external authors will also be able to get credit for their contributions, after import into our internal Source of Truth monorepo).

The issue I'm running into is that it's not clear that I have a sustainable workflow here. After this code is open-sourced, we may later decide to add libs/baz to the list of open-sourced libraries. When I add libs/baz to the list of origin_files, I find that copybara perceives no change and makes no change to the destination repo. This is, presumably, because it sees no new git commits in the source repo.

Maybe our best bet when this situation arises will be to essentially recreate the entire destination repo history from scratch, using the same workflow but an empty repository and the new set of origin_files. I am wondering - will our git commit hashes for previously-published commits remain the same in cases where the same exact set of files were changed? Presumably they will be different for any commits where files from libs/baz were present (but not included in the first export). What other side effects might we observe?

Thanks for this tool. It seems quite powerful and I have a feeling I've only scratched the surface of the ways we could tweak the workflow that I now have basically working.

petergaultney commented 11 months ago

The more I think about this, the more it seems like some combination of --force and --init-history is the intended approach when needing to go back and recreate the entire repository history in order to include previously excluded files.

I would be happy to be shown a better way, though. :)