Github Restructuring

[ghindman]

All,

We have some pending product requirements that will necessitate getting some OpenAvnu components into common distros. Towards that end I would like to reopen the discussion on restructuring the OpenAvnu repo to make the various components more consumable by external distributions.

We would like to start with breaking the following folders out into top-level repos under the AVnu github project: kmod/igb lib/igb lib/libavtp daemons/gptp

If github admins can create the top-level repos I will commit to moving the code over and untangling any build/test dependencies so that everyone still builds properly.

From a timing/coordination standpoint, if we do a clone then no exiting project dependencies will be broken until everything is fixed up, the only downside would be changes would have to be tracked between the master and the clone until the master is retired. Alternatively we could either freeze those project until completed, or push all of the changes at once, but the latter would require considerable coordination.

Thoughts/concerns?

[andrew-elder]

@ghindman - we have talked about splitting the repo in the past (sometime in the last 12 months), but at the time we didn't feel the extra maintenance, testing and co-ordination of APs (documentation) was worth the final outcome. It sounds like you are in position where you are motivated and think the re-org will provide definite payoffs?

[ghindman]

@andrew-elder yes, near term we have plans to get the above components integrated into the Clear Linux project https://clearlinux.org and other OSVs like Canonical also have a growing interest in TSN/AVB. The repo structure as it is now makes it difficult for distros to consume the redistributable components, and forces all components into a monolithic release/versioning cycle.

We're willing to do the lift on breaking out the components noted here, and we leave it to other component owners (avtp_pipeline, daemons, etc.) to request the same as it makes sense for them.

[andrew-elder]

@ghindman - is the plan to set up submodules in the main repo?

[ghindman]

By main repo you mean the top level AVnu project? We postponed investigating a new structure until we had agreement with the community. If there is consensus then we can come back with a proposal. Libavtp will be the easy one since I don't believe there are yet dependents for it.

[andrew-elder]

@ghindman - so far you and I are the only ones to comment. If we don't get any push back (or other comments) in 48 hours, I'd suggest you go ahead and come back with a proposal.

[ghindman]

Sounds reasonable. It will likely be a few weeks - we have some preliminary tasks in the queue ahead of this.

[pinealservo]

I don't have any objections, for whatever my say is worth. Anything that makes life easier for the people who are actively contributing sounds like a good thing to me.

FYI, I'm no longer employed by Harman and thus I can't speak with any real authority about it, but I'm happy to continue contributing a small amount of my time to code reviews and maintenance tasks until someone takes away my commit bit.

@ghindman, you might find it fruitful to poke the Intel folks who participate in the Avnu TWG to see if you can get some direction about Avnu's intentions for continued maintenance here from that channel.

[ghindman]

Thanks @pinealservo , yes we're in close coordination with the other Intel folks on the TWG.

[avinash-palleti]

Please check below personal repositories which gives an idea how the restructuring looks at the end,

https://github.com/avinash-palleti/OpenAvnu https://github.com/avinash-palleti/igb_avb https://github.com/avinash-palleti/gptp https://github.com/avinash-palleti/libavtp

To put briefly,

• lib/igb, kmod/igb, daemons/gptp, lib/libavtp folders are removed under OpenAVnu
• lib/igb, kmod/igb are combined and created as single repo and this repo is again added as submodule under OpenAVnu as lib/common is dependent on that.
• lib/libavtp is created as separate repo
• daemons/gptp is created as separate repo

Let me know your suggestions/comments on the proposal.

[andrew-elder]

I looks ok to me. I have a few questions 1) Why isn't mrpd in it's own repo? 2) How will travis CI testing be maintained? 3) What do you think about doing this in stages? Move one repo at a time and make sure CI is still working and give folks a week to comment and then proceed to split off another repo. It will take a little longer, but most likely is more manageable. What do you think? 4) Do we lose commit history when we split the repo? I think there is a way to maintain it, but I haven't researched it.

[ghindman]

1) Why isn't mrpd in it's own repo?

• The rest of OpenAvnu could be broken out as desired by owners of those pieces, we were just starting with the pieces that we depend upon near-term and have some nominal ownership in. Hopefully this initial pass can be an example for other components.

2) How will travis CI testing be maintained?

• maintaining travis CI is the goal. I assumed by keeping all the builds from breaking that it would be invisible to travis - is this not the case?

3) What do you think about doing this in stages? Move one repo at a time and make sure CI is still working and give folks a week to comment and then proceed to split off another repo. It will take a little longer, but most likely is more manageable. What do you think?

• I think we're flexible here, though nobody else yet depends on libavtp, so that one should be invisible. and gPTPd depends on libigb, so I don't know if moving them separate makes sense (it could be done, but would be more interim steps).

4)Do we lose commit history when we split the repo? I think there is a way to maintain it, but I haven't researched it.

• git history should be maintained.

[andrew-elder]

Ok, so what do you need me to do next to support the libavtp move?

Then we could do gPTP and libigb ...

[ghindman]

I think the only remaining opens are: 1) who can create new repos in the Avnu project root 2) how do you want to establish maintainership/commit rights of the new repos? Are there official maintainers of gPTPd and igb_avb where they are now that we need to enlist? Do you want to give someone on my team (Avinash) commit rights for at least the initial move/copy?

[andrew-elder]

I'm happy to create initial root repos. Just tell me what you need.

@christopher-s-hall is the gPTP maintainer. I'm not sure who maintains igb_avb (probably someone at Intel). I'm open to giving an external person additional rights, but I'd still like like to see changes come in as pull requests rather than someone doing merges in the background.

[avinash-palleti]

Do we lose commit history when we split the repo? I think there is a way to maintain it, but I haven't researched it.

For daemons/gptp and lib/libavtp we can maintain full commit history using git-subtree command as they are going to be individual git repos. For lib/igb and kmod/igb, we cant have commit history in new repo as two folders going into single repo.

I'm happy to create initial root repos. Just tell me what you need.

As Gavin suggested we will start with libavtp, Can you please create a new repo libavtp. I will send two PR one to newly created repo and one to OpenAVnu to remove libavtp.

[ghindman]

perhaps we should not combine lib/igb and kmod/igb if it means losing the git history? Nothing requires us to merge them, it was just a bit cleaner since they should always be built together. Is there a preference?

[andrew-elder]

I have created libavtp. I think the description of the repo could be improved since it is not really a full implementation of avtp.

[aguedes]

Thanks, Andrew. IMO, the description is OK since in the README file of the project we specify what AVTP formats are currently supported.

[ethan-stewart-iol]

Combining lib/igb and kmod/igb while keeping git history should be possible (see http://gbayer.com/development/moving-files-from-one-git-repository-to-another-preserving-history/). I assume git-subtree would be used to move one folder, and then the (long) process in the link could be used to move the other.

[andrew-elder]

I would vote for preserving history - however that turns out to be implemented.

[avinash-palleti]

I have created libavtp.

@andrew-elder Can you please create an empty readme file in repo. I cant fork empty repo and send PR.

[andrew-elder]

readme.md, or just readme.txt, or something else?

[avinash-palleti]

Empty readme.md should be fine.

[andrew-elder]

Done, I think...

[avinash-palleti]

Done. Created two PRs one on this repo and other on libavtp repo. Please check. https://github.com/AVnu/OpenAvnu/pull/813 https://github.com/AVnu/libavtp/pull/1

[avinash-palleti]

If everyone is fine with above first set of patches. I will go ahead and submit another patch to move another folder out.

[andrew-elder]

libavtp doesn't integrate Travis CI yet....

[avinash-palleti]

@andrew-elder I see that you have added libavtp repo to travis (https://travis-ci.org/AVnu/libavtp) My PR already has travis files, that needs to be merged to trigger the build. https://github.com/AVnu/libavtp/pull/1/commits/8d645224342ebd9629c9eb35c38e6891ef6efaa4

[andrew-elder]

@avinash-palleti - I didn't do anything - weird!

[aguedes]

I think Travis adds github repos automatically, that's why libavtp repo shows up in https://travis-ci.org/AVnu already. To trigger automatic builds, we need to enable it in the profile settings. @andrew-elder, do you have access to it?

[andrew-elder]

@aguedes - I'll have more of a look at this over the weekend.

[andrew-elder]

I merged libavtp to master. Want branch/merge policy do we want for libavtp? OpenAvnu uses the convention that all pull requests come to open-avb-next. Do we want something similar in libavtp? I don't much mind what is decided as long as it is written up in the readme. Comments?

[avinash-palleti]

I merged libavtp to master.

Thanks @andrew-elder Can you please merge below patch as well https://github.com/AVnu/OpenAvnu/pull/814

And also can you create another new repo igb_avb with readme.md file.

[andrew-elder]

Done

[avinash-palleti]

@andrew-elder Created two other PRs, https://github.com/AVnu/OpenAvnu/pull/815 https://github.com/AVnu/igb_avb/pull/1

815 needs an update on submodule commit SHA after the second PR is merged.

[avinash-palleti]

To trigger automatic builds, we need to enable it in the profile settings.

@andrew-elder Can you please enable for igb-avb repo also.

[avinash-palleti]

@andrew-elder Can you please create another new repo "gptp" under AVnu and enable travis-ci for the same.

[andrew-elder]

Wow - no rest :) Is gptp the best name? That name isn't taken anywhere else?

[avinash-palleti]

There are some repos in external GitHub with gptp name but not under AVnu, i think that should be fine to have gptp name.

[andrew-elder]

ok

[andrew-elder]

Created gptp. It doesn't seem to show up in Travis yet....

[avinash-palleti]

@andrew-elder daemons/gptp folder has around ~250 commits in commit history. We need to rebase all of them on to new repo on top of initial commit. It is giving merge conflicts once for every ~2 commits. It will be difficult to resolve all conflicts. Can you please provide me admin privileges to that repo so that i can do force push all those ~250 commits and later you can reset permissions. Please let me know if there is any other way i can do this.

[andrew-elder]

Are all the conflicts in the readme.md file?

[andrew-elder]

Is it the gptp repo you need admin privileges for?

[andrew-elder]

@avinash-palleti - have you read https://demisx.github.io/git/rebase/2015/07/02/git-rebase-keep-my-branch-changes.html ?

[andrew-elder]

I guess you still need to do a push force after resolving the conflicts?

[jdkoftinoff]

No, don't do it that way. Follow this: https://help.github.com/articles/splitting-a-subfolder-out-into-a-new-repository/

[avinash-palleti]

No, don't do it that way. Follow this: https://help.github.com/articles/splitting-a-subfolder-out-into-a-new-repository/

I followed same steps. Now Problem is,

• We need to run git push origin, where origin is my forked repo
• My forked repo will get updated will all the commit history
• But if i want to create PR it is not allowing as AVnu/gptp has initial commit which is not there in my forked repo
• If we have empty upstream repo, fork is not possible

Its like deadlock. If i have admin rights on AVnu/gptp repo, i can set origin as upstream repo where i can submit all commit history

[andrew-elder]

@avinash-palleti - you should have push permission to gptp

[jdkoftinoff]

git rebase the forked repo on top of the initial commit.