CODE CODE REVISION 2: Keeping CCRP platforms in synch

traveller195 commented 5 months ago

Describe the solution

this issue was discussed in the engineering meeting on 2nd of february, 2024
problem is how to sync local Colouring repository e.g. Colouring Dresden with Core after months of different commits according to language labels, different building attributes and different extended functionalities
option could be to pick specific commits which should be merged to local Colouring platform and handle possible conflicts manually.
Git cherrypick might be useful for this too https://git-scm.com/book/en/v2/Appendix-C%3A-Git-Commands-Patching it should allow you to create a patch by selectively adding commits
other way could be using new fork of Colouring Core and merge each Colouring Dresden commit within last year to it and solve conflict, could maybe be faster way

traveller195 commented 5 months ago

@matkoniecz @popcorndoublefeature @polly64 this is the new issue for collecting experiences how to sync Colouring Core with local Colouring platforms...

matkoniecz commented 4 months ago

Another option is to merge in master upstream branch.

Main benefit here is that future merging will be much easier. While for example manual cherry picking or abandoning source control and manually copy pasting code will make things more complicated in future. Main problem is that doing this may be laborious and require solving many edit conflicts if there were significant changes to code.

It may be better to do it in steps to avoid gigantic merge conflicts - merge in master branch up to some date, resolve conflicts, confirm that platform is working. Then merge in next group of commits.

other way could be using new fork of Colouring Core and merge each Colouring Dresden commit within last year to it and solve conflict, could maybe be faster way

that also works, but will get harder to do as you get more commits to reapply

polly64 commented 4 months ago

Discussion in CCRP March engineering meeting re new revision release. Aim to release Colouring Cities Revision 2 in June 2024 and for all CCRP platforms upgrade to this revision by July 2024 engineering meeting.

@matkoniecz and @mdsimpson42 is there a way you can extract any new code from Colouring Canada, Colouring Germany and Colouring Indonesia and Colouring Australia and Colouring Greece engineers automatically and list in an excel file to start?

polly64 commented 4 months ago

see also https://github.com/colouring-cities/colouring-core/issues/1086 for walkability index inclusion

matkoniecz commented 4 months ago

is there a way you can extract any new code from Colouring Canada, Colouring Germany and Colouring Indonesia and Colouring Australia and Colouring Greece engineers automatically and list in an excel file to start?

If we have access to their code repositories (for example, they published them on github like we are doing) then it likely would be possible to list any diverging commits.

Do we have links to their repositories?

polly64 commented 4 months ago

@matkoniecz and @mdsimpson42 (UK), @ogavalda or @demianAdli, (canada), @allanhawas or @leohatvani (sweden) @jtdimasaka (Philippines), @traveller195 or @popcorndoublefeature (Germany)and @zarashabrina (Indonesia) could you comment below on your thoughts re how you think access & decision making re the core repository should work under each item thanks so it is easy to read and so we we can move on this quite fast? If you agree with the recommendation and have no comments, do just put agree with your username.

a) Who should make decisions on what new content is planned for the next revision? - We suggest a PI group representing Global Hubs be the best? This would keep meetings manageable and decision making fast. (this is marked as core administrators in list added by @matkoniecz and @polly64 below

b) How often should revisions be released? - so all CCRP platforms should update to be in sync- considering the speed with which we are now working. Every 6 months? @matkoniecz agrees or once a year

c) Who should make decisions on new code integration relating significant technical issues? Shall we do through the Monthly engineering group? Shall we give access to all engineers to this group until it gets so big it is not helpful, but for there to be a designated engineering lead or leads for each country where possible just to help speed up discussions/decision making? Or through pull request etc

d) Who should have overall control of the core code repository? Turing for moment but that we need to start talking about longer term collaboration on this too ?

e) Who should have what level access? what are the security risks? How can we best maximise security and collaborative working and prevent any country/individual implementing negative changes - (mitigated already by being an academic initiative). (please recommendation list below - and then we ned to allocated roles to people)

f) Are there standard templates for RSE protocols/best working practise that are commonly used that could be useful to include for new RSEs coming in board? Or are these not needed if we only working with RSE's within academia as ethical frameworks within institutions will already apply?

if you are able to comment as soon as possible on this it would be very helpful and we can crack on at Turing with getting the system set up and written up in the open manual. Many thanks!

[edited during meeting]

matkoniecz commented 4 months ago

could you add a table in a comment below with current relative access and restrictions so we can start setting up these groups and ensure access is adequate but security is also in place to protect code from malicious or accidental negative interventions

role overview:

for specific repository:

https://docs.github.com/en/organizations/managing-user-access-to-your-organizations-repositories/managing-repository-roles/repository-roles-for-an-organization

Read: Recommended for non-code contributors who want to view or discuss your project Triage: Recommended for contributors who need to proactively manage issues, discussions, and pull requests without write access Write: Recommended for contributors who actively push to your project Maintain: Recommended for project managers who need to manage the repository without access to sensitive or destructive actions Admin: Recommended for people who need full access to the project, including sensitive and destructive actions like managing security or deleting a repository

across entire organization:

https://docs.github.com/en/organizations/managing-peoples-access-to-your-organization-with-roles/roles-in-an-organization

1) Organization owners - full access across all repositories in https://github.com/colouring-cities/ (membership includes current Turing engineers)

2) Core administrators - people who need full access to colouring-core repository. They do not have full access across all repositories ( https://github.com/orgs/colouring-cities/teams/core ) For people doing significant work on core code and maintaining repositories, which includes @matkoniecz that has elevated rights in colouring-core and colouring-britain

3) Core contributors - have write permission to colouring-core repository. Note that anyone can submit pull request without this level of access (and it is in fact welcome). Currently no people are at this level.

4) Organisation members - can create new repositories within https://github.com/colouring-cities/ namespace, see all organization members and teams and other rights (see details in table here )

All CCRP international academic partners/engineering teams have access as members.

( Note: some specific repositories may have additional people with elevated access rights. Each CCRP platform may have associated repository with people having admin/write and/or triage permissions. This is coordinated by CCRP principal investigators. )

colouring-cities / colouring-core

CODE CODE REVISION 2: Keeping CCRP platforms in synch #1324