Migrate away from Google Code #2458

Open GoogleCodeExporter opened 9 years ago

Google code is going to be shut down in January 2016, and will turn read-only 
in August 2015 [1].

I suggest moving to gitlab.com [2] instead of the obvious choice github.com,
to support the free software GitLab.

I already did some small contributions to the F-Droid project hosted at 
gitlab.com, and it seems the workflow is very similar to GitHub.

[1] http://google-opensource.blogspot.de/2015/03/farewell-to-google-code.html
[2] https://about.gitlab.com/gitlab-com/

Original issue reported on code.google.com by robert.p...@mykolab.com on 13 Mar 2015 at 11:48

I see that there already is https://github.com/TortoiseGit/TortoiseGit .

Still I want to mention that GitLab has a GitHub importer: 
https://about.gitlab.com/images/7_7/import.png

Original comment by robert.p...@mykolab.com on 13 Mar 2015 at 12:05

We already have that GitHub mirror for a long time - we only used it as a 
read-only mirror as well as an easy way to receive and discuss pull requests - 
lots of users have an account on GitHub so it's quite easy for them to create 
pull requests. Wiki and issue management are disabled there right now.

Main question is: Where to move our main repository and issue management.

It's planned that tortoisegit.org should hold the main webpage, the FAQ and 
download instructions... But setting up our own issue management system and 
maybe also git server is an overkill and might also hinter people from filing 
issues as a special registration would be required then.

Migrating to GitLab is also an option (I have my own gitlab server running for 
my personal projects and its quite similar to GitHub). I like the protected 
branch feature. Does GitLab support CI-tools like AppVeyor? maybe someone has 
contacts to GitLab and can ask if they are planning to provide a GoogleCode 
importer... - So, if we want to use that, people have to create accounts there 
for filing issues (and providing merge requests) - we also need a way to get 
all filed issues there. (Even if we decide to move our issue tracker to GitLab, 
I would still vote for keeping the GitHub mirror. With Git that's not a 
problem, SVN might be more problematic...) Last point is prolematic with all 
other services as the numbering will not be the same (we need a translation 
mapping in any case).

SourceForge is ruled our as the issue tracker is really broken there and as it 
doesn't provide a nice PR/MR management.

Mailing lists are not affected as Google Groups are unaffected - lets see if 
those get closed next year.

In any case we don't have to rush and should deeply consider all pros and cons.

(Good luck that we recently moved the auto update check url to our own website)

Original comment by sstrickr...@googlemail.com on 13 Mar 2015 at 9:17

Changed state: Discuss
Added labels: Priority-High, Type-Task
Removed labels: Priority-Medium, Type-Defect

Does the "Export to GitHub" button export the issues? Or it just export the 
source code and vision history?

Original comment by universe...@gmail.com on 15 Mar 2015 at 9:10

I want to know why that button is showed for normal user, so just having a try.
Oops~ it's failed. (see attached file.)

Original comment by yuelinho...@gmail.com on 15 Mar 2015 at 10:00

Attachments:

export_err.png

https://code.google.com/p/support-tools/wiki/GitHubExporterFAQ#Error_%22Project_
cannot_be_migrated_because_it_has_too_many_i

Original comment by yfdyh000 on 15 Mar 2015 at 12:00

The FAQ reads like a nightmare: if a repository with same name already exists 
one has to manually renamt the existing one...

Original comment by sstrickr...@googlemail.com on 15 Mar 2015 at 1:44

Another problem with GitHub is that there are no attachments allowd for issues

Original comment by sstrickr...@googlemail.com on 15 Mar 2015 at 1:58

According to https://about.gitlab.com/gitlab-ce-features/, GitLab provides 
issue attachments.

Maybe we could even contact GitLab (i.e. Sytse Sijbrandij, sytse@gitlab.com) 
directly and ask for help with the issue import? I am sure they would be very 
interested in hosting TortoiseGit. (And I guess then we might even be able to 
preserve the issue numbers.)

Original comment by robert.p...@mykolab.com on 15 Mar 2015 at 5:30

We're very open to helping you import to GitLab.com, there are issue importers 
for BitBucket and GitHub.com is GitLab, it would be awesome if we can extend 
that to Google Code together.

Original comment by sytses on 16 Mar 2015 at 1:11

This inofficial migration tool sounds promising: 
https://gitlab.com/o9000/google-code-to-gitlab .

(Found via https://code.google.com/p/support-tools/wiki/IssueExporterTool .)

Original comment by robert.p...@mykolab.com on 19 Mar 2015 at 7:31

> Does GitLab support CI-tools like AppVeyor?

Yes, see https://about.gitlab.com/gitlab-ci/ .

Original comment by robert.p...@mykolab.com on 19 Mar 2015 at 7:35

For issue reporting:
gitlab can use google oauth to login. So no separate registration is required.

Original comment by ch3co...@gmail.com on 19 Mar 2015 at 11:38

IIRC there is no support from AppVeyor and I'm not aware of any GitLib CI 
provides which supports VS2013

Original comment by sstrickr...@googlemail.com on 19 Mar 2015 at 11:06

There is nothing wrong with obvious choices, so github is actually a good 
choice.

Original comment by purgina...@gmail.com on 20 Mar 2015 at 10:09

I tried google takeout and I got all issues, however, without attachments. Does 
someone know how to automatically get those?

GitHub:
- No binary attachments for issues
- Issue numbering: We lose all our current watchers/forks when we re-create the 
repository (and also all current pull requests) OR we have issue numbers which 
are not unique (the google code issue ids get in each other's way with the 
current pull requests; we could ignore that and provide our own http redirector 
which will redirect to the right numbers on github).
+ AppVeyor works for pull requests

GitLab:
- Not supported by AppVeyor (AppVeyor can check master and mail about results, 
but cannot handle Merge Requests like on GitHub with displaying their results 
on the merge page)
+ Login using Google Accounts
+ Recently added support for binary issue attachments, but still no way to 
import them (https://gitlab.com/o9000/google-code-to-gitlab/issues/1)

Open points:
* How to import the issues in a smooth way?
* How to downlaod issue attachments from google code?
* How to handle issue attachments in the issues?
* We need a nice layout for our tortoisegit.org website.

In any case: Even if we decide to use GitLab we can still have our current 
GitHub repository as a mirror (and maybe also for handling GitHub PRs)...

Original comment by sstrickr...@googlemail.com on 27 Mar 2015 at 12:49

AppVeyor is planning to better support GitLab, see 
http://help.appveyor.com/discussions/suggestions/632-better-integrate-with-gitla
b.

Original comment by robert.p...@mykolab.com on 28 Mar 2015 at 9:53

Incremental update:
* I was able to download all issues and also all issue attachments from 
GoogleCode
* It seems as if Google provides stable links for old issue attachments 
(https://code.google.com/p/support-tools/wiki/IssueMirror)
* AppVeyor opened an issue for supporting GitLab

All current info combined:
GitHub:
- No binary attachments for new issues (Old issue attachments are/can be linked 
to Google storage services)
- Issue numbering: We lose all our current watchers/forks if we re-create the 
repository (and also all current pull requests) OR we have issue numbers which 
are not unique (the google code issue ids get in each other's way with the 
current pull requests on GitHub; we could ignore that and provide our own http 
redirector which will redirect to the right numbers on github.
+ AppVeyor works for pull requests
* On GitHub we would need to use a python script, because Google cannot import 
> 1000 issues automatically

GitLab:
+ GitLab is working on an importer for issues (including attachments, see 
https://gitlab.com/gitlab-org/gitlab-ce/issues/1257)
- Not supported by AppVeyor yet (AppVeyor can check master and mail about 
results, but cannot handle Merge Requests like on GitHub with displaying their 
results on the merge page) - as mentioned above, AppVeyor opened an issue for 
supporting GitLab
+ Login using Google accounts possible
+ Support binary issue attachments

Open points:
* Decide whether to go to GitHub or GitLab for our main repository and issue 
tracker
* We need a nice layout for our tortoisegit.org website

In any case: Even if we decide to use GitLab we can still have our current 
GitHub repository as a mirror (and maybe also for handling GitHub PRs) as we do 
now...

Original comment by sstrickr...@googlemail.com on 31 Mar 2015 at 1:53

Some more thoughs on GitLab:
* If you're not registered its not possible to search for (public) projects 
(you need to know the exact link)
* Creating pull requests from a branch in a forked repository is not as 
intuitive as on GitHub (at least for me)

If you want to have a look at GitLab, here is a test repo: 
https://gitlab.com/tortoisegit/tortoisegit/, the page for the project is this 
https://gitlab.com/tortoisegit/

Original comment by sstrickr...@googlemail.com on 31 Mar 2015 at 2:03

[deleted comment]

I was in contact with GitHub support and they have no plans for adding support 
for arbitrary file attachments.

Original comment by sstrickr...@googlemail.com on 8 Apr 2015 at 8:25

Please use the obvious choice. Most causal contributors are already familiar 
with Github and it is a pain in the a** to get accustomed to another platform 
(no matter how easy it is).

Original comment by addit...@networksunshine.de on 8 Apr 2015 at 8:26

ZenHub might be a solution for arbitrary attachments 
(http://stackoverflow.com/a/24944041/21728) - it is free for OSS projects.

I would prefer GitHub just because of the familiarity which I think would be 
the case for most people but if you devs will be happy on some other platform 
for some specific reasons, it's ultimately you who will spend the most time on 
it so I'm sure you will choose well.

Original comment by borekb on 8 Apr 2015 at 9:26

[deleted comment]

Obviously, please use GitHub.

Original comment by apav...@gmail.com on 9 Apr 2015 at 12:35

BitBucket provides downloads, is supported by AppVeyor, allows attachments to 
issues, and just is more "professional" in a variety of ways imho

Original comment by zar...@gmail.com on 9 Apr 2015 at 12:46

 I have to say GitHub is a pretty good choice. The PR handling and the interface is great. If you're planning to host your own bug tracker, that solves the problem of no attachments in issue reports.

So I vote GitHub.

Original comment by Exodia...@gmail.com on 9 Apr 2015 at 1:42

I vote github

Original comment by luiscarl...@gmail.com on 9 Apr 2015 at 1:54

@zarglu: We don't need downloads, we already have our own infrastructure for us.

Hosting our own bug tracker has the same issues as GitHub vs. GitLab 
discussion: A new login is required.

Right now my favourite is GitLab (as the main repository and issue tracker) - 
while still having a mirror on GitHub for pull requests. Being able to handle 
arbitrary attachments is key for us as we often request log files, 
repositories, or other files. - Hosting on DropBox or so is no option as those 
are not persistent and not connected to the issue.

Original comment by sstrickr...@googlemail.com on 9 Apr 2015 at 3:45

Git hub support attach image. It should match most user case.

Original comment by lzn...@gmail.com on 9 Apr 2015 at 3:50

@sstrickroth Is it possible to delete current github mirror so that we can 
reuse TortoiseGit/TortoiseGit.git on github?

Original comment by ch3co...@gmail.com on 9 Apr 2015 at 4:06

@ch3cooli: Yeah, deleting is possible, but then we lose all current pull 
requests and all watchers.

Original comment by sstrickr...@googlemail.com on 9 Apr 2015 at 5:20

Transfer rather than delete it if necessary.
https://help.github.com/articles/transferring-a-repository/

https://github.com/gorhill/uBlock has done similar actions in recently.

Original comment by yfdyh000 on 9 Apr 2015 at 7:43

I vote github, is the natural evolution.
People already registered and familiar with the workflow.

Original comment by githubr...@gmail.com on 10 Apr 2015 at 9:20

It's impressive how many people are voting for GitHub or saying it's the 
obvious choice. - We don't receive much pull requests provided by people 
outside the core team there, so "people are familiar with the workflows" is not 
a perfect argument. Even if we decide to use any other platform, PRs on GitHub 
will still be possible. As GitLab is open-source we might have more control or 
even ways to create our own pull-requests for the platform itself.

As Git is a decentralized VCS the whole discussion boils down to "where to host 
the official issue tracker" - we still have our first repository on repo.oc.cz 
as a mirror and will also keep GitHub as a mirror. So, an account is only 
needed for creating issues - retrieving as well as browsing the source or 
browsing the issues doesn't require any account. Right now, people need a 
Google account for creating issues.

@yfdyh000 I don't see how transferring a repository will help us. You mean we 
rename the current one and import a new one?

Btw. some statistics: Of our approx. 2600 issues approx. 655 have attachments 
(images and binary files combined).

Regardless of the issue tracker, we still need a webdesigner for 
tortoisegit.org...

Original comment by sstrickr...@googlemail.com on 10 Apr 2015 at 9:48

@sstrickroth Yes, rename / transferring the current repo and create or import 
new one instead of delete it, if we necessary.
Maybe it's not necessary, you may only be in response to @ch3cooli 's question.

Original comment by yfdyh000 on 10 Apr 2015 at 9:56

A few ideas for tortoisegit.org

Original comment by sstrickr...@googlemail.com on 10 Apr 2015 at 10:25

Attachments:

I would suggest bitbucket instead of github. As an opensource project you can 
also get Jira from Altassian for free and imho that beats anything github has 
to offer.
Bitbucket (at least on the opensource/academic license) provides greater 
collaborative and team functionality than github.

In regards to basic functionality, github, gitlab and bitbucket offer all of 
the same basic functionality so it's a matter of preference for the people that 
will actually be using the repository.

Original comment by alka.set...@gmail.com on 11 Apr 2015 at 3:01

Is there a reason I'm overlooking that makes GitHub a bad choice, despite being 
the "obvious choice"? Looks like GitLab and Bitbucket are good choices when you 
have many private repositories, which isn't the case here, right?

Binary attachments on issues is merely a quirk when compared to the community 
of developers and designers you immerse yourself into. Hosting elsewhere only 
leads to isolation and therefore, less overall collaboration. Doesn't it make 
sense to make the decision based on collaboration, rather than technical 
features that, to me, seem like nice-to-haves?

I know a lot of people who aren't git experts, but are good developers. It's 
not that they won't stray from GitHub, it's more about weighing the options of 
learning a new ecosystem / system to make a small fix vs. spending their time 
actually writing code in another project, many will simply choose the latter.

Am I missing something? I'm genuinely curious about why open source projects 
where collaboration is a crucial component stray from "mainstream."

Original comment by espo58 on 11 Apr 2015 at 3:46

@espo58: You made some good points. Having a separate platform could detain 
people from reporting issues - for pull requests this isn't a problem since 
we'll keep a GitHub mirror

I don't like GitHub for not having binary attachments, as I wanted to report 
several issues for on different projects which are hosted there and where I 
could not attach logs or other example files. It's a pain to upload files 
somewhere else - and then - the files might vanish after some time...

I'm going to test the github importer today just to see how the results would 
look like...

Original comment by sstrickr...@googlemail.com on 11 Apr 2015 at 3:59

The default issue importer introduces issue id mismatches :(
https://github.com/TortoiseGit/import-test/issues/30

Original comment by sstrickr...@googlemail.com on 11 Apr 2015 at 4:22

The github importer imports deleted comments: 
https://code.google.com/p/support-tools/issues/detail?id=72

Original comment by sstrickr...@googlemail.com on 11 Apr 2015 at 4:47

And the script doesn't handle attachments AT ALL! :( It's a pain!

Original comment by sstrickr...@googlemail.com on 11 Apr 2015 at 5:36

The google migration script doesn't migrate label, status or title changes: 
https://code.google.com/p/support-tools/issues/detail?id=73

Original comment by sstrickr...@googlemail.com on 11 Apr 2015 at 6:45

OK, importing to GitHub doesn't work automatically and has several issues:
 * we cannot use the "export to github" button as we have > 1000 issues
 * no attachments migrated (https://code.google.com/p/support-tools/issues/detail?id=10), patch created
 * delted comments are migrated (https://code.google.com/p/support-tools/issues/detail?id=72), patch created
 * mismatching IDs as deleted issues are ignored instead of "dummy issues" added (https://code.google.com/p/support-tools/issues/detail?id=74), no patch available so far
 * label, title and status changes are not migrated - "marked duplicate" issues are just closed an no pointer to the other issue is inserted (https://code.google.com/p/support-tools/issues/detail?id=73), no patch available

Wasted ~3 hours on this now -> all these points do work for the GitLab 
importer...

Original comment by sstrickr...@googlemail.com on 11 Apr 2015 at 6:55

And another issue with the GitHub exporter:
 * all usernames are replaced by the repository owner (https://code.google.com/p/support-tools/issues/detail?id=28), no patch available

Original comment by sstrickr...@googlemail.com on 11 Apr 2015 at 8:45

Another 3 hours later:
I created patches for all of these issues (excluding the "deleted issues" 
issue, I created dummy issues manually in the Google Takeout export file) and 
migrating to GitHub might work. Importing is really slow, we can only import 
one comment every two seconds as GitHub seems to sort comments by creation time 
-> importing the first 1000 issues lasted ~5 hours.

I still have two issues:
 * Inlining images from storage.googleapis.com does not work, as visible here:
https://github.com/TortoiseGit/import-test2/issues/1045
 * Linking to issues which do not yet exist not possible, as visible here:
https://github.com/TortoiseGit/import-test2/issues/960

Original comment by sstrickr...@googlemail.com on 11 Apr 2015 at 10:13

[deleted comment]

+infinity for Git __HUB___

- everyone is there
- we're all familiar with it
- it has won the war. lets stop this beta vs vhs fight.

** most people tend to be on GitHub for public (free) repo's and BitBucket for 
their (personal) private repo's. This is a public OSS - so goto GH please

keep up the good work!

Original comment by WDpurekr...@gmail.com on 14 Apr 2015 at 12:40

I think it's hyperbole to say GitHub has "won the war". I have been in two 
major projects/companies which have left GitHub because of its shortcomings. I 
don't hate GitHub in any way, but just because it's the largest hosting service 
by volume doesn't mean that we need to have this project on it because - as 
some people put it - it's the biggest provider. GitHub doesn't make using Git 
_that_ much easier (it's gonna be the same with BitBucket and GitLab), and 
there sure as hell aren't going to be any significant loss of community updates 
because it's on BitBucket or GitLab.

Go with the most appropriate hosting service, whether it's GitHub, GitLab, or 
BitBucket, but ignore the advice about GitHub being mainstream.

Original comment by matthew....@gmail.com on 14 Apr 2015 at 9:21

My company maintains its own gitlab services running on DigitalOcean servers 
and I find the monthly costs are reasonable. The gitlab resources to perfectly 
great for what I do. 

It's probably about $25/month for the DigitalOcean. Another $150/year for the 
SSL certificate. And the $10/year for domain name. Plus hosting costs. I've 
used BlueHost for as long as I can remember paying someone to do so and I'm 
happy with what I get for the price.

I'm a huge fan of JFrog's Artifactory and I've looked into bintray a few times 
but never got to a point where I needed it in the "cloud". But it's also a 
potential contender.

Original comment by stayl...@gmail.com on 14 Apr 2015 at 2:47

guberm / tortoisegit

Migrate away from Google Code #2458