ken1714 / dehazing-sample

画像ファイルを入力し、ヘイズ(霧)を除去した画像ファイルを出力するc++プログラム。
0 stars 0 forks source link

GitLabからGitHubへの移行 #20

Closed ken1714 closed 2 years ago

ken1714 commented 2 years ago

本リポジトリをGitLabからGitHubに移行する際に行った操作をまとめていく。

ken1714 commented 2 years ago
ken1714 commented 2 years ago

ソースコードの移行はCUIで行う。過去のコミット履歴等もすべて移行できるが、コミットメッセージ等の必要な修正は全て終わっているものとする。

ラベル、issueの移行にはhttps://github.com/piceaTech/node-gitlab-2-github/tree/c4951e557df2263049bac439ce746b0deb061f7dのREADMEに記載された手順に沿って作業を実行する。

大まかにまとめると、

  1. ソースコードの移行
  2. GitLab/GitHubでアクセストークンの発行
  3. Node.jsおよびnpmのインストール
  4. 移行設定ファイルの修正
  5. 移行作業の実行
ken1714 commented 2 years ago

1. ソースコードの移行

SSH鍵はGitHubに登録済みとする。GitLabのprogramming-trainingリポジトリをdehazing-sampleリポジトリに移行している。

$ git clone --mirror git@gitlab.com:ken1714/programming-training.git
$ cd programming-training.git
$ git remote add --mirror=push dehazing-sample git@github.com:ken1714/dehazing-sample.git
$ git push dehazing-sample 
ログ詳細
```shell $ git clone --mirror git@gitlab.com:ken1714/programming-training.git $ cd programming-training.git $ git remote add --mirror=push dehazing-sample git@github.com:ken1714/dehazing-sample.git $ git push dehazing-sample Warning: Permanently added the ECDSA host key for IP address 'AAA.BBB.CCC.DDD' to the list of known hosts. Enumerating objects: 297, done. Counting objects: 100% (297/297), done. Delta compression using up to 8 threads Compressing objects: 100% (121/121), done. Writing objects: 100% (297/297), 5.87 MiB | 2.98 MiB/s, done. Total 297 (delta 168), reused 297 (delta 168) remote: Resolving deltas: 100% (168/168), done. To github.com:ken1714/dehazing-sample.git * [new branch] master -> master * [new branch] refs/merge-requests/1/head -> refs/merge-requests/1/head * [new branch] refs/merge-requests/1/merge -> refs/merge-requests/1/merge * [new branch] refs/merge-requests/10/head -> refs/merge-requests/10/head * [new branch] refs/merge-requests/10/merge -> refs/merge-requests/10/merge * [new branch] refs/merge-requests/11/head -> refs/merge-requests/11/head * [new branch] refs/merge-requests/11/merge -> refs/merge-requests/11/merge * [new branch] refs/merge-requests/12/head -> refs/merge-requests/12/head * [new branch] refs/merge-requests/12/merge -> refs/merge-requests/12/merge * [new branch] refs/merge-requests/2/head -> refs/merge-requests/2/head * [new branch] refs/merge-requests/2/merge -> refs/merge-requests/2/merge * [new branch] refs/merge-requests/3/head -> refs/merge-requests/3/head * [new branch] refs/merge-requests/3/merge -> refs/merge-requests/3/merge * [new branch] refs/merge-requests/4/head -> refs/merge-requests/4/head * [new branch] refs/merge-requests/4/merge -> refs/merge-requests/4/merge * [new branch] refs/merge-requests/5/head -> refs/merge-requests/5/head * [new branch] refs/merge-requests/5/merge -> refs/merge-requests/5/merge * [new branch] refs/merge-requests/6/head -> refs/merge-requests/6/head * [new branch] refs/merge-requests/6/merge -> refs/merge-requests/6/merge * [new branch] refs/merge-requests/7/head -> refs/merge-requests/7/head * [new branch] refs/merge-requests/7/merge -> refs/merge-requests/7/merge * [new branch] refs/merge-requests/8/head -> refs/merge-requests/8/head * [new branch] refs/merge-requests/8/merge -> refs/merge-requests/8/merge * [new branch] refs/merge-requests/9/head -> refs/merge-requests/9/head * [new branch] refs/merge-requests/9/merge -> refs/merge-requests/9/merge ```
ken1714 commented 2 years ago

2. GitLab/GitHubでアクセストークンの発行

node-gitlab-2-githubのREADME.mdには

gitlab.token
Go to Settings / Access Tokens. Create a new Access Token with api and read_repository scopes and copy that into the settings.ts

github.token Go to Settings / Developer settings / Personal access tokens. Generate a new token with repo scope and copy that into the settings.ts

と記載されているので、その通りに進めればよい。下記は実際に発行したアクセストークンの権限。

GitLab
![gitlab-access-token](https://user-images.githubusercontent.com/96936578/147867983-50a0460f-e8cd-4b1c-9c91-404cbb3fb660.png)
GitHub
![github-access-token](https://user-images.githubusercontent.com/96936578/147867999-7192a58c-84b1-42e5-a731-256457eff957.png)
ken1714 commented 2 years ago

3.Node.jsおよびnpmのインストール

aptによりnpmをインストールすることは可能であるが、Node.jsのバージョンが12未満の場合、issueおよびmerge requestの移行処理の実行時にエラーが起きてしまう。

$ sudo apt install npm
$ nodejs -v
v10.19.0

Node.jsのv10.19.0を使用した際のnode-gitlab-2-github実行時ログ

$ cd node-gitlab-2-github
$ npm i
$ npm run start

> gitlab-2-github@0.1.5 start /root/node-gitlab-2-github
> node node_modules/ts-node/dist/bin.js ./src/index.ts

==================================
Transferring Milestones
==================================
==================================
Transferring Labels
==================================
Already exists: c++
Already exists: docker
Already exists: python
Already exists: ros
Already exists: setup
Already exists: has attachment
Already exists: gitlab merge request
==================================
Transferring Issues
==================================
Transferring 19 issues.

Migrating issue #1 ('Dockerインストール')...
    ...ERROR while migrating issue #1.
DEBUG:
 TypeError: body.matchAll is not a function
    at /root/node-gitlab-2-github/src/utils.ts:45:24
    at step (/root/node-gitlab-2-github/src/utils.ts:52:23)
    at Object.next (/root/node-gitlab-2-github/src/utils.ts:33:53)
    at /root/node-gitlab-2-github/src/utils.ts:27:71
    at new Promise (<anonymous>)
    at __awaiter (/root/node-gitlab-2-github/src/utils.ts:23:12)
    at Object.exports.migrateAttachments (/root/node-gitlab-2-github/src/utils.ts:36:132)
    at GithubHelper.<anonymous> (/root/node-gitlab-2-github/src/githubHelper.ts:828:36)
    at step (/root/node-gitlab-2-github/src/githubHelper.ts:52:23)
    at Object.next (/root/node-gitlab-2-github/src/githubHelper.ts:33:53)
    -> creating a replacement issue...
    ...ERROR: Could not create replacement issue either!

そのため、Node.jsが12以上となるようにインストールする必要がある。今回はnode-gitlab-2-githubの実行用にDockerコンテナを用意する。

ken1714 commented 2 years ago

下記サイトを参考にしつつ、Node.jsのバージョンを12.0.0とする。

使用したDockerfile

FROM ubuntu:20.04
ENV DEBIAN_FRONTEND=noninteractive
ENV TZ=Asia/Tokyo

# Install npm and node.js
RUN apt-get update && apt-get install -y curl
RUN apt-get install -y npm nodejs

# Install n and the node.js v12.0.0
RUN npm install -g n
RUN n 12.0.0

Dockerイメージのビルドおよびコンテナの起動

$ cd node-gitlab-2-github
$ docker image build . -t node-gitlab2github-image
$ docker run -it -v $PWD:/root/node-gitlab-2-github --name node-gitlab2github node-gitlab2github-image bash
ken1714 commented 2 years ago

余談ではあるが、2022/01/02現在の最新のNode.jsであるv17.3.0を使用してnode-gitlab-2-githubの実行を試みると失敗した。

$ npm i
npm WARN old lockfile 
npm WARN old lockfile The package-lock.json file was created with an old version of npm,
npm WARN old lockfile so supplemental metadata must be fetched from the registry.
npm WARN old lockfile 
npm WARN old lockfile This is a one-time fix-up, please be patient...
npm WARN old lockfile 

up to date, audited 221 packages in 6s

24 packages are looking for funding
  run `npm fund` for details

3 moderate severity vulnerabilities

To address all issues, run:
  npm audit fix

Run `npm audit` for details.
$ npm run start

> gitlab-2-github@0.1.5 start
> node node_modules/ts-node/dist/bin.js ./src/index.ts

node:internal/modules/cjs/loader:936
  throw err;
  ^

Error: Cannot find module '/root/node-gitlab-2-github/node_modules/ts-node/dist/bin.js'
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:933:15)
    at Function.Module._load (node:internal/modules/cjs/loader:778:27)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
    at node:internal/main/run_main_module:17:47 {
  code: 'MODULE_NOT_FOUND',
  requireStack: []
}

Node.js v17.3.0
ken1714 commented 2 years ago

4. 移行設定ファイルの修正

projectIdにnullを指定すると、GitLab内の各リポジトリのprojectIdが表示されるので、移行したいprojectIdを指定して再度実行する。

import Settings from './src/settings';

export default {
  gitlab: {
    // url: 'https://gitlab.com',
    token: '(GitLabのアクセストークン),
    projectId: (表示されたprojectIdを指定。1回目の実行時はnullとする),
    sessionCookie: null,
  },
  github: {
    // baseUrl: 'api.github.com',
    owner: 'ken1714',
    token: '(GitHubのアクセストークン)',
    repo: 'dehazing-sample',
  },
  // s3: {
  //   accessKeyId: '{{accessKeyId}}',
  //   secretAccessKey: '{{secretAccessKey}}',
  //   bucket: 'my-gitlab-bucket',
  // },
  usermap: {
    'ken1714': 'ken1714',
    '(GitLabでの表示名)': 'ken1714',  // ここはおそらく不要
  },
  projectmap: {
    'ken1714/programming-training': 'ken1714/dehazing-sample',
  },
  conversion: {
    useLowerCaseLabels: true,
  },
  transfer: {
    milestones: false,
    labels: false,
    issues: false,
    mergeRequests: true,
  },
  debug: false,
  usePlaceholderIssuesForMissingIssues: true,
  useReplacementIssuesForCreationFails: true,
  useIssuesForAllMergeRequests: false,
  filterByLabel: null,
  skipMatchingComments: [],
  mergeRequests: {
    logFile: './merge-requests.json',
    log: false,
  },
} as Settings;
ken1714 commented 2 years ago

5. 移行作業の実行

Dockerコンテナ上で下記コマンドを実行。

$ cd node-gitlab-2-github
$ npm i
$ npm run start

マージリクエストについては、同名のタイトルのissueをすでに移行してしまったため、名前が重複して移行できなかった。移行元のマージリクエストにはほとんどコメントがなかったので、ひとまずこのままでOKとする。

実行ログ
```shell $ npm i npm WARN @octokit/plugin-throttling@3.5.2 requires a peer of @octokit/core@^3.5.0 but none is installed. You must install peer dependencies yourself. audited 222 packages in 1.589s found 1 moderate severity vulnerability run `npm audit fix` to fix them, or `npm audit` for details $ npm run start > gitlab-2-github@0.1.5 start /root/node-gitlab-2-github > node node_modules/ts-node/dist/bin.js ./src/index.ts ================================== Transferring Issues ================================== Transferring 19 issues. Migrating issue #1 ('Dockerインストール')... Migrating issue comments... ...Done creating issue comments (migrated 3 comments, skipped 0 comments) ...DONE migrating issue #1. Migrating issue #2 ('Pythonで機械学習を行う環境を構築')... Migrating issue comments... ...Done creating issue comments (migrated 15 comments, skipped 0 comments) ...DONE migrating issue #2. Migrating issue #3 ('ROSインストール')... Migrating issue comments... ...Done creating issue comments (migrated 4 comments, skipped 0 comments) ...DONE migrating issue #3. Migrating issue #4 ('ROS Tutorials')... Migrating issue comments... ...Done creating issue comments (migrated 6 comments, skipped 0 comments) ...DONE migrating issue #4. Migrating issue #5 ('Ubuntu20.04インストール(デュアルブート)')... Migrating issue comments... ...Done creating issue comments (migrated 11 comments, skipped 0 comments) ...DONE migrating issue #5. Migrating issue #6 ('Dockerfile作成チュートリアル')... Migrating issue comments... ...Done creating issue comments (migrated 12 comments, skipped 0 comments) ...DONE migrating issue #6. Migrating issue #7 ('ヘイズ除去サンプルプログラム: main関数の実装')... Migrating issue comments... ...Done creating issue comments (migrated 7 comments, skipped 0 comments) ...DONE migrating issue #7. Migrating issue #8 ('ヘイズ除去サンプルプログラム: Google Testの導入')... Migrating issue comments... ...Done creating issue comments (migrated 8 comments, skipped 0 comments) ...DONE migrating issue #8. Migrating issue #9 ('ヘイズ除去のCPU実装: DarkChannelPriorクラスの実装')... Migrating issue comments... ...Done creating issue comments (migrated 5 comments, skipped 0 comments) ...DONE migrating issue #9. Migrating issue #10 ('ヘイズ除去サンプルプログラム: CI環境の構築')... Migrating issue comments... ...Done creating issue comments (migrated 2 comments, skipped 0 comments) ...DONE migrating issue #10. Migrating issue #11 ('ヘイズ除去のCPU実装: TransmissonMapクラスの実装')... Migrating issue comments... ...Done creating issue comments (migrated 6 comments, skipped 0 comments) ...DONE migrating issue #11. Migrating issue #12 ('ヘイズ除去のCPU実装: AtmosphericLightクラスの実装')... Migrating issue comments... ...Done creating issue comments (migrated 5 comments, skipped 0 comments) ...DONE migrating issue #12. Migrating issue #13 ('ヘイズ除去のCPU実装: RemoveHazeクラスの実装')... Migrating issue comments... ...Done creating issue comments (migrated 5 comments, skipped 0 comments) ...DONE migrating issue #13. Migrating issue #14 ('ヘイズ除去のCPU実装: 透過率マップの計算の修正')... Migrating issue comments... ...Done creating issue comments (migrated 5 comments, skipped 0 comments) ...DONE migrating issue #14. Migrating issue #15 ('ヘイズ除去サンプルプログラムのリファクタ')... Migrating issue comments... ...Done creating issue comments (migrated 9 comments, skipped 0 comments) ...DONE migrating issue #15. Migrating issue #16 ('ヘイズ除去のCPU実装: Dehazingクラスの実装')... Migrating issue comments... ...Done creating issue comments (migrated 6 comments, skipped 0 comments) ...DONE migrating issue #16. Migrating issue #17 ('CUDAキャッチアップ')... Migrating issue comments... ...Done creating issue comments (migrated 4 comments, skipped 0 comments) ...DONE migrating issue #17. Migrating issue #18 ('各オブジェクトをスマートポインタで管理するよう修正')... Migrating issue comments... ...Done creating issue comments (migrated 5 comments, skipped 0 comments) ...DONE migrating issue #18. Migrating issue #19 ('過去コミットのAuthor及びメールアドレスを書き換える')... Migrating issue comments... ...Done creating issue comments (migrated 6 comments, skipped 0 comments) ...DONE migrating issue #19. DONE creating issues. Statistics: Total nr. of issues: 19 Nr. of used placeholder issues: 0 Nr. of used replacement issues: 0 Nr. of issue migration fails: 0 ================================== Transferring Merge Requests ================================== Transferring 12 merge requests Gitlab merge request already exists (as github issue): 1 - Dockerfile作成チュートリアル Gitlab merge request already exists (as github issue): 2 - ヘイズ除去サンプルプログラム: main関数の実装 Gitlab merge request already exists (as github issue): 3 - ヘイズ除去サンプルプログラム: Google Testの導入 Gitlab merge request already exists (as github issue): 4 - ヘイズ除去のCPU実装: DarkChannelPriorクラスの実装 Gitlab merge request already exists (as github issue): 5 - ヘイズ除去のCPU実装: TransmissonMapクラスの実装 Gitlab merge request already exists (as github issue): 6 - ヘイズ除去のCPU実装: AtmosphericLightクラスの実装 Gitlab merge request already exists (as github issue): 7 - ヘイズ除去のCPU実装: 透過率マップの計算の修正 Gitlab merge request already exists (as github issue): 8 - ヘイズ除去のCPU実装: RemoveHazeクラスの実装 Gitlab merge request already exists (as github issue): 9 - ヘイズ除去サンプルプログラムのリファクタ Gitlab merge request already exists (as github issue): 10 - ヘイズ除去のCPU実装: Dehazingクラスの実装 Gitlab merge request already exists (as github issue): 11 - 各オブジェクトをスマートポインタで管理するよう修正 Gitlab merge request already exists (as github issue): 12 - 過去コミットのAuthor及びメールアドレスを書き換える Transfer complete! ```