Closed chickenandpork closed 4 years ago
Thanks for your high quality bug report ❤️
It would be interesting if SQLite3 breaks on auto migration... because unit tests are ran against a real SQLite3 backend 🤦 Let me see what's going on...
Hmm, I just pulled gogs/gogs:8be46f3aa2ce (the current gogs/gogs:latest), and I can restart the container as many times as I want, with SQLite3 as database. Something is missing from my environment.
Hello,
I got the same issue here today : Using Gogs with Docker, image gogs/gogs:latest, with sqlite3 database.
Updated today to Docker image c275d7ee307e / git rev 41f56ad05d12520bb90f91889fa979465d0b3d6b
Gogs cannot start with the following errors in log :
gogs.log
2020/04/11 13:55:52 [ INFO] Gogs 0.12.0+dev
2020/04/11 13:55:52 [FATAL] [...o/gogs/internal/route/install.go:75 GlobalInit()] Failed to initialize ORM engine: migrate schemes: duplicate column name: id
gorm.log
2020/04/11 14:15:27 [err] [gogs.io/gogs/internal/db/db.go:161] duplicate column name: id
My last previous gogs/gogs:latest working image was from 2020/04/04 so it should be from git rev bae1d6ccd81cd427382a2456e7c3646bdac9cf46
Thanks @nricheton! Would you be able to dump your SQLite3 table schemas out of your Docker container? My current guess is that there are legacy columns somehow causes GORM to have problems.
Hi @unknwon
Here is the dump :
CREATE TABLE `user` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `lower_name` TEXT NOT NULL, `name` TEXT NOT NULL, `full_name` TEXT NULL, `email` TEXT NOT NULL, `passwd` TEXT NOT NULL, `login_type` INTEGER NULL, `login_source` INTEGER DEFAULT 0 NOT NULL, `login_name` TEXT NULL, `type` INTEGER NULL, `location` TEXT NULL, `website` TEXT NULL, `rands` TEXT NULL, `salt` TEXT NULL, `created_unix` INTEGER NULL, `updated_unix` INTEGER NULL, `last_repo_visibility` INTEGER NULL, `max_repo_creation` INTEGER DEFAULT -1 NOT NULL, `is_active` INTEGER NULL, `is_admin` INTEGER NULL, `allow_git_hook` INTEGER NULL, `allow_import_local` INTEGER NULL, `prohibit_login` INTEGER NULL, `avatar` TEXT NOT NULL, `avatar_email` TEXT NOT NULL, `use_custom_avatar` INTEGER NULL, `num_followers` INTEGER NULL, `num_following` INTEGER DEFAULT 0 NOT NULL, `num_stars` INTEGER NULL, `num_repos` INTEGER NULL, `description` TEXT NULL, `num_teams` INTEGER NULL, `num_members` INTEGER NULL);
CREATE TABLE sqlite_sequence(name,seq);
CREATE UNIQUE INDEX `UQE_user_lower_name` ON `user` (`lower_name`);
CREATE UNIQUE INDEX `UQE_user_name` ON `user` (`name`);
CREATE TABLE `public_key` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `owner_id` INTEGER NOT NULL, `name` TEXT NOT NULL, `fingerprint` TEXT NOT NULL, `content` TEXT NOT NULL, `mode` INTEGER DEFAULT 2 NOT NULL, `type` INTEGER DEFAULT 1 NOT NULL, `created_unix` INTEGER NULL, `updated_unix` INTEGER NULL);
CREATE INDEX `IDX_public_key_owner_id` ON `public_key` (`owner_id`);
CREATE TABLE `access_token` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `uid` INTEGER NULL, `name` TEXT NULL, `sha1` TEXT NULL, `created_unix` INTEGER NULL, `updated_unix` INTEGER NULL);
CREATE UNIQUE INDEX `UQE_access_token_sha1` ON `access_token` (`sha1`);
CREATE INDEX `IDX_access_token_uid` ON `access_token` (`uid`);
CREATE TABLE `two_factor` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `user_id` INTEGER NULL, `secret` TEXT NULL, `created_unix` INTEGER NULL);
CREATE UNIQUE INDEX `UQE_two_factor_user_id` ON `two_factor` (`user_id`);
CREATE TABLE `two_factor_recovery_code` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `user_id` INTEGER NULL, `code` TEXT NULL, `is_used` INTEGER NULL);
CREATE TABLE `repository` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `owner_id` INTEGER NULL, `lower_name` TEXT NOT NULL, `name` TEXT NOT NULL, `description` TEXT NULL, `website` TEXT NULL, `default_branch` TEXT NULL, `size` INTEGER DEFAULT 0 NOT NULL, `use_custom_avatar` INTEGER NULL, `num_watches` INTEGER NULL, `num_stars` INTEGER NULL, `num_forks` INTEGER NULL, `num_issues` INTEGER NULL, `num_closed_issues` INTEGER NULL, `num_pulls` INTEGER NULL, `num_closed_pulls` INTEGER NULL, `num_milestones` INTEGER DEFAULT 0 NOT NULL, `num_closed_milestones` INTEGER DEFAULT 0 NOT NULL, `is_private` INTEGER NULL, `is_bare` INTEGER NULL, `is_mirror` INTEGER NULL, `enable_wiki` INTEGER DEFAULT 1 NOT NULL, `allow_public_wiki` INTEGER NULL, `enable_external_wiki` INTEGER NULL, `external_wiki_url` TEXT NULL, `enable_issues` INTEGER DEFAULT 1 NOT NULL, `allow_public_issues` INTEGER NULL, `enable_external_tracker` INTEGER NULL, `external_tracker_url` TEXT NULL, `external_tracker_format` TEXT NULL, `external_tracker_style` TEXT NULL, `enable_pulls` INTEGER DEFAULT 1 NOT NULL, `pulls_ignore_whitespace` INTEGER DEFAULT 0 NOT NULL, `pulls_allow_rebase` INTEGER DEFAULT 0 NOT NULL, `is_fork` INTEGER DEFAULT 0 NOT NULL, `fork_id` INTEGER NULL, `created_unix` INTEGER NULL, `updated_unix` INTEGER NULL);
CREATE UNIQUE INDEX `UQE_repository_s` ON `repository` (`owner_id`,`lower_name`);
CREATE INDEX `IDX_repository_lower_name` ON `repository` (`lower_name`);
CREATE INDEX `IDX_repository_name` ON `repository` (`name`);
CREATE TABLE `deploy_key` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `key_id` INTEGER NULL, `repo_id` INTEGER NULL, `name` TEXT NULL, `fingerprint` TEXT NULL, `created_unix` INTEGER NULL, `updated_unix` INTEGER NULL);
CREATE UNIQUE INDEX `UQE_deploy_key_s` ON `deploy_key` (`key_id`,`repo_id`);
CREATE INDEX `IDX_deploy_key_key_id` ON `deploy_key` (`key_id`);
CREATE INDEX `IDX_deploy_key_repo_id` ON `deploy_key` (`repo_id`);
CREATE TABLE `collaboration` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `repo_id` INTEGER NOT NULL, `user_id` INTEGER NOT NULL, `mode` INTEGER DEFAULT 2 NOT NULL);
CREATE UNIQUE INDEX `UQE_collaboration_s` ON `collaboration` (`repo_id`,`user_id`);
CREATE INDEX `IDX_collaboration_repo_id` ON `collaboration` (`repo_id`);
CREATE INDEX `IDX_collaboration_user_id` ON `collaboration` (`user_id`);
CREATE TABLE `access` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `user_id` INTEGER NULL, `repo_id` INTEGER NULL, `mode` INTEGER NULL);
CREATE UNIQUE INDEX `UQE_access_s` ON `access` (`user_id`,`repo_id`);
CREATE TABLE `upload` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `uuid` UUID NULL, `name` TEXT NULL);
CREATE UNIQUE INDEX `UQE_upload_uuid` ON `upload` (`uuid`);
CREATE TABLE `watch` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `user_id` INTEGER NULL, `repo_id` INTEGER NULL);
CREATE UNIQUE INDEX `UQE_watch_watch` ON `watch` (`user_id`,`repo_id`);
CREATE TABLE `star` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `uid` INTEGER NULL, `repo_id` INTEGER NULL);
CREATE UNIQUE INDEX `UQE_star_s` ON `star` (`uid`,`repo_id`);
CREATE TABLE `follow` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `user_id` INTEGER NULL, `follow_id` INTEGER NULL);
CREATE UNIQUE INDEX `UQE_follow_follow` ON `follow` (`user_id`,`follow_id`);
CREATE TABLE `action` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `user_id` INTEGER NULL, `op_type` INTEGER NULL, `act_user_id` INTEGER NULL, `act_user_name` TEXT NULL, `repo_id` INTEGER NULL, `repo_user_name` TEXT NULL, `repo_name` TEXT NULL, `ref_name` TEXT NULL, `is_private` INTEGER DEFAULT 0 NOT NULL, `content` TEXT NULL, `created_unix` INTEGER NULL);
CREATE INDEX `IDX_action_repo_id` ON `action` (`repo_id`);
CREATE TABLE `issue` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `repo_id` INTEGER NULL, `index` INTEGER NULL, `poster_id` INTEGER NULL, `name` TEXT NULL, `content` TEXT NULL, `milestone_id` INTEGER NULL, `priority` INTEGER NULL, `assignee_id` INTEGER NULL, `is_closed` INTEGER NULL, `is_pull` INTEGER NULL, `num_comments` INTEGER NULL, `deadline_unix` INTEGER NULL, `created_unix` INTEGER NULL, `updated_unix` INTEGER NULL);
CREATE UNIQUE INDEX `UQE_issue_repo_index` ON `issue` (`repo_id`,`index`);
CREATE INDEX `IDX_issue_repo_id` ON `issue` (`repo_id`);
CREATE TABLE `pull_request` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `type` INTEGER NULL, `status` INTEGER NULL, `issue_id` INTEGER NULL, `index` INTEGER NULL, `head_repo_id` INTEGER NULL, `base_repo_id` INTEGER NULL, `head_user_name` TEXT NULL, `head_branch` TEXT NULL, `base_branch` TEXT NULL, `merge_base` TEXT NULL, `has_merged` INTEGER NULL, `merged_commit_id` TEXT NULL, `merger_id` INTEGER NULL, `merged_unix` INTEGER NULL);
CREATE INDEX `IDX_pull_request_issue_id` ON `pull_request` (`issue_id`);
CREATE TABLE `comment` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `type` INTEGER NULL, `poster_id` INTEGER NULL, `issue_id` INTEGER NULL, `commit_id` INTEGER NULL, `line` INTEGER NULL, `content` TEXT NULL, `created_unix` INTEGER NULL, `updated_unix` INTEGER NULL, `commit_sha` TEXT NULL);
CREATE INDEX `IDX_comment_issue_id` ON `comment` (`issue_id`);
CREATE TABLE `attachment` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `uuid` UUID NULL, `issue_id` INTEGER NULL, `comment_id` INTEGER NULL, `release_id` INTEGER NULL, `name` TEXT NULL, `created_unix` INTEGER NULL);
CREATE UNIQUE INDEX `UQE_attachment_uuid` ON `attachment` (`uuid`);
CREATE INDEX `IDX_attachment_issue_id` ON `attachment` (`issue_id`);
CREATE INDEX `IDX_attachment_release_id` ON `attachment` (`release_id`);
CREATE TABLE `issue_user` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `uid` INTEGER NULL, `issue_id` INTEGER NULL, `repo_id` INTEGER NULL, `milestone_id` INTEGER NULL, `is_read` INTEGER NULL, `is_assigned` INTEGER NULL, `is_mentioned` INTEGER NULL, `is_poster` INTEGER NULL, `is_closed` INTEGER NULL);
CREATE INDEX `IDX_issue_user_uid` ON `issue_user` (`uid`);
CREATE INDEX `IDX_issue_user_repo_id` ON `issue_user` (`repo_id`);
CREATE TABLE `label` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `repo_id` INTEGER NULL, `name` TEXT NULL, `color` TEXT NULL, `num_issues` INTEGER NULL, `num_closed_issues` INTEGER NULL);
CREATE INDEX `IDX_label_repo_id` ON `label` (`repo_id`);
CREATE TABLE `issue_label` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `issue_id` INTEGER NULL, `label_id` INTEGER NULL);
CREATE UNIQUE INDEX `UQE_issue_label_s` ON `issue_label` (`issue_id`,`label_id`);
CREATE TABLE `milestone` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `repo_id` INTEGER NULL, `name` TEXT NULL, `content` TEXT NULL, `is_closed` INTEGER NULL, `num_issues` INTEGER NULL, `num_closed_issues` INTEGER NULL, `completeness` INTEGER NULL, `deadline_unix` INTEGER NULL, `closed_date_unix` INTEGER NULL);
CREATE INDEX `IDX_milestone_repo_id` ON `milestone` (`repo_id`);
CREATE TABLE `mirror` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `repo_id` INTEGER NULL, `interval` INTEGER NULL, `enable_prune` INTEGER DEFAULT 1 NOT NULL, `updated_unix` INTEGER NULL, `next_update_unix` INTEGER NULL);
CREATE TABLE `release` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `repo_id` INTEGER NULL, `publisher_id` INTEGER NULL, `tag_name` TEXT NULL, `lower_tag_name` TEXT NULL, `target` TEXT NULL, `title` TEXT NULL, `sha1` TEXT NULL, `num_commits` INTEGER NULL, `note` TEXT NULL, `is_draft` INTEGER DEFAULT 0 NOT NULL, `is_prerelease` INTEGER NULL, `created_unix` INTEGER NULL);
CREATE TABLE `login_source` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `type` INTEGER NULL, `name` TEXT NULL, `is_actived` INTEGER DEFAULT 0 NOT NULL, `is_default` INTEGER DEFAULT 0 NULL, `cfg` TEXT NULL, `created_unix` INTEGER NULL, `updated_unix` INTEGER NULL);
CREATE UNIQUE INDEX `UQE_login_source_name` ON `login_source` (`name`);
CREATE TABLE `webhook` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `repo_id` INTEGER NULL, `org_id` INTEGER NULL, `url` TEXT NULL, `content_type` INTEGER NULL, `secret` TEXT NULL, `events` TEXT NULL, `is_ssl` INTEGER NULL, `is_active` INTEGER NULL, `hook_task_type` INTEGER NULL, `meta` TEXT NULL, `last_status` INTEGER NULL, `created_unix` INTEGER NULL, `updated_unix` INTEGER NULL);
CREATE TABLE `hook_task` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `repo_id` INTEGER NULL, `hook_id` INTEGER NULL, `uuid` TEXT NULL, `type` INTEGER NULL, `url` TEXT NULL, `signature` TEXT NULL, `payload_content` TEXT NULL, `content_type` INTEGER NULL, `event_type` TEXT NULL, `is_ssl` INTEGER NULL, `is_delivered` INTEGER NULL, `delivered` INTEGER NULL, `is_succeed` INTEGER NULL, `request_content` TEXT NULL, `response_content` TEXT NULL);
CREATE INDEX `IDX_hook_task_repo_id` ON `hook_task` (`repo_id`);
CREATE TABLE `protect_branch` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `repo_id` INTEGER NULL, `name` TEXT NULL, `protected` INTEGER NULL, `require_pull_request` INTEGER NULL, `enable_whitelist` INTEGER NULL, `whitelist_user_i_ds` TEXT NULL, `whitelist_team_i_ds` TEXT NULL);
CREATE UNIQUE INDEX `UQE_protect_branch_protect_branch` ON `protect_branch` (`repo_id`,`name`);
CREATE TABLE `protect_branch_whitelist` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `protect_branch_id` INTEGER NULL, `repo_id` INTEGER NULL, `name` TEXT NULL, `user_id` INTEGER NULL);
CREATE UNIQUE INDEX `UQE_protect_branch_whitelist_protect_branch_whitelist` ON `protect_branch_whitelist` (`repo_id`,`name`,`user_id`);
CREATE TABLE `team` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `org_id` INTEGER NULL, `lower_name` TEXT NULL, `name` TEXT NULL, `description` TEXT NULL, `authorize` INTEGER NULL, `num_repos` INTEGER NULL, `num_members` INTEGER NULL);
CREATE INDEX `IDX_team_org_id` ON `team` (`org_id`);
CREATE TABLE `org_user` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `uid` INTEGER NULL, `org_id` INTEGER NULL, `is_public` INTEGER NULL, `is_owner` INTEGER NULL, `num_teams` INTEGER NULL);
CREATE UNIQUE INDEX `UQE_org_user_s` ON `org_user` (`uid`,`org_id`);
CREATE INDEX `IDX_org_user_uid` ON `org_user` (`uid`);
CREATE INDEX `IDX_org_user_org_id` ON `org_user` (`org_id`);
CREATE TABLE `team_user` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `org_id` INTEGER NULL, `team_id` INTEGER NULL, `uid` INTEGER NULL);
CREATE UNIQUE INDEX `UQE_team_user_s` ON `team_user` (`team_id`,`uid`);
CREATE INDEX `IDX_team_user_org_id` ON `team_user` (`org_id`);
CREATE TABLE `team_repo` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `org_id` INTEGER NULL, `team_id` INTEGER NULL, `repo_id` INTEGER NULL);
CREATE UNIQUE INDEX `UQE_team_repo_s` ON `team_repo` (`team_id`,`repo_id`);
CREATE INDEX `IDX_team_repo_org_id` ON `team_repo` (`org_id`);
CREATE TABLE `notice` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `type` INTEGER NULL, `description` TEXT NULL, `created_unix` INTEGER NULL);
CREATE TABLE `email_address` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `uid` INTEGER NOT NULL, `email` TEXT NOT NULL, `is_activated` INTEGER NULL);
CREATE UNIQUE INDEX `UQE_email_address_email` ON `email_address` (`email`);
CREATE INDEX `IDX_email_address_uid` ON `email_address` (`uid`);
CREATE TABLE `version` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `version` INTEGER NULL);
Thank you very much! I'm taking a look.
@nricheton I looked down the root problem, which is GORM thinks the id
column does not exist by the way it queries the SQLite3 DB, and it could be caused by the file format has changed since your Gogs instance created the SQLite3 DB.
Could you run this query and post the result here?
SELECT name, sql FROM sqlite_master WHERE tbl_name = 'access_token';
The latest master
now only uses GORM to create tables when they are not exist, so should resolve the immediate problem. Docker Hub takes ~30 mins to build a new image.
Here is the result :
access_token|CREATE TABLE `access_token` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `uid` INTEGER NULL, `name` TEXT NULL, `sha1` TEXT NULL, `created_unix` INTEGER NULL, `updated_unix` INTEGER NULL)
UQE_access_token_sha1|CREATE UNIQUE INDEX `UQE_access_token_sha1` ON `access_token` (`sha1`)
IDX_access_token_uid|CREATE INDEX `IDX_access_token_uid` ON `access_token` (`uid`)
gogs/gogs:latest image with the fix works. Thank you @unknwon !
@nricheton Good to know it works :)
For the record, the way GORM to determine if a column exists in SQLite3 (didn't check other dialects) is pretty buggy as of v1.9.12. It will try to find both sql LIKE '%"id" %'
and sql LIKE '%id %'
but not "id" with backquotes and could over match other columns/indices end with "id" (e.g. "uid").
wow, that's a fast fix. Thanks!
Describe the bug Gogs is exiting fatally with a error containing the text at subject:
Gogs version, commit or Docker image SHA broken: Gogs 0.12.0+dev gogs/gogs:8be46f3aa2ce working: Gogs 0.12.0+dev gogs/gogs:39ac52096fc8
Git version
git version 2.22.2
from inside the docker imageOperating system Alpine Linux v3.10.4 from inside the gogs/gogs container
Database sqlite3
To Reproduce
gogs/gogs:39ac52096fc8
gogs/gogs:latest
and restartgogs/gogs:8be46f3aa2ce
Can you reproduce the bug at https://try.gogs.io? No; this seems to be a problem between GORM and the database, or something in that level.
Expected behavior Service starts up, stays running, doesn't fatally quit.
Actual behavior Fatal error as logged.
Screenshots not applicable beyond logs
Similar Bugs gogs/gogs#2747 is a migration, has a similar error, maybe points at a difference in how GORM talks to a MySQL vs sqlite3 DB, but I don't see how it was resolved. This is a sqlite config stopping, then starting up, as opposed to a MySQL stopping, and a sqlite starting up. The fix might apply, but I'm not sure how to guess which ID field is the duplicate.
Additional context
gogs.log
:gorm.log
:serv.log
andxorm.log
empty.I wold guess it seems to be starting, failing, and the s6 supervisor seems to be restarting it, but I don't know which table is corrupt, and I'm log looking at the fatal error and noticing that it's running under s6 supervision.
Backing out to
gogs/gogs:39ac52096fc8
seems to let it go forward, but we all know that running with an unknown schema is a risk.A Naive guess makes me wonder about #6086 which changes the
AutoMigrate
and adds a columnID
but it's a naive guess where the text matches. I'd like to see if I can narrow it down.I would like to try to bisect the various versions uploaded to hub.docker.com but I don't know the sha256 for the versions between
gogs/gogs:39ac52096fc8
andgogs/gogs:8be46f3aa2ce
(you might find them in the build info in https://hub.docker.com/r/gogs/gogs/builds but I'm not sure)