search

sourcegraph / sourcegraph-public-snapshot

Code AI platform with Code Search & Cody
https://sourcegraph.com
Other
10.1k stars 1.28k forks source link

Auto-upgrade Broken in v5.3.0 #60528

Closed DaedalusG closed 8 months ago

DaedalusG commented 8 months ago

Steps to reproduce:

  1. Attempt an upgrade of Sourcegraph using Autoupgrade to v5.3.0
  2. Migrator will attempt to runa multiversion upgrade and encounters an issue in the construction of the stitched-migration graph
➜  docker-compose git:(v5.3.0) ✗ dkr ps -a
CONTAINER ID   IMAGE                                         COMMAND                  CREATED          STATUS                             PORTS                                                         NAMES
039c93883349   sourcegraph/frontend:5.3.0                    "/sbin/tini -- /fron…"   36 seconds ago   Created                                                                                          sourcegraph-frontend-0
674256c1cd2d   sourcegraph/frontend:5.3.0                    "/sbin/tini -- /fron…"   36 seconds ago   Up 2 seconds (health: starting)                                                                  sourcegraph-frontend-internal
f0920deb76d1   sourcegraph/migrator:5.3.0                    "/sbin/tini -- /migr…"   36 seconds ago   Exited (0) 20 seconds ago                                                                        migrator
fc9c931c87b3   caddy:2.7.6-alpine                            "caddy run --config …"   36 seconds ago   Up 34 seconds                      0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp, 443/udp, 2019/tcp   caddy
734b70ce1721   sourcegraph/indexed-searcher:5.3.0            "/entry.sh"              36 seconds ago   Up 35 seconds (healthy)                                                                          zoekt-webserver-0
e216798df4b1   sourcegraph/codeintel-db:5.3.0                "/postgres.sh"           36 seconds ago   Up 34 seconds (healthy)            5432/tcp                                                      codeintel-db
f3ab063c907d   sourcegraph/codeinsights-db:5.3.0             "/postgres.sh"           36 seconds ago   Up 34 seconds (healthy)            5432/tcp                                                      codeinsights-db
28379923000c   sourcegraph/cadvisor:5.3.0                    "/usr/bin/entrypoint…"   36 seconds ago   Up 3 seconds                       48080/tcp                                                     cadvisor
22928f101e17   sourcegraph/gitserver:5.3.0                   "/sbin/tini -- /gits…"   36 seconds ago   Up 34 seconds                                                                                    gitserver-0
13fdaf7977a9   sourcegraph/redis-store:5.3.0                 "redis-server /etc/r…"   36 seconds ago   Up 35 seconds                                                                                    redis-store
c9b38c1a5f2a   sourcegraph/syntax-highlighter:5.3.0          "/bin/sh -c '/usr/lo…"   36 seconds ago   Up 34 seconds (healthy)                                                                          syntect-server
06c33ff5f086   sourcegraph/grafana:5.3.0                     "/entry.sh"              36 seconds ago   Up 34 seconds                      0.0.0.0:3370->3370/tcp                                        grafana
9cf2ea40e0d8   sourcegraph/symbols:5.3.0                     "/sbin/tini -- /symb…"   36 seconds ago   Up 35 seconds (health: starting)                                                                 symbols-0
b11837225e1c   sourcegraph/worker:5.3.0                      "/sbin/tini -- /work…"   36 seconds ago   Up 35 seconds                                                                                    worker
051f97b4b303   sourcegraph/node-exporter:5.3.0               "/usr/bin/node_expor…"   36 seconds ago   Up 36 seconds                      9100/tcp                                                      node-exporter
bd9c717bdeb8   sourcegraph/postgres_exporter:5.3.0           "/usr/bin/postgres_e…"   36 seconds ago   Up 35 seconds                                                                                    pgsql-exporter
02cee851ebec   sourcegraph/blobstore:5.3.0                   "/sbin/tini -- /opt/…"   36 seconds ago   Up 35 seconds (healthy)                                                                          blobstore
97779d345b79   sourcegraph/search-indexer:5.3.0              "/sbin/tini -- zoekt…"   36 seconds ago   Up 35 seconds                                                                                    zoekt-indexserver-0
a71513960cc8   sourcegraph/repo-updater:5.3.0                "/sbin/tini -- /repo…"   36 seconds ago   Up 35 seconds                                                                                    repo-updater
8b84fa164d7d   sourcegraph/prometheus:5.3.0                  "/bin/prom-wrapper"      36 seconds ago   Up 34 seconds                      0.0.0.0:9090->9090/tcp                                        prometheus
d53dc9f23236   sourcegraph/postgres-12-alpine:5.3.0          "/postgres.sh"           36 seconds ago   Up 35 seconds (healthy)            5432/tcp                                                      pgsql
f56c30747b1a   sourcegraph/postgres_exporter:5.3.0           "/usr/bin/postgres_e…"   36 seconds ago   Up 35 seconds                                                                                    codeintel-db-exporter
91ab56e5070a   sourcegraph/precise-code-intel-worker:5.3.0   "/sbin/tini -- /prec…"   36 seconds ago   Up 34 seconds (health: starting)                                                                 precise-code-intel-worker
1dc427a854b1   sourcegraph/searcher:5.3.0                    "/sbin/tini -- /sear…"   36 seconds ago   Up 35 seconds (unhealthy)                                                                        searcher-0
ebe2efe20ff5   sourcegraph/postgres_exporter:5.3.0           "/usr/bin/postgres_e…"   36 seconds ago   Up 35 seconds                                                                                    codeinsights-db-exporter
511c5f75e2f7   sourcegraph/opentelemetry-collector:5.3.0     "/bin/otelcol-source…"   36 seconds ago   Up 34 seconds                                                                                    otel-collector
cb0c4ab9134d   sourcegraph/redis-cache:5.3.0                 "redis-server /etc/r…"   36 seconds ago   Up 36 seconds                                                                                    redis-cache
➜  docker-compose git:(v5.3.0) ✗ dkr logs f0920deb76d1
✱ Sourcegraph migrator 5.3.0
Attempting connection to frontend...
✅ Connection to frontend succeeded
ℹ️ Auto-upgrade flag is set, delegating upgrade to frontend instance
➜  docker-compose git:(v5.3.0) ✗ dkr logs sourcegraph-frontend-internal
Attempting connection to frontend...
✅ Connection to frontend succeeded
Attempting connection to codeintel...
✅ Connection to codeintel succeeded
Attempting connection to codeinsights...
✅ Connection to codeinsights succeeded
👉 Migrating to v5.3 (step 1 of 1)
👉 Running schema migrations
{"SeverityText":"ERROR","Timestamp":1707934740827220510,"InstrumentationScope":"frontend.migrations.Up","Caller":"store/store.go:388","Function":"github.com/sourcegraph/sourcegraph/internal/database/migration/store.(*Store).Up","Body":"operation.error","Resource":{"service.name":"frontend","service.version":"5.3.0","service.instance.id":"674256c1cd2d"},"TraceId":"40ac33329766af9fc2b201801f364218","SpanId":"873bd485213db28f","Attributes":{"count":1,"elapsed":0.00180825,"error":"ERROR: relation \"gitserver_localclone_jobs\" does not exist (SQLSTATE 42P01)"}}
{"SeverityText":"ERROR","Timestamp":1707934740840522260,"InstrumentationScope":"frontend.migrations.WithMigrationLog","Caller":"store/store.go:462","Function":"github.com/sourcegraph/sourcegraph/internal/database/migration/store.(*Store).WithMigrationLog","Body":"operation.error","Resource":{"service.name":"frontend","service.version":"5.3.0","service.instance.id":"674256c1cd2d"},"TraceId":"8da922438573b77edfd54c042c3e9268","SpanId":"fbb5747848270d68","Attributes":{"count":1,"elapsed":0.022427458,"error":"failed to apply migration 1648115472:\n```\nCREATE OR REPLACE VIEW gitserver_localclone_jobs_with_repo_name AS\n  SELECT glj.*, r.name AS repo_name\n  FROM gitserver_localclone_jobs glj\n  JOIN repo r ON r.id = glj.repo_id;\n```: ERROR: relation \"gitserver_localclone_jobs\" does not exist (SQLSTATE 42P01)"}}
{"SeverityText":"FATAL","Timestamp":1707934740847957385,"InstrumentationScope":"sourcegraph","Caller":"svcmain/svcmain.go:167","Function":"github.com/sourcegraph/sourcegraph/internal/service/svcmain.run.func1","Body":"failed to start service","Resource":{"service.name":"frontend","service.version":"5.3.0","service.instance.id":"674256c1cd2d"},"Attributes":{"service":"frontend","error":"frontend.tryAutoUpgrade: error during auto-upgrade: failed to run migration for schema \"frontend\": failed to apply migration 1648115472:\n```\nCREATE OR REPLACE VIEW gitserver_localclone_jobs_with_repo_name AS\n  SELECT glj.*, r.name AS repo_name\n  FROM gitserver_localclone_jobs glj\n  JOIN repo r ON r.id = glj.repo_id;\n```: ERROR: relation \"gitserver_localclone_jobs\" does not exist (SQLSTATE 42P01)"}}
Attempting connection to frontend...
✅ Connection to frontend succeeded
Attempting connection to codeintel...
✅ Connection to codeintel succeeded
Attempting connection to codeinsights...
✅ Connection to codeinsights succeeded
👉 Migrating to v5.3 (step 1 of 1)
👉 Running schema migrations
{"SeverityText":"WARN","Timestamp":1707934744807482637,"InstrumentationScope":"frontend","Caller":"runner/runner.go:382","Function":"github.com/sourcegraph/sourcegraph/internal/database/migration/runner.validateSchemaState","Body":"Attempting to re-try migration that previously failed","Resource":{"service.name":"frontend","service.version":"5.3.0","service.instance.id":"674256c1cd2d"},"Attributes":{}}
{"SeverityText":"ERROR","Timestamp":1707934744819255220,"InstrumentationScope":"frontend.migrations.Up","Caller":"store/store.go:388","Function":"github.com/sourcegraph/sourcegraph/internal/database/migration/store.(*Store).Up","Body":"operation.error","Resource":{"service.name":"frontend","service.version":"5.3.0","service.instance.id":"674256c1cd2d"},"TraceId":"65b60d02ed71ad10e14d69ca25fe3006","SpanId":"79d37544a599e0b2","Attributes":{"count":1,"elapsed":0.001820541,"error":"ERROR: relation \"gitserver_localclone_jobs\" does not exist (SQLSTATE 42P01)"}}
{"SeverityText":"ERROR","Timestamp":1707934744832766137,"InstrumentationScope":"frontend.migrations.WithMigrationLog","Caller":"store/store.go:462","Function":"github.com/sourcegraph/sourcegraph/internal/database/migration/store.(*Store).WithMigrationLog","Body":"operation.error","Resource":{"service.name":"frontend","service.version":"5.3.0","service.instance.id":"674256c1cd2d"},"TraceId":"77e324a1452599c1fe8e0f503485bd86","SpanId":"8e5cf7dd7ac22ef4","Attributes":{"count":1,"elapsed":0.023229917,"error":"failed to apply migration 1648115472:\n```\nCREATE OR REPLACE VIEW gitserver_localclone_jobs_with_repo_name AS\n  SELECT glj.*, r.name AS repo_name\n  FROM gitserver_localclone_jobs glj\n  JOIN repo r ON r.id = glj.repo_id;\n```: ERROR: relation \"gitserver_localclone_jobs\" does not exist (SQLSTATE 42P01)"}}
{"SeverityText":"FATAL","Timestamp":1707934744840278554,"InstrumentationScope":"sourcegraph","Caller":"svcmain/svcmain.go:167","Function":"github.com/sourcegraph/sourcegraph/internal/service/svcmain.run.func1","Body":"failed to start service","Resource":{"service.name":"frontend","service.version":"5.3.0","service.instance.id":"674256c1cd2d"},"Attributes":{"service":"frontend","error":"frontend.tryAutoUpgrade: error during auto-upgrade: failed to run migration for schema \"frontend\": failed to apply migration 1648115472:\n```\nCREATE OR REPLACE VIEW gitserver_localclone_jobs_with_repo_name AS\n  SELECT glj.*, r.name AS repo_name\n  FROM gitserver_localclone_jobs glj\n  JOIN repo r ON r.id = glj.repo_id;\n```: ERROR: relation \"gitserver_localclone_jobs\" does not exist (SQLSTATE 42P01)"}}
Attempting connection to frontend...
✅ Connection to frontend succeeded
Attempting connection to codeintel...
✅ Connection to codeintel succeeded
Attempting connection to codeinsights...
✅ Connection to codeinsights succeeded
👉 Migrating to v5.3 (step 1 of 1)
👉 Running schema migrations
{"SeverityText":"WARN","Timestamp":1707934749224792500,"InstrumentationScope":"frontend","Caller":"runner/runner.go:382","Function":"github.com/sourcegraph/sourcegraph/internal/database/migration/runner.validateSchemaState","Body":"Attempting to re-try migration that previously failed","Resource":{"service.name":"frontend","service.version":"5.3.0","service.instance.id":"674256c1cd2d"},"Attributes":{}}
{"SeverityText":"ERROR","Timestamp":1707934749237288541,"InstrumentationScope":"frontend.migrations.Up","Caller":"store/store.go:388","Function":"github.com/sourcegraph/sourcegraph/internal/database/migration/store.(*Store).Up","Body":"operation.error","Resource":{"service.name":"frontend","service.version":"5.3.0","service.instance.id":"674256c1cd2d"},"TraceId":"962b191a9288058766c91879e8e20ea7","SpanId":"a87ad30e22492b69","Attributes":{"count":1,"elapsed":0.002155083,"error":"ERROR: relation \"gitserver_localclone_jobs\" does not exist (SQLSTATE 42P01)"}}
{"SeverityText":"ERROR","Timestamp":1707934749251186291,"InstrumentationScope":"frontend.migrations.WithMigrationLog","Caller":"store/store.go:462","Function":"github.com/sourcegraph/sourcegraph/internal/database/migration/store.(*Store).WithMigrationLog","Body":"operation.error","Resource":{"service.name":"frontend","service.version":"5.3.0","service.instance.id":"674256c1cd2d"},"TraceId":"91b372c8afa9e705c23a7ea7711f6dee","SpanId":"c80ef04f1c6fd887","Attributes":{"count":1,"elapsed":0.024475541,"error":"failed to apply migration 1648115472:\n```\nCREATE OR REPLACE VIEW gitserver_localclone_jobs_with_repo_name AS\n  SELECT glj.*, r.name AS repo_name\n  FROM gitserver_localclone_jobs glj\n  JOIN repo r ON r.id = glj.repo_id;\n```: ERROR: relation \"gitserver_localclone_jobs\" does not exist (SQLSTATE 42P01)"}}
{"SeverityText":"FATAL","Timestamp":1707934749258715958,"InstrumentationScope":"sourcegraph","Caller":"svcmain/svcmain.go:167","Function":"github.com/sourcegraph/sourcegraph/internal/service/svcmain.run.func1","Body":"failed to start service","Resource":{"service.name":"frontend","service.version":"5.3.0","service.instance.id":"674256c1cd2d"},"Attributes":{"service":"frontend","error":"frontend.tryAutoUpgrade: error during auto-upgrade: failed to run migration for schema \"frontend\": failed to apply migration 1648115472:\n```\nCREATE OR REPLACE VIEW gitserver_localclone_jobs_with_repo_name AS\n  SELECT glj.*, r.name AS repo_name\n  FROM gitserver_localclone_jobs glj\n  JOIN repo r ON r.id = glj.repo_id;\n```: ERROR: relation \"gitserver_localclone_jobs\" does not exist (SQLSTATE 42P01)"}}
Attempting connection to frontend...
✅ Connection to frontend succeeded
Attempting connection to codeintel...
✅ Connection to codeintel succeeded
Attempting connection to codeinsights...
✅ Connection to codeinsights succeeded
👉 Migrating to v5.3 (step 1 of 1)
👉 Running schema migrations
{"SeverityText":"WARN","Timestamp":1707934753761671918,"InstrumentationScope":"frontend","Caller":"runner/runner.go:382","Function":"github.com/sourcegraph/sourcegraph/internal/database/migration/runner.validateSchemaState","Body":"Attempting to re-try migration that previously failed","Resource":{"service.name":"frontend","service.version":"5.3.0","service.instance.id":"674256c1cd2d"},"Attributes":{}}
{"SeverityText":"ERROR","Timestamp":1707934753772920293,"InstrumentationScope":"frontend.migrations.Up","Caller":"store/store.go:388","Function":"github.com/sourcegraph/sourcegraph/internal/database/migration/store.(*Store).Up","Body":"operation.error","Resource":{"service.name":"frontend","service.version":"5.3.0","service.instance.id":"674256c1cd2d"},"TraceId":"7925e1b87386f735212abe86dc4aadec","SpanId":"ea255d1ff6870837","Attributes":{"count":1,"elapsed":0.001737,"error":"ERROR: relation \"gitserver_localclone_jobs\" does not exist (SQLSTATE 42P01)"}}
{"SeverityText":"ERROR","Timestamp":1707934753785831127,"InstrumentationScope":"frontend.migrations.WithMigrationLog","Caller":"store/store.go:462","Function":"github.com/sourcegraph/sourcegraph/internal/database/migration/store.(*Store).WithMigrationLog","Body":"operation.error","Resource":{"service.name":"frontend","service.version":"5.3.0","service.instance.id":"674256c1cd2d"},"TraceId":"08c1d190292900edb1a2f09e64551b44","SpanId":"7daa5791d912f040","Attributes":{"count":1,"elapsed":0.02222,"error":"failed to apply migration 1648115472:\n```\nCREATE OR REPLACE VIEW gitserver_localclone_jobs_with_repo_name AS\n  SELECT glj.*, r.name AS repo_name\n  FROM gitserver_localclone_jobs glj\n  JOIN repo r ON r.id = glj.repo_id;\n```: ERROR: relation \"gitserver_localclone_jobs\" does not exist (SQLSTATE 42P01)"}}
{"SeverityText":"FATAL","Timestamp":1707934753793690460,"InstrumentationScope":"sourcegraph","Caller":"svcmain/svcmain.go:167","Function":"github.com/sourcegraph/sourcegraph/internal/service/svcmain.run.func1","Body":"failed to start service","Resource":{"service.name":"frontend","service.version":"5.3.0","service.instance.id":"674256c1cd2d"},"Attributes":{"service":"frontend","error":"frontend.tryAutoUpgrade: error during auto-upgrade: failed to run migration for schema \"frontend\": failed to apply migration 1648115472:\n```\nCREATE OR REPLACE VIEW gitserver_localclone_jobs_with_repo_name AS\n  SELECT glj.*, r.name AS repo_name\n  FROM gitserver_localclone_jobs glj\n  JOIN repo r ON r.id = glj.repo_id;\n```: ERROR: relation \"gitserver_localclone_jobs\" does not exist (SQLSTATE 42P01)"}}
Attempting connection to frontend...
✅ Connection to frontend succeeded
Attempting connection to codeintel...
✅ Connection to codeintel succeeded
Attempting connection to codeinsights...
✅ Connection to codeinsights succeeded
👉 Migrating to v5.3 (step 1 of 1)
👉 Running schema migrations
{"SeverityText":"WARN","Timestamp":1707934758553496462,"InstrumentationScope":"frontend","Caller":"runner/runner.go:382","Function":"github.com/sourcegraph/sourcegraph/internal/database/migration/runner.validateSchemaState","Body":"Attempting to re-try migration that previously failed","Resource":{"service.name":"frontend","service.version":"5.3.0","service.instance.id":"674256c1cd2d"},"Attributes":{}}
{"SeverityText":"ERROR","Timestamp":1707934758565909587,"InstrumentationScope":"frontend.migrations.Up","Caller":"store/store.go:388","Function":"github.com/sourcegraph/sourcegraph/internal/database/migration/store.(*Store).Up","Body":"operation.error","Resource":{"service.name":"frontend","service.version":"5.3.0","service.instance.id":"674256c1cd2d"},"TraceId":"65f8b730eb739921923d220d3ec07328","SpanId":"8f6ba708b113369c","Attributes":{"count":1,"elapsed":0.001916209,"error":"ERROR: relation \"gitserver_localclone_jobs\" does not exist (SQLSTATE 42P01)"}}
{"SeverityText":"ERROR","Timestamp":1707934758578804046,"InstrumentationScope":"frontend.migrations.WithMigrationLog","Caller":"store/store.go:462","Function":"github.com/sourcegraph/sourcegraph/internal/database/migration/store.(*Store).WithMigrationLog","Body":"operation.error","Resource":{"service.name":"frontend","service.version":"5.3.0","service.instance.id":"674256c1cd2d"},"TraceId":"bad2741ddc17451ac9263c66d3c3e314","SpanId":"3f00b84776d8d2e5","Attributes":{"count":1,"elapsed":0.02349025,"error":"failed to apply migration 1648115472:\n```\nCREATE OR REPLACE VIEW gitserver_localclone_jobs_with_repo_name AS\n  SELECT glj.*, r.name AS repo_name\n  FROM gitserver_localclone_jobs glj\n  JOIN repo r ON r.id = glj.repo_id;\n```: ERROR: relation \"gitserver_localclone_jobs\" does not exist (SQLSTATE 42P01)"}}
{"SeverityText":"FATAL","Timestamp":1707934758586868921,"InstrumentationScope":"sourcegraph","Caller":"svcmain/svcmain.go:167","Function":"github.com/sourcegraph/sourcegraph/internal/service/svcmain.run.func1","Body":"failed to start service","Resource":{"service.name":"frontend","service.version":"5.3.0","service.instance.id":"674256c1cd2d"},"Attributes":{"service":"frontend","error":"frontend.tryAutoUpgrade: error during auto-upgrade: failed to run migration for schema \"frontend\": failed to apply migration 1648115472:\n```\nCREATE OR REPLACE VIEW gitserver_localclone_jobs_with_repo_name AS\n  SELECT glj.*, r.name AS repo_name\n  FROM gitserver_localclone_jobs glj\n  JOIN repo r ON r.id = glj.repo_id;\n```: ERROR: relation \"gitserver_localclone_jobs\" does not exist (SQLSTATE 42P01)"}}
Attempting connection to frontend...
✅ Connection to frontend succeeded
Attempting connection to codeintel...
✅ Connection to codeintel succeeded
Attempting connection to codeinsights...
✅ Connection to codeinsights succeeded
👉 Migrating to v5.3 (step 1 of 1)
👉 Running schema migrations
{"SeverityText":"WARN","Timestamp":1707934764169737215,"InstrumentationScope":"frontend","Caller":"runner/runner.go:382","Function":"github.com/sourcegraph/sourcegraph/internal/database/migration/runner.validateSchemaState","Body":"Attempting to re-try migration that previously failed","Resource":{"service.name":"frontend","service.version":"5.3.0","service.instance.id":"674256c1cd2d"},"Attributes":{}}
{"SeverityText":"ERROR","Timestamp":1707934764181273965,"InstrumentationScope":"frontend.migrations.Up","Caller":"store/store.go:388","Function":"github.com/sourcegraph/sourcegraph/internal/database/migration/store.(*Store).Up","Body":"operation.error","Resource":{"service.name":"frontend","service.version":"5.3.0","service.instance.id":"674256c1cd2d"},"TraceId":"772d5e97b1f7c5c168e4c36c9803479a","SpanId":"7fd6c9a5dc566c03","Attributes":{"count":1,"elapsed":0.001940083,"error":"ERROR: relation \"gitserver_localclone_jobs\" does not exist (SQLSTATE 42P01)"}}
{"SeverityText":"ERROR","Timestamp":1707934764194234798,"InstrumentationScope":"frontend.migrations.WithMigrationLog","Caller":"store/store.go:462","Function":"github.com/sourcegraph/sourcegraph/internal/database/migration/store.(*Store).WithMigrationLog","Body":"operation.error","Resource":{"service.name":"frontend","service.version":"5.3.0","service.instance.id":"674256c1cd2d"},"TraceId":"140b1bbc67f1504e256fb1907003541f","SpanId":"764956fbc13f3c96","Attributes":{"count":1,"elapsed":0.022445458,"error":"failed to apply migration 1648115472:\n```\nCREATE OR REPLACE VIEW gitserver_localclone_jobs_with_repo_name AS\n  SELECT glj.*, r.name AS repo_name\n  FROM gitserver_localclone_jobs glj\n  JOIN repo r ON r.id = glj.repo_id;\n```: ERROR: relation \"gitserver_localclone_jobs\" does not exist (SQLSTATE 42P01)"}}
{"SeverityText":"FATAL","Timestamp":1707934764202230090,"InstrumentationScope":"sourcegraph","Caller":"svcmain/svcmain.go:167","Function":"github.com/sourcegraph/sourcegraph/internal/service/svcmain.run.func1","Body":"failed to start service","Resource":{"service.name":"frontend","service.version":"5.3.0","service.instance.id":"674256c1cd2d"},"Attributes":{"service":"frontend","error":"frontend.tryAutoUpgrade: error during auto-upgrade: failed to run migration for schema \"frontend\": failed to apply migration 1648115472:\n```\nCREATE OR REPLACE VIEW gitserver_localclone_jobs_with_repo_name AS\n  SELECT glj.*, r.name AS repo_name\n  FROM gitserver_localclone_jobs glj\n  JOIN repo r ON r.id = glj.repo_id;\n```: ERROR: relation \"gitserver_localclone_jobs\" does not exist (SQLSTATE 42P01)"}}
Attempting connection to frontend...
✅ Connection to frontend succeeded
Attempting connection to codeintel...
✅ Connection to codeintel succeeded
Attempting connection to codeinsights...
✅ Connection to codeinsights succeeded
👉 Migrating to v5.3 (step 1 of 1)
👉 Running schema migrations
{"SeverityText":"WARN","Timestamp":1707934771445271094,"InstrumentationScope":"frontend","Caller":"runner/runner.go:382","Function":"github.com/sourcegraph/sourcegraph/internal/database/migration/runner.validateSchemaState","Body":"Attempting to re-try migration that previously failed","Resource":{"service.name":"frontend","service.version":"5.3.0","service.instance.id":"674256c1cd2d"},"Attributes":{}}
{"SeverityText":"ERROR","Timestamp":1707934771456469844,"InstrumentationScope":"frontend.migrations.Up","Caller":"store/store.go:388","Function":"github.com/sourcegraph/sourcegraph/internal/database/migration/store.(*Store).Up","Body":"operation.error","Resource":{"service.name":"frontend","service.version":"5.3.0","service.instance.id":"674256c1cd2d"},"TraceId":"1b3589cb090fa4093f1b3901360cbf43","SpanId":"ca84ffd15ee6c424","Attributes":{"count":1,"elapsed":0.00187275,"error":"ERROR: relation \"gitserver_localclone_jobs\" does not exist (SQLSTATE 42P01)"}}
{"SeverityText":"ERROR","Timestamp":1707934771469094135,"InstrumentationScope":"frontend.migrations.WithMigrationLog","Caller":"store/store.go:462","Function":"github.com/sourcegraph/sourcegraph/internal/database/migration/store.(*Store).WithMigrationLog","Body":"operation.error","Resource":{"service.name":"frontend","service.version":"5.3.0","service.instance.id":"674256c1cd2d"},"TraceId":"22b7542132dedc30c25d108531fe6682","SpanId":"1c23c10819eb35be","Attributes":{"count":1,"elapsed":0.021982,"error":"failed to apply migration 1648115472:\n```\nCREATE OR REPLACE VIEW gitserver_localclone_jobs_with_repo_name AS\n  SELECT glj.*, r.name AS repo_name\n  FROM gitserver_localclone_jobs glj\n  JOIN repo r ON r.id = glj.repo_id;\n```: ERROR: relation \"gitserver_localclone_jobs\" does not exist (SQLSTATE 42P01)"}}
{"SeverityText":"FATAL","Timestamp":1707934771477233594,"InstrumentationScope":"sourcegraph","Caller":"svcmain/svcmain.go:167","Function":"github.com/sourcegraph/sourcegraph/internal/service/svcmain.run.func1","Body":"failed to start service","Resource":{"service.name":"frontend","service.version":"5.3.0","service.instance.id":"674256c1cd2d"},"Attributes":{"service":"frontend","error":"frontend.tryAutoUpgrade: error during auto-upgrade: failed to run migration for schema \"frontend\": failed to apply migration 1648115472:\n```\nCREATE OR REPLACE VIEW gitserver_localclone_jobs_with_repo_name AS\n  SELECT glj.*, r.name AS repo_name\n  FROM gitserver_localclone_jobs glj\n  JOIN repo r ON r.id = glj.repo_id;\n```: ERROR: relation \"gitserver_localclone_jobs\" does not exist (SQLSTATE 42P01)"}}
Attempting connection to frontend...
✅ Connection to frontend succeeded
Attempting connection to codeintel...
✅ Connection to codeintel succeeded
Attempting connection to codeinsights...
✅ Connection to codeinsights succeeded
👉 Migrating to v5.3 (step 1 of 1)
👉 Running schema migrations
{"SeverityText":"WARN","Timestamp":1707934781836035793,"InstrumentationScope":"frontend","Caller":"runner/runner.go:382","Function":"github.com/sourcegraph/sourcegraph/internal/database/migration/runner.validateSchemaState","Body":"Attempting to re-try migration that previously failed","Resource":{"service.name":"frontend","service.version":"5.3.0","service.instance.id":"674256c1cd2d"},"Attributes":{}}
{"SeverityText":"ERROR","Timestamp":1707934781847833959,"InstrumentationScope":"frontend.migrations.Up","Caller":"store/store.go:388","Function":"github.com/sourcegraph/sourcegraph/internal/database/migration/store.(*Store).Up","Body":"operation.error","Resource":{"service.name":"frontend","service.version":"5.3.0","service.instance.id":"674256c1cd2d"},"TraceId":"1019bed6596788cf826f4906094259ca","SpanId":"03300acd0be8aba4","Attributes":{"count":1,"elapsed":0.001910166,"error":"ERROR: relation \"gitserver_localclone_jobs\" does not exist (SQLSTATE 42P01)"}}
{"SeverityText":"ERROR","Timestamp":1707934781860677418,"InstrumentationScope":"frontend.migrations.WithMigrationLog","Caller":"store/store.go:462","Function":"github.com/sourcegraph/sourcegraph/internal/database/migration/store.(*Store).WithMigrationLog","Body":"operation.error","Resource":{"service.name":"frontend","service.version":"5.3.0","service.instance.id":"674256c1cd2d"},"TraceId":"7575418f7ec23b814fc52caf33b42c30","SpanId":"f115807566dc6b18","Attributes":{"count":1,"elapsed":0.02270375,"error":"failed to apply migration 1648115472:\n```\nCREATE OR REPLACE VIEW gitserver_localclone_jobs_with_repo_name AS\n  SELECT glj.*, r.name AS repo_name\n  FROM gitserver_localclone_jobs glj\n  JOIN repo r ON r.id = glj.repo_id;\n```: ERROR: relation \"gitserver_localclone_jobs\" does not exist (SQLSTATE 42P01)"}}
{"SeverityText":"FATAL","Timestamp":1707934781868479209,"InstrumentationScope":"sourcegraph","Caller":"svcmain/svcmain.go:167","Function":"github.com/sourcegraph/sourcegraph/internal/service/svcmain.run.func1","Body":"failed to start service","Resource":{"service.name":"frontend","service.version":"5.3.0","service.instance.id":"674256c1cd2d"},"Attributes":{"service":"frontend","error":"frontend.tryAutoUpgrade: error during auto-upgrade: failed to run migration for schema \"frontend\": failed to apply migration 1648115472:\n```\nCREATE OR REPLACE VIEW gitserver_localclone_jobs_with_repo_name AS\n  SELECT glj.*, r.name AS repo_name\n  FROM gitserver_localclone_jobs glj\n  JOIN repo r ON r.id = glj.repo_id;\n```: ERROR: relation \"gitserver_localclone_jobs\" does not exist (SQLSTATE 42P01)"}}

This is a resurfacing of an old issue: https://github.com/sourcegraph/sourcegraph/pull/46969 in which a migration that should be marked as run is being added to the migration plan.

Customers are advised to use the standard upgrade path from v5.2x -> v5.3.0 for now

If you would like immediate help on this, please email support@sourcegraph.com (you can still create the issue, but there are no SLAs on issues like there are for support requests).

DaedalusG commented 8 months ago

This is resolved in the latest migrator release v5.3.1: https://github.com/sourcegraph/sourcegraph/pull/60630