make it easy to build dependent changes

lolopinto commented 2 years ago

steps

make change 1 in branch A (which has db changes)
make change 2 in branch B (which has db changes)
switch back to branch A to run code, get this error

> docker-compose -f docker-compose.dev.yml run --rm app tsent codegen

Creating backend_app_run ... done
checking for custom graphql definitions...
Missing baseUrl in compilerOptions. tsconfig-paths will be skipped
updating db...
Traceback (most recent call last):
  File "/opt/venv/lib/python3.8/site-packages/alembic/script/base.py", line 162, in _catch_revision_errors
    yield
  File "/opt/venv/lib/python3.8/site-packages/alembic/script/base.py", line 227, in get_revisions
    return self.revision_map.get_revisions(id_)
  File "/opt/venv/lib/python3.8/site-packages/alembic/script/revision.py", line 319, in get_revisions
    return sum([self.get_revisions(id_elem) for id_elem in id_], ())
  File "/opt/venv/lib/python3.8/site-packages/alembic/script/revision.py", line 319, in <listcomp>
    return sum([self.get_revisions(id_elem) for id_elem in id_], ())
  File "/opt/venv/lib/python3.8/site-packages/alembic/script/revision.py", line 322, in get_revisions
    return tuple(
  File "/opt/venv/lib/python3.8/site-packages/alembic/script/revision.py", line 323, in <genexpr>
    self._revision_for_ident(rev_id, branch_label)
  File "/opt/venv/lib/python3.8/site-packages/alembic/script/revision.py", line 386, in _revision_for_ident
    raise ResolutionError(
alembic.script.revision.ResolutionError: No such revision or branch '7dec8a7405e6'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/venv/bin/auto_schema", line 8, in <module>
    sys.exit(main())
  File "/opt/venv/lib/python3.8/site-packages/auto_schema/cli/__init__.py", line 72, in main
    r.run()
  File "/opt/venv/lib/python3.8/site-packages/auto_schema/runner.py", line 187, in run
    self._apply_changes(diff)
  File "/opt/venv/lib/python3.8/site-packages/auto_schema/runner.py", line 202, in _apply_changes
    raise err
  File "/opt/venv/lib/python3.8/site-packages/auto_schema/runner.py", line 196, in _apply_changes
    self.revision(diff)
  File "/opt/venv/lib/python3.8/site-packages/auto_schema/runner.py", line 225, in revision
    self.cmd.revision(message)
  File "/opt/venv/lib/python3.8/site-packages/auto_schema/command.py", line 50, in revision
    command.revision(self.alembic_cfg, message, autogenerate=True)
  File "/opt/venv/lib/python3.8/site-packages/alembic/command.py", line 214, in revision
    script_directory.run_env()
  File "/opt/venv/lib/python3.8/site-packages/alembic/script/base.py", line 489, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/opt/venv/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 98, in load_python_file
    module = load_module_py(module_id, path)
  File "/opt/venv/lib/python3.8/site-packages/alembic/util/compat.py", line 184, in load_module_py
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/opt/venv/lib/python3.8/site-packages/auto_schema/env.py", line 133, in <module>
    run_migrations_online()
  File "/opt/venv/lib/python3.8/site-packages/auto_schema/env.py", line 127, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/opt/venv/lib/python3.8/site-packages/alembic/runtime/environment.py", line 846, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/opt/venv/lib/python3.8/site-packages/alembic/runtime/migration.py", line 509, in run_migrations
    for step in self._migrations_fn(heads, self):
  File "/opt/venv/lib/python3.8/site-packages/alembic/command.py", line 190, in retrieve_migrations
    revision_context.run_autogenerate(rev, context)
  File "/opt/venv/lib/python3.8/site-packages/alembic/autogenerate/api.py", line 442, in run_autogenerate
    self._run_environment(rev, migration_context, True)
  File "/opt/venv/lib/python3.8/site-packages/alembic/autogenerate/api.py", line 453, in _run_environment
    if set(self.script_directory.get_revisions(rev)) != set(
  File "/opt/venv/lib/python3.8/site-packages/alembic/script/base.py", line 227, in get_revisions
    return self.revision_map.get_revisions(id_)
  File "/usr/local/lib/python3.8/contextlib.py", line 131, in __exit__
    self.gen.throw(type, value, traceback)
  File "/opt/venv/lib/python3.8/site-packages/alembic/script/base.py", line 194, in _catch_revision_errors
    compat.raise_from_cause(util.CommandError(resolution))
  File "/opt/venv/lib/python3.8/site-packages/alembic/util/compat.py", line 308, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=exc_value)
  File "/opt/venv/lib/python3.8/site-packages/alembic/util/compat.py", line 301, in reraise
    raise value.with_traceback(tb)
  File "/opt/venv/lib/python3.8/site-packages/alembic/script/base.py", line 162, in _catch_revision_errors
    yield
  File "/opt/venv/lib/python3.8/site-packages/alembic/script/base.py", line 227, in get_revisions
    return self.revision_map.get_revisions(id_)
  File "/opt/venv/lib/python3.8/site-packages/alembic/script/revision.py", line 319, in get_revisions
    return sum([self.get_revisions(id_elem) for id_elem in id_], ())
  File "/opt/venv/lib/python3.8/site-packages/alembic/script/revision.py", line 319, in <listcomp>
    return sum([self.get_revisions(id_elem) for id_elem in id_], ())
  File "/opt/venv/lib/python3.8/site-packages/alembic/script/revision.py", line 322, in get_revisions
    return tuple(
  File "/opt/venv/lib/python3.8/site-packages/alembic/script/revision.py", line 323, in <genexpr>
    self._revision_for_ident(rev_id, branch_label)
  File "/opt/venv/lib/python3.8/site-packages/alembic/script/revision.py", line 386, in _revision_for_ident
    raise ResolutionError(
alembic.util.exc.CommandError: Can't locate revision identified by '7dec8a7405e6'
Error: exit status 1

there should be an easy command that resets the db back to state in a branch

lolopinto commented 2 years ago

manual fix:

docker-compose -f docker-compose.dev.yml run --rm app tsent alembic history
get the revision we want to revert to
update alembic_version set version_num = {rev}
run npm run codegen
the other changes in other branch get dropped with new
rerun update alembic_version set version_num = {rev}
delete the file(s) which were generated to revert those changes

lolopinto commented 2 years ago

another solution is to breakout codegen into separate commands:

codegen that only generates code e.g. codegen --step codegen && codegen --step graphql codegen+db generates code and updates db

when switching between branches, most times don't need the db change to be reflected anyways since it's just adding on top of other change e.g. adding tables, nullable columns etc

if a column or table is dropped in one or a non-nullable column is added, this is still needed

lolopinto commented 2 years ago

ran into this again. steps i took:

docker-compose -f docker-compose.dev.yml run --rm app tsent alembic history to find head
update alembic_version set version_num = {entry_above}
npm run psql. drop the tables that had been added.

need to fix this soon.

https://alembic.sqlalchemy.org/en/latest/branches.html relevant related: https://github.com/lolopinto/ent/issues/316

lolopinto commented 2 years ago

when going through to restore to current head or whatever, we can ignore edge changes since they're harmless or make sure to delete from assoc_edge_config which i haven't been doing as I'm dropping columns

lolopinto commented 2 years ago

another option is reset db and just rerun everything, simpler in the beginning but will be annoying when there's a lot of changes if it takes a while to replay

lolopinto commented 2 years ago

if you have stacked diffs and change the base of the previous diff because of changes in the stack and come back to new change in the branch, everything is broken

what I did was find current version, get the rev, update down_revision in one diff and that fixes the stack

lolopinto commented 2 years ago

related: should maybe have a codegen that skips db. if you're in a different branch that doesn't have the db history and just want to recodegen code, it's annoying to have to go to other branch and figure out how many to downgrade by

lolopinto / ent

make it easy to build dependent changes #499