search

18F / rdbms-subsetter

Generates a subset of a relational database that respects foreign key constraints
Creative Commons Zero v1.0 Universal
313 stars 30 forks source link

Is there a way to adapt this to work with dbs that have postgis/geometry data? #46

Open whatnext opened 4 years ago

whatnext commented 4 years ago

Hi there - thanks for your work on this, it looks really useful! However, I hit a snag when trying to use this with a database that has the postgis extension, and geometry data. I get a lot of errors saying things like "Did not recognize type 'geometry'..." followed by the following error:

"Did not recognize type '%s' of column '%s'" % (attype, name) Traceback (most recent call last): File "/usr/bin/rdbms-subsetter", line 8, in sys.exit(generate()) File "/usr/lib/python3.6/site-packages/subsetter.py", line 641, in generate source = Db(args.source, args, schemas) File "/usr/lib/python3.6/site-packages/subsetter.py", line 243, in init fix_postgres_array_of_enum(self.conn, tbl) File "/usr/lib/python3.6/site-packages/dialects/postgres.py", line 45, in fix_postgres_array_of_enum col_str = str(col.type) File "/usr/lib64/python3.6/site-packages/sqlalchemy/sql/type_api.py", line 607, in str return str(self.compile()) File "/usr/lib64/python3.6/site-packages/sqlalchemy/sql/type_api.py", line 590, in compile return dialect.typecompiler.process(self) File "/usr/lib64/python3.6/site-packages/sqlalchemy/sql/compiler.py", line 400, in process return type._compiler_dispatch(self, kw) File "/usr/lib64/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 92, in _compiler_dispatch return meth(self, kw) File "/usr/lib64/python3.6/site-packages/sqlalchemy/sql/compiler.py", line 3387, in visitnull "type on this Column?" % type sqlalchemy.exc.CompileError: Can't generate DDL for NullType(); did you forget to specify a type on this Column?

I wondered if you had any advice on this. Can this code be massaged into working with this kind of db, and if so how can it be done?

Many thanks for your help.