18F / rdbms-subsetter

Generates a subset of a relational database that respects foreign key constraints
Creative Commons Zero v1.0 Universal
313 stars 30 forks source link

RuntimeError: maximum recursion depth exceeded while getting the str of an object #34

Open murphyke opened 8 years ago

murphyke commented 8 years ago

Using rdbms-subsetter==0.2.5 and PostgreSQL 9.5, I'm trying to extract a subset, but I get a long stack trace followed by the error "RuntimeError: maximum recursion depth exceeded while getting the str of an object".

Does this ring any bells?

$ rdbms-subsetter postgresql://postgres@localhost/foo postgresql://postgres@localhost/foo_subset .00005
Create 0 rows from 0 in .concept_ancestor
Create 0 rows from 0 in .source_to_concept_map
Create 1 rows from 265 in .concept_class
Create 1 rows from 3 in .version_history
Create 1 rows from 382 in .relationship
Create 1 rows from 39 in .domain
Create 1 rows from 62 in .vocabulary
Create 136 rows from 2728500 in .concept
Create 19 rows from 380069 in .drug_strength
Create 209 rows from 4193420 in .concept_synonym
Create 793 rows from 15873400 in .concept_relationship
Proceed? (Y/n) y
2016-05-25 15:24:37,596 INFO  lowest completeness score (in concept) at 0.000000
Traceback (most recent call last):
  File "/Users/murphyke/Documents/pedsnet/pedsnet_env/bin/rdbms-subsetter", line 9, in <module>
    load_entry_point('rdbms-subsetter==0.2.5', 'console_scripts', 'rdbms-subsetter')()
  File "/Users/murphyke/Documents/pedsnet/pedsnet_env/lib/python2.7/site-packages/subsetter.py", line 554, in generate
    source.create_subset_in(target)
  File "/Users/murphyke/Documents/pedsnet/pedsnet_env/lib/python2.7/site-packages/subsetter.py", line 446, in create_subset_in
    prioritized=prioritized)
  File "/Users/murphyke/Documents/pedsnet/pedsnet_env/lib/python2.7/site-packages/subsetter.py", line 340, in create_row_in
    self.create_row_in(source_parent_row, target_db, target_parent)

... and hundreds of row pairs identical to the previous two, then ...

  File "/Users/murphyke/Documents/pedsnet/pedsnet_env/lib/python2.7/site-packages/subsetter.py", line 337, in create_row_in
    target_parent_row = target_db.conn.execute(slct).first()
  File "/Users/murphyke/Documents/pedsnet/pedsnet_env/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 914, in execute
    return meth(self, multiparams, params)
  File "/Users/murphyke/Documents/pedsnet/pedsnet_env/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/Users/murphyke/Documents/pedsnet/pedsnet_env/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1003, in _execute_clauseelement
    inline=len(distilled_params) > 1)
  File "<string>", line 1, in <lambda>
  File "/Users/murphyke/Documents/pedsnet/pedsnet_env/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 494, in compile
    return self._compiler(dialect, bind=bind, **kw)
  File "/Users/murphyke/Documents/pedsnet/pedsnet_env/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 500, in _compiler
    return dialect.statement_compiler(dialect, self, **kw)
  File "/Users/murphyke/Documents/pedsnet/pedsnet_env/lib/python2.7/site-packages/sqlalchemy/sql/compiler.py", line 392, in __init__
    Compiled.__init__(self, dialect, statement, **kwargs)
  File "/Users/murphyke/Documents/pedsnet/pedsnet_env/lib/python2.7/site-packages/sqlalchemy/sql/compiler.py", line 190, in __init__
    self.string = self.process(self.statement, **compile_kwargs)
  File "/Users/murphyke/Documents/pedsnet/pedsnet_env/lib/python2.7/site-packages/sqlalchemy/sql/compiler.py", line 213, in process
    return obj._compiler_dispatch(self, **kwargs)
  File "/Users/murphyke/Documents/pedsnet/pedsnet_env/lib/python2.7/site-packages/sqlalchemy/sql/visitors.py", line 81, in _compiler_dispatch
    return meth(self, **kw)
  File "/Users/murphyke/Documents/pedsnet/pedsnet_env/lib/python2.7/site-packages/sqlalchemy/sql/compiler.py", line 1605, in visit_select
    text, select, inner_columns, froms, byfrom, kwargs)
  File "/Users/murphyke/Documents/pedsnet/pedsnet_env/lib/python2.7/site-packages/sqlalchemy/sql/compiler.py", line 1702, in _compose_select_body
    t = select._whereclause._compiler_dispatch(self, **kwargs)
  File "/Users/murphyke/Documents/pedsnet/pedsnet_env/lib/python2.7/site-packages/sqlalchemy/sql/visitors.py", line 93, in _compiler_dispatch
    return meth(self, **kw)
  File "/Users/murphyke/Documents/pedsnet/pedsnet_env/lib/python2.7/site-packages/sqlalchemy/sql/compiler.py", line 921, in visit_binary
    return self._generate_generic_binary(binary, opstring, **kw)
  File "/Users/murphyke/Documents/pedsnet/pedsnet_env/lib/python2.7/site-packages/sqlalchemy/sql/compiler.py", line 938, in _generate_generic_binary
    binary.right._compiler_dispatch(self, **kw)
  File "/Users/murphyke/Documents/pedsnet/pedsnet_env/lib/python2.7/site-packages/sqlalchemy/sql/visitors.py", line 93, in _compiler_dispatch
    return meth(self, **kw)
  File "/Users/murphyke/Documents/pedsnet/pedsnet_env/lib/python2.7/site-packages/sqlalchemy/sql/compiler.py", line 1067, in visit_bindparam
    name = self._truncate_bindparam(bindparam)
  File "/Users/murphyke/Documents/pedsnet/pedsnet_env/lib/python2.7/site-packages/sqlalchemy/sql/compiler.py", line 1123, in _truncate_bindparam
    bind_name = self._truncated_identifier("bindparam", bind_name)
  File "/Users/murphyke/Documents/pedsnet/pedsnet_env/lib/python2.7/site-packages/sqlalchemy/sql/compiler.py", line 1134, in _truncated_identifier
    anonname = name.apply_map(self.anon_map)
  File "/Users/murphyke/Documents/pedsnet/pedsnet_env/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 3609, in apply_map
    return self % map_
  File "/Users/murphyke/Documents/pedsnet/pedsnet_env/lib/python2.7/site-packages/sqlalchemy/util/_collections.py", line 728, in __missing__
    self[key] = val = self.creator(key)
  File "/Users/murphyke/Documents/pedsnet/pedsnet_env/lib/python2.7/site-packages/sqlalchemy/sql/compiler.py", line 1153, in _process_anon
    return derived + "_" + str(anonymous_counter)
RuntimeError: maximum recursion depth exceeded while getting the str of an object
$ 
alexeybozhev commented 7 years ago

bump

chimeno commented 6 years ago

Exactly the same here with: rdbms-subsetter==0.2.5 and PostgreSQL 9.4.6

nfm commented 6 years ago

When I hit this issue, it was because there was a circular reference in my database schema. I had accounts.owner_id referencing users.id, and users.account_id referencing accounts.id, which caused infinite recursion ending in the above backtrace.