Memory Allocation (Out of Heap Space)

[chriszwickerocteris]

Unfortunately, I've come across a problem when mapping "deep" object structures. I haven't yet been able to create a reproducing test as I'm not quite sure what exactly drives the problem - my theory is that it's more the depth of the object hierarchy than number of columns to map.

I've profiled a request that ran out of heap space and am attaching screenshots of the allocation graph; maybe this allows you to identify the problem quickly?

[arnaudroger]

humm interesting, could you send me an email with the column you have for the query? also what can of object structure are you mapping to ?

[chriszwickerocteris]

Ok, I've finally managed to come up with a test that reproduces the problem. Turns out if I add the aliases for the cases I missed in the generator, my stuff works; there still seems to be a problem - see below.

public class SfmMemoryTests extends SfmTestBase {
  private static Connection conn;

  @BeforeClass
  public static void init() throws SQLException {
    conn = getConnection();
  }

  @Test
  public void failing() throws SQLException {
    final String query = String.join("\n",
      "with t (",
        "id, foo_id, source_id, foo_bars_id, source_foo_id, source_source_id, source_foo_bars_id, source_source_foo_id, source_source_source_id, source_source_foo_bars_id, source_source_source_foo_id, source_source_source_source_id, source_source_source_foo_bars_id, source_source_source_foo_bars$1231511225$, source_source_source_source_foo_id, source_source_source_source_source_id, source_source_source_source_foo_bars_id, source_source_source_source_source_foo_id, source_source_source_source_source_source_id",
      " ) as ( values",
        "('aa', 'bb', NULL, 'cc', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL)",
      ")",
      "select * from t"
    );
    PreparedStatement stmt = conn.prepareStatement(query);
    ResultSet rs = stmt.executeQuery();
    final JdbcMapperFactory mapperFactory = JdbcMapperFactory.newInstance()
      .useAsm(false)
      .ignorePropertyNotFound()
      .addKeys("id")
      .addKeys("foo_bars_id")
      .addKeys("source_foo_bars_id")
      .addKeys("source_source_foo_bars_id")
      .addKeys("source_source_source_foo_bars_id")
      .addKeys("source_source_source_foo_bars$1231511225$")
      .addKeys("source_source_source_source_foo_bars_id")
    ;
    final Iterator<Root> iterator = mapperFactory.newMapper(Root.class).iterator(rs);

    assertTrue(iterator.hasNext());
    final Root found = iterator.next();
    assertNotNull(found);
  }

  @SuppressWarnings("unused")
  public static class Root {
    private String id;
    private Foo foo;
    private Root source;

    public Root() {}

    public Root(final String id) {
      this.id = id;
    }
    public Root(final String id, final Foo foo, final Root source) {
      this.id = id;
      this.foo = foo;
      this.source = source;
    }
  }

  @SuppressWarnings("unused")
  public static class Foo {
    private String id;
    private Set<Bar> bars;

    public Foo() {}
    public Foo(final String id) {
      this.id = id;
    }
    public Foo(final String id, final Set<Bar> bars) {
      this.id = id;
      this.bars = bars;
    }
  }

  @SuppressWarnings("unused")
  public static class Bar {
    private String id;

    public Bar() {}
    public Bar(final String id) {
      this.id = id;
    }
  }
}

[arnaudroger]

was able to reproduce I think just doing that

        JdbcMapperBuilder<Root> builder = JdbcMapperFactory.newInstance().newBuilder(Root.class);
        builder.addMapping("source_source_source_foo_bars$1231511225$");

[arnaudroger]

question what would you expect "source_source_source_foo_bars$1231511225$ to map to?

[chriszwickerocteris]

question what would you expect "source_source_source_foo_bars$1231511225$ to map to?

That's the thing, I was missing aliases in the factory config. I'm abbreviating long identifiers (due to length restrictions in the DB) and have to map back. In this example, it might map to e.g. "source_source_source_foo_bars_alpha_beta_id" (of course these classes are missing in the test). I would then have .addAlias("source_source_source_foo_bars$1231511225$", "source_source_source_foo_bars_alpha_beta_id") in the config.

For my purposes, the problem is not urgent any longer (I just have to do add the aliases), but I think it would be better to throw an Exception at some point warning the user that something is wrong (and how to possibly fix - if my mistake is a likely mistake) rather than have the JVM crash ;-)

[arnaudroger]

yes totally agreed, it should not have oom, thanks for the report.