com-lihaoyi / scalasql

Scala ORM to query SQL databases from Scala via concise, type-safe, and familiar case classes and collection operations. Connects to Postgres, MySql, H2, and Sqlite out of the box
195 stars 22 forks source link

Huge memory increase in `renderSql` with scala 2.13.13 (OOM) #5

Closed alexdupre closed 7 months ago

alexdupre commented 8 months ago

I wanted to try this library and immediately got an OutOfMemoryError for a trivial select statement of a single row on a single table. Reducing the fields count in the table definition allowed me to run the query, but it still took several seconds to generate the query, something was clearly wrong. Reverting to scala 2.13.12 (from 2.13.13) fixed the issue, apparently the last two commit of the ArrayBuilder broke something: https://github.com/scala/scala/commits/5aa3dc5ea77800ddd2a3bdf9e7feeea57484421c/src/library/scala/collection/mutable/ArrayBuilder.scala

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
        at java.base/java.lang.reflect.Array.newArray(Native Method)
        at java.base/java.lang.reflect.Array.newInstance(Array.java:78)
        at java.base/java.util.Arrays.copyOf(Arrays.java:3514)
        at java.base/java.util.Arrays.copyOf(Arrays.java:3482)
        at scala.collection.mutable.ArrayBuilder$ofRef.mkArray(ArrayBuilder.scala:110)
        at scala.collection.mutable.ArrayBuilder$ofRef.resize(ArrayBuilder.scala:114)
        at scala.collection.mutable.ArrayBuilder.ensureSize(ArrayBuilder.scala:39)
        at scala.collection.mutable.ArrayBuilder.addAll(ArrayBuilder.scala:53)
        at scala.collection.mutable.ArrayBuilder.addAll(ArrayBuilder.scala:49)
        at scalasql.core.SqlStr$.rec$1(SqlStr.scala:102)
        at scalasql.core.SqlStr$.rec$1(SqlStr.scala:120)
        at scalasql.core.SqlStr$.rec$1(SqlStr.scala:120)
        at scalasql.core.SqlStr$.flatten(SqlStr.scala:134)
        at scalasql.query.SimpleSelect$Renderer.render(SimpleSelect.scala:271)
        at scalasql.query.CompoundSelect$Renderer.render(CompoundSelect.scala:119)
        at scalasql.query.Select.renderSql(Select.scala:193)
        at scalasql.query.Select.renderSql$(Select.scala:190)
        at scalasql.query.CompoundSelect.renderSql(CompoundSelect.scala:10)
        at scalasql.core.SqlStr$Renderable$.renderSql(SqlStr.scala:218)
        at scalasql.query.Query$Single.renderSql(Query.scala:83)
        at scalasql.query.Query$QueryQueryable.renderSql(Query.scala:72)
        at scalasql.query.Query$QueryQueryable.renderSql(Query.scala:66)
        at scalasql.core.DbApi$.unpackQueryable(DbApi.scala:112)
        at scalasql.core.DbApi$.renderSql(DbApi.scala:119)
        at scalasql.core.DbApi$Impl.renderSql(DbApi.scala:424)
alexdupre commented 8 months ago

I think I've found the problem. I'll open an issue on the scala repository, but it should be possible to implement a workaround in scalasql, too.

alexdupre commented 8 months ago

https://github.com/scala/bug/issues/12971

lihaoyi commented 7 months ago

Should be fixed in Scala 2.13.14. Thanks for reporting and getting this resolved!