Not all Slick sources stream by default

[arnold-jr]

This is an issue of documentation clarity: as per Slick's documentation, not all databases (most notably PostgreSQL) require .withStatementParameters in order to get results returned asynchronously. So a truly streaming query would look like

val typedSelectAllUsers = TableQuery[Users].result
          .withStatementParameters(
            rsType = ResultSetType.ForwardOnly,
            rsConcurrency = ResultSetConcurrency.ReadOnly,
            fetchSize = conf.getInt("batchSize")
          )

This might be a useful addition to the docs, since DBs that will fit entirely into memory are more the exception than the rule :)

Thanks!

P.S. Incidentally, for PostgreSQL, the fetchSize is not respected!

[2m]

Ahh, good insight! Would you be up for creating a PR for documentation amendment?

[WellingR]

Before I have done some tests with this. And I indeed found that the above is almost the full solution. The last thing that is needed (which is also mentioned in https://github.com/slick/slick/issues/1305 ) is to wrap the query in a .transactionally block

So the truly streaming query would look like

val typedSelectAllUsers = TableQuery[Users].result
          .withStatementParameters(
            rsType = ResultSetType.ForwardOnly,
            rsConcurrency = ResultSetConcurrency.ReadOnly,
            fetchSize = conf.getInt("batchSize")
          )
          .transactionally

[Sh1ftry]

Shouldn't this be explicitly pointed in the Slick Alpakka documentation? By looking at example that is present in the documentation I can assume that using Slick.source(TableQuery[Users].result) is enough to enable streaming, which is not true, because whole result would be first loaded into memory for some databases.