A lot of views to generate and a lot of data

[CatXeN]

Hello, I am encountering a problem because I want to use this library to generate emails with a large amount of data. The problem arises when I have to deal with a significant number of such mailings and a large amount of data I have in the database, because the process can get much longer. I wonder if there is a way to retrieve data from the database in real time. Unfortunately, I personally have not found anything in the documentation that allows such operations. I tried directly inserting IQueryable as data into the model, but I noticed that this IQueryable list is immediately fetched during compilation, and subsequent conditions are handled locally.

So the question is whether there is any concrete idea how I could solve this problem ? Maybe it would be possible to define a function that would return the data or something else.

[hishamco]

Fetching caching & pagination for the data is the developer's responsibility, I'm not sure where's the problem in the Fluid itself

FYI we already use Fluid in many projects including Orchard Core

[CatXeN]

Generally yes, but when I have to process 20,000 such views each one could be generated 2 seconds faster by downloading things from the database in real time e.g. "iqueryable", this gives almost 40000 seconds faster results. Instead of operating on large collections outside the database.

So there is the option of downloading data in real time, e.g. from a database.?

[sebastienros]

You can inject a custom helper to do the query from within the template. Maybe an easy way would be to create a custom "function" like this that will return an entity from the database from an identifier.

However I would suggest you do the queries out of the template, by paging the items you want to process, and then process each item with a template evaluation for each item of the paged result. This will, 1000%, be the fastest option. If you think it's slower than anything else then I would question how you are doing the queries and processing the templates from the disconnected dataset. In that case feel free to share some code you are using to do the queries, and how you currently evaluate the template with the data coming out of the database.

[CatXeN]

Thank you, this has solved my problem