sraoss / pg_ivm

IVM (Incremental View Maintenance) implementation as a PostgreSQL extension
Other
862 stars 24 forks source link

`refresh_ivm` that refreshes incrementally not all #21

Open sudoStatus200 opened 1 year ago

sudoStatus200 commented 1 year ago

I need to use ivm to create incremental view, I can create ivm from query. But whenever I try to insert/delete to base table , I get this error:

server closed the connection unexpectedly
        This probably means the server terminated abnormally
        before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.

I am assuming get this error cause pg_ivm immedietely tries to update ivm but to update ivm requires a heavy computation and that causing db crash. So basically I cannot use immediet view maintanence.

pg_ivm has another refresh utility but that utility completely replaces old content and takes time. It works same as REFRESH command in built in materialized table. Hence no point of using pg_ivm for this.

I am requesting a feature, Where I can use something like refresh manually (deferred maintanence) that only increments view not replaces all content. In that way refreshing view take less time and less resources.

yugo-n commented 1 year ago

I am assuming get this error cause pg_ivm immedietely tries to update ivm but to update ivm requires a heavy computation and that causing db crash. So basically I cannot use immediet view maintanence.

I am not sure why db crashed exactly, but maybe it was due to a heavy computation if the base table had too large data, the view had many joins, or the change proportion was large, and so on. Otherwise, there might be other causes so could you please details about the data size, the view definition, and the executed commands?

pg_ivm has another refresh utility but that utility completely replaces old content and takes time. It works same as REFRESH command in built in materialized table. Hence no point of using pg_ivm for this.

Yes, pg_immv_refresh is basically the same as REFRESH and not do deferred incremental maintenance. In such cases where such "refresh" is more effective than incremental maintenance for certain table modification, it would be better not to use pg_ivm, or tentatively to disable incremental maintenance before such modifications by using pg_immv_refresh.

I am requesting a feature, Where I can use something like refresh manually (deferred maintanence) that only increments view not replaces all content. In that way refreshing view take less time and less resources.

Thank you for your request. We've also been thinking to support deferred maintenance feature in a future release.