search

pg-nano / pg-parser

Parse your Postgres queries into a 100% type-safe AST (powered by libpg_query)
161 stars 4 forks source link

How to efficiently parse big pgdumps in text format? #11

Open lionelhorn opened 1 day ago

lionelhorn commented 1 day ago

Hello and thanks for your library :)

Any suggestions on how to parse big pg dumps (100GB+) in an efficient manner ?

I assume considering only \n as statement boundary wouldn't be enough in case of multine line statements terminated by ;

eg 

ALTER TABLE public.table
ADD PRIMARY KEY (col_pk);

My use case would be to programatically change the schema in the whole dump.

So all schema ref are modified.

lionelhorn commented 1 day ago

Just saw this : Note: There is no deparse function (for turning an AST back into a string) included, as this isn't needed for my use case. So it seems I won't be able to use pg nano parser for my intended use case.