Using async iterator in copy_records_to_table

[alex-eri]

• asyncpg version: 0.22.0
• PostgreSQL version: 13
• Do you use a PostgreSQL SaaS? If so, which? Can you reproduce the issue with a local PostgreSQL install?: no
• Python version: 3.8, 3.9
• Platform: Windows, Linux
• Do you use pgbouncer?: no
• Did you install asyncpg with pip?: git
• If you built asyncpg locally, which version of Cython did you use?: 0.29.21
• Can the issue be reproduced under both asyncio and uvloop?: yes

I want to stream data to base with COPY, but i have async code, that produce data. I can pass regular iterator to records, but it ends then Queue empty. So reconnect needed.

queue = asyncio.Queue(2<<10)
loop.create_task(feed(queue))

async def get_records(queue):
    while True:
        r = await queue.get()
        yield r
        self.q.task_done()

async with pg.acquire() as connection:
    await connection.copy_records_to_table(
                     'radarlog2',
                     records = get_records(queue)
                 )

[robhaswell]

I too was surprised that this wasn't supported.

It seems that copy_to_table supports an async iterator but it must generate the Postgres binary protocol? I couldn't find a protocol formatter anywhere even though I assume that asyncpg must contain one somewhere in order to support this function.

If anyone has some guidance on how to use copy_to_table then that would be a good alternative, thanks.

[elprans]

713 adds support for async iterables in copy_records_to_table.

[robhaswell]

Thanks!