Download 100K records in csv format

[kaushik1979]

Hi,

Can you give an example of how to download huge amount of data in csf format from browser/python code? I have something similar to the following right now but it obviously doesnt work. Also, I am not sure how to stream data into the client-side temporary file in small chunks of 10K instead of choking the whole pipe with 100k+ records.

import pandas as pd
from datetime import datetime
from dateutil.relativedelta import relativedelta
from typing import Any
from typing_extensions import Required
from flask import request
from flask_restx import Namespace, Resource, fields
from db import prod_db

@api.route('/daily_order_book')
class DailyOrderBook(Resource):
    @api.doc('get_daily_order_book')
    @api.marshal_with(daily_order_book_model)
    @api.produces(mimetypes="application/vnd.ms-excel")
    def get(self):
        '''This method retrieves daily order book.'''
        today: datetime = datetime.today() 
        yesterday: datetime = today - relativedelta(days=1)
        sql: str = f'''
        select * from daily_order_book'
        '''

        df: pd.DataFrame = prod_db.read(sql=sql)
        csv: Any = df.to_csv()

        return csv

[OneCricketeer]

yield each row

https://flask.palletsprojects.com/en/2.2.x/patterns/streaming/