How to extract data more efficiently.

[pedma7]

Hi!

I am using this code below but it takes forever to get the data. Its taking line by line, and if I take a request of 1 million likes it should take several hours. Is there any work aroud this?

def create1min():
    lookupClient.Connect()
    timeframe_1min = lookupClient.Historical.GetHistoryIntervalDays("AAPL", 60, 2000000, 10000000)
    for timeframe_1mi in timeframe_1min:
        timeframe = str(timeframe_1min.get_Timestamp())
        high = timeframe_1min.get_High()
        low = timeframe_1min.get_Low()
        open = timeframe_1min.get_Open()
        close = timeframe_1min.get_Close()

    empresas = Empresas1min(timeframe, high, low, open, close)
    db.session.add(empresas)
    db.session.commit()

Thank you!

[mathpaquette]

@pmadeira123 what do you want to do exactly? save to disk?

[pedma7]

Hi,

Sorry for the delay in the response. Yes , it takes way too much time to get 1 by 1 into desktop or a database. Am I doing something wrong or can be improved in efficiency?

Thank you!

[mathpaquette]

@pmadeira123 please have a look into this example. https://github.com/mathpaquette/IQFeed.CSharpApiClient/blob/master/src/IQFeed.CSharpApiClient.Examples/Examples/ConcurrentFileHistorical/ConcurrentFileHistoricalExample.cs

Tell me if it helps or not.

[mathpaquette]

@pmadeira123 you can increase NumberOfConcurrentClients to 15 to get optimal performance.

[pedma7]

Thank you, I will test this out later today and come with a response. Thank you very much for your assistance.