Attempting to run a for loop that will cycle through the months of 2016 and grab historical LMP data for a select number of nodes. I'm getting a 502 error very sporadically as the program runs, it will grab one or two node lmp data than give me the error for the next 5-6. I've noticed that as I restart the program the 502 errors are not for the same nodes so I don't believe it's a problem of not having the data.
Would anyone have any thoughts on why this may happen (code below)? I thought it may have been too much for the server so I tried with various sleep options, but it hasn't helped.
count = 0
caiso = client_factory("CAISO", timeout_seconds=360)
year = 2016
for start_month in range(1,6):
end_month = start_month + 1
start = str(start_month) + "/" + str(year)
end = str(end_month) + "/"+str(year)
date1 = datetime.strptime(start, "%m/%Y")
date2 = datetime.strptime(end, "%m/%Y")
month_name = datetime.strftime(date1,"%b"+"_"+"%Y")
for node in nodes:
sleep(65)
try:
count += 1
data = caiso.get_lmp(start_at=date1, end_at=date2, node_id=node)
df = pd.DataFrame(data)
df.to_csv("LMP_"+node+month_name+".csv")
print(count, node, date1)
except:
pass
print("COULD NOT GRAB DATA FOR"+node+"on"+date1)
Hello,
Attempting to run a for loop that will cycle through the months of 2016 and grab historical LMP data for a select number of nodes. I'm getting a 502 error very sporadically as the program runs, it will grab one or two node lmp data than give me the error for the next 5-6. I've noticed that as I restart the program the 502 errors are not for the same nodes so I don't believe it's a problem of not having the data.
Would anyone have any thoughts on why this may happen (code below)? I thought it may have been too much for the server so I tried with various sleep options, but it hasn't helped.