time.sleep(2) # sleep for X second(s) to avoid hitting the rate limit --batch-size 5 works
this is a hack, should be some parameter.
2) for restore, I got errors with fractional timestamp precision, so I had to chage the below to make this work:
FROM:
def convert_timestamp(val):
"""Convert Unix epoch timestamp to datetime.datetime object."""
# yuck, we aren't actually storing timestamps in the GYB
# database. I blame the original developer :-)
# return datetime.datetime.fromtimestamp(int(val))
return datetime.datetime.strptime(val.decode('UTF-8'), '%Y-%m-%d %H:%M:%S')
TO:
def convert_timestamp(val):
val = val.decode('UTF-8') # Assuming `val` is a bytes object that needs decoding.
# Check if the datetime string contains fractional seconds.
if '.' in val:
# If yes, parse with fractional seconds.
format_str = '%Y-%m-%d %H:%M:%S.%f'
else:
# If no, parse without fractional seconds.
format_str = '%Y-%m-%d %H:%M:%S'
return datetime.datetime.strptime(val, format_str)
to handle both values with and without factional values.
1) To make the gyb.py script work (main branch) I had to insert the sleep() call and use batch size of 5 to pull about 40K messages
def callGAPI(service, function, soft_errors=False, throw_reasons=[], retry_reasons=[], **kwargs):
time.sleep(2) # sleep for X second(s) to avoid hitting the rate limit --batch-size 5 works
this is a hack, should be some parameter.
2) for restore, I got errors with fractional timestamp precision, so I had to chage the below to make this work:
FROM:
TO:
to handle both values with and without factional values.