search

jim256 / ebay

0 stars 0 forks source link

date_listed > date_found #15

Closed jim256 closed 4 years ago

jim256 commented 4 years ago

I noticed this and was wondering what might cause it:

image

I always wanted to double - check: we are converting all dates to UTC and storing them in UTC, right?

james-carpenter commented 4 years ago

Yes, all date fields are UTC, both in the app and in eBay. The reason this may occur is that I fix the execution time when the program starts so it's feasible something popped in while we were executing (or clock differences I suppose).

The reasons I chose to do it that way are:

  1. The date_found or date_updated for a given batch run will be consistent in the database so you can see which records came in at the same time or were updated in a subsequent run versus later in the same one.
  2. That's the reference date used for the next run to make sure we don't miss anything.
jim256 commented 4 years ago

Sounds good. I appreciate the explanation.