search

USDA-ARS-NWRC / inicheck

Automated handling, checking, and outputting of complicated .ini configuration files.
Other
1 stars 4 forks source link

Use dateparser instead of pd.to_datetime #42

Closed micah-prime closed 4 years ago

micah-prime commented 4 years ago

Should decrease package size.

scotthavens commented 4 years ago

Hi Prime! I pulled this code almost verbatim into another package and it works great. Glad to not have to use pandas for date parsing.

micah-prime commented 4 years ago

Hey Scott, glad to hear it's getting used!

micahjohnson150 commented 4 years ago

Found it passes all my tests in inicheck. Also very thorough @micah-prime

I tested it smrf to be sure and found an error. 

time                 start_date                     Date is after end_date value                                
time                 end_date                       Date is before start_date value                             
Errors in the config file. See configuration status report above.

(Note: Error message cleaned up for readability)

I went and added the dates in the config file to my tests and it sitll passed depsite this issue above. Looking at the types the scenario in my tests is a little different than how inicheck will deal with it from end to end. In my tests I pass strings, cast, and compare them. In smrf the items can already be casted before checking. Sooooo I did the following.

s = "1998-01-14 15:00:00"
v = parse_date(s)
vv = parse_date(v)
print(v)
datetime.datetime(1998, 1, 14, 15, 0)
print(vv)
datetime.datetime(1998, 1, 14, 0, 0)

Notice the cutting off of the hours when you do this.

I will add a conditional to date_parse to not attempt to cast it again if its already in datetime. Seems odd to me that they wouldn't deal with this but perhaps they have it as a reason.

micahjohnson150 commented 4 years ago

More details. So in the date_parse function, @micah-prime dealt with the problem of repeat casting but python apparently sees a datetime object as an instance of date so the hours are trimmed off.

I added a test for this double casting problem. In the logic of parse_date I moved up the datetime check to the first check and date to the middle. The new test passes plus the tests that prime added pass. And most importantly... run_smrf works!

micahjohnson150 commented 4 years ago

@micah-prime thanks for the very first non-subnaught contribution to inicheck!!!!!