Ale0430 / MiniBuoy-App

Its another app
GNU General Public License v3.0
0 stars 0 forks source link

Pendant date reading correction #36

Closed cailadd90 closed 10 months ago

cailadd90 commented 11 months ago

Pendant data format is in mm-dd-yyyy, so fastPOSIXct() can't recognise it should be dd-mm-yyyy. This fix works, but it won't work if the data is ever in any other format.

mcwimm commented 11 months ago

I introduced an automated format recognition. @cailadd90 can try if this works with your pendant data?

Furthermore, the datetime column in the saved csv is now a character without letters....

cailadd90 commented 11 months ago

I introduced an automated format recognition. @cailadd90 can try if this works with your pendant data?

Furthermore, the datetime column in the saved csv is now a character without letters....

Unfortunately not. The App is still mixing up day and month - looks like it's reading it as mm-dd-yy instead of yyyy-mm-dd as it expects

![Uploading Screenshot 2023-08-10 at 14.28.36.png…]()

cailadd90 commented 11 months ago

Also, I still see the T and Z.

Screenshot 2023-08-10 at 14 34 44

And just to make sure, I am on the updated branch I think!

Screenshot 2023-08-10 at 14 36 26
mcwimm commented 11 months ago

Ok. The T-Z thing should work now.

To work on the format thing, I need pendant data ;)

cailadd90 commented 11 months ago

Awesome, I've uploaded zipped example data on GitHub in the 'data' folder, if you'd like to now give that a try?

cailadd90 commented 11 months ago

FYI - I'm searching for a solution for R to detect if format is YYYY/MM/DD or YYYY/DD/MM. I'm not suggesting another package, but this one will try to solve this ambiguity by going through more lines, to work out which format it likely is: https://search.r-project.org/CRAN/refmans/dataPreparation/html/identify_dates.html

Maybe one of our existing packages does something similar?

mcwimm commented 11 months ago

Awesome, I've uploaded zipped example data on GitHub in the 'data' folder, if you'd like to now give that a try?

Now I see the problem. Our function recognizes the datetime but the provided format is not "unique" (could be read in several ways). If this is the standard output of the logger, we need to fix it to this format or at least we could do 2 steps:

  1. try to read the file with the fixed format and if this throws an error
  2. use the automatic recognition that*s currently used.
cailadd90 commented 11 months ago

Awesome, I've uploaded zipped example data on GitHub in the 'data' folder, if you'd like to now give that a try?

Now I see the problem. Our function recognizes the datetime but the provided format is not "unique" (could be read in several ways). If this is the standard output of the logger, we need to fix it to this format or at least we could do 2 steps:

  1. try to read the file with the fixed format and if this throws an error
  2. use the automatic recognition that*s currently used.

Good idea. I'm not sure if the date format can be customised, all the Pendant data I've received has been in this mm-dd-yy format so far. But just in case, your 2 step idea is a good one, to cover several eventualities?

cailadd90 commented 11 months ago

Maybe this can be used in a clever way: https://lubridate.tidyverse.org/reference/guess_formats.html

mcwimm commented 11 months ago

@cailadd90 can you test it again?

cailadd90 commented 11 months ago

Arg no, still seems to be reading it wrongly:

Screenshot 2023-08-10 at 15 56 06
mcwimm commented 11 months ago

I hope it works now :)

image

cailadd90 commented 11 months ago

Nice! Yes it's reading correctly now. I do get errors when trying to analyse the data, but I'll go through that now and see where the issue is.

mcwimm commented 10 months ago

Ok. We'll merge the change but open a new PR to deal with the error when uploading pendant data.

cailadd90 commented 10 months ago

Yes happy with that.