🐛 📆 DatePicker graying out valid dates

[Abby-Wheelis]

On one of my test phones, the earliest "end date" that I can pick is September 4th (today is September 20th, and I have carried the phone places over the last couple of weeks). I have yet to be able to see more recent trips, even in a draft state.

The app has also been lagging and freezing fairly often on this device as I've been trying to update the date range, so it wouldn't be too shocking if it was an issue with the phone and not the app.

I've had the phone connected to my hotspot (can't use lab wifi on this device) for at least a couple hours now in hope that it would sync and show more recent trips, but no such luck thus far. The situation is fully reproducible in the emulator --

[Abby-Wheelis]

I just checked my other test phone (first was LG, now looking at Samsung) and am having the same issue! It's the same as described above, but the latest available date is Sept 14th, which is later, but still not today. My test iPhone is fine.

I know all three phones were charged for my commute home and back into the building today, and I brought the LG phone on a bike ride yesterday, so they should all have 2-3 trips in the last 24 hours, but the only phone that had trips recently is the test iPhone (and my personal phone on production).

[Abby-Wheelis]

This issue persists, even though the phones have been on wifi for most of the past several days. I'm not sure what's going on, but since it's on two test phones I'm concerned it will be a consistent pattern. Any ideas of next steps to debug? @jgreenlee @shankari for visibility.

[Abby-Wheelis]

Now my test iPhone is also stuck!! Sensed logs on the Samsung show last (and only?) entry as Sept. 18th. But wait, tracking was toggled off on that phone, but it isn't on the other phones? This is still so confusing to me.

[JGreenlee]

@Abby-Wheelis If you privately send me those opcodes, I can check server logs to see if anything is going wrong in the pipeline.

[Abby-Wheelis]

But wait, tracking was toggled off on that phone,

Samsung is tracking again, I was just able to label the trip I took yesterday after turning it back on

[shankari]

Do you know how it got turned off? Was it during testing? Was it during the upgraded (value initialized to false)? I don't want tracking on all phones to turn off when this goes to production....

[Abby-Wheelis]

Do you know how it got turned off? Was it during testing? Was it during the upgraded (value initialized to false)? I don't want tracking on all phones to turn off when this goes to production....

Because I have been testing the profile changes off and on over the past few weeks, I'm sure I was checking the toggle for tracking (since we've had lag issues off and on for those) and forgot to ensure that I left it on. I don't think it's universal, since my other two test phones still showed that they have tracking on.

[Abby-Wheelis]

2/3 test phones are back to tracking, to recap:

• iPhone - brief "stuck" datepicker due to no travel/dead phone, once I charged and traveled, the datepicker caught up with the presence of a trip that day
• Samsung - significant "stuck" datepicker due to no tracked trips, as I had accidentally turned tracking off at some point, once tracking was on and I had a new trip, the datepicker caught up
• LG - extremely "stuck" datepicker, best guess due to server or sync issue

The LG phone is showing Sept 4th as the latest date to select, even though today is the 27th. This is the same as I'm seeing with that opcode on the emulator.

@JGreenlee has proposed partially solving this issue by using "'today' as the max date for the datepicker instead of pipelineEnd. That way, if the pipeline is far behind for a user, they can still access their draft trips up to 'today'"

The fact that the phone pipeline is that far behind might be indicative of a server or sync problem, and either @JGreenlee or @shankari intend to look at the server logs when they get the chance.

[JGreenlee]

I signed into the LG opcode and confirmed that we are getting this pipeline range from the server:

pipelineRange: {
  end_ts: 1693849368.797
  start_ts: 1691502428.199
}

Which really does put end_ts as September 4 (by https://www.epochconverter.com/). I also saw that there is 1 draft trip on September 4 that apparently never got processed. So yes, there is definitely something weird on the server.

[JGreenlee]

@JGreenlee has proposed partially solving this issue by using "'today' as the max date for the datepicker instead of pipelineEnd. That way, if the pipeline is far behind for a user, they can still access their draft trips up to 'today'"

I am second guessing myself now that I am remembering how I wrote the datepicker and why I did it that way.

The datepicker controls the range of dates for which processed trips are loaded. That is why I constrained it to the pipeline range. If, and only if, the selected end date is the same as or later than the pipeline end_ts, all draft trips from pipeline end_ts to today are loaded and appended to the end. This means that even if the last few weeks are greyed out, we are showing all of the draft trips during that range of time.

I did it this way for a reason, and after careful consideration - because I didn't want the datepicker to give the impression that there were trips during a time when there weren't. Hypothetically, consider someone who has had their phone turned off for all of September and comes back to the app now. When they open the app, they'll see August's trips. If they go to the datepicker, I would expect to see August show up and September be greyed out. It would be weird for September to show up; then they'd have to scroll back to see anything.

I think that right now, as far as the phone is concerned, the LG opcode is in that situation - there server is reporting no recorded activity past September 4. So, the phone it doesn't allow you to click past September 4. I think this is what we should expect and rather than changing anything on the phone, we should just focus on fixing the server problem.

[shankari]

Fair enough. Can somebody please confirm (potentially with screenshots) that we do show draft trips for this token that were after Sept 4th?

In that case, I will move this off showstopper status and we can go ahead with the release.

[JGreenlee]

Abby said:

I have yet to be able to see more recent trips, even in a draft state.

I believe there are no draft trips after September 4. I can verify that they are queried for up to today, and the only one that comes back is on September 4. If others exist, they must be lost on the server or something.

[shankari]

Checking the logs, there has been no new data since the 4th

2023-09-27T18:05:53.414-07:00 | 2023-09-28 01:05:53,414:INFO:139671750993728:**********UUID ...: moving to long term**********

2023-09-28 01:05:53,422:INFO:139671750993728:For stage PipelineStages.USERCACHE, start_ts = 2023-09-04T23:10:16.119000

2023-09-28 01:05:53,443:DEBUG:139671750993728:Found 0 messages in response to query {'user_id': UUID('...'), '$or': [{'metadata.type': 'message'}, {'metadata.type': 'sensor-data'}, {'metadata.type': 'rw-document'}], 'metadata.write_ts': {'$lte': 1695863148.4230638, '$gte': 1693869016.119}}

2023-09-28 01:05:53,443:INFO:139671750993728:In moveToLongTerm, no entries to save

[shankari]

Searched for the number of entries processed in the move to long term stage - from 3rd to 5th sept, found one non-zero entry. Found messages in response to query a1f1c01f write_ts

2023-09-04 23:05:49,644:DEBUG:140405127366464:Found 0 messages in response to query {'user_id': UUID('...'), '$or': [{'metadata.type': 'message'}, {'metadata.type': 'sensor-data'}, {'metadata.type': 'rw-document'}], 'metadata.write_ts': {'$lte': 1693868744.629475, '$gte': 1693529130.245}}
2023-09-05 00:05:51,781:DEBUG:139903506347840:Found 661 messages in response to query {'user_id': UUID('...'), '$or': [{'metadata.type': 'message'}, {'metadata.type': 'sensor-data'}, {'metadata.type': 'rw-document'}], 'metadata.write_ts': {'$lte': 1693872346.7596185, '$gte': 1693529130.245}}
2023-09-05 01:05:50,206:DEBUG:140126421841728:Found 0 messages in response to query {'user_id': UUID('...'), '$or': [{'metadata.type': 'message'}, {'metadata.type': 'sensor-data'}, {'metadata.type': 'rw-document'}], 'metadata.write_ts': {'$lte': 1693875945.2002192, '$gte': 1693869016.119}}

I also tried to use cloudwatch regular expressions to find only non-zero entries https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/FilterAndPatternSyntax.html#matching-terms-regex

Found %[1-9]*% messages in response to query a1f1c01f write_ts

but got an error

Invalid character(s) in term '%'

Regardless, this is pretty clearly not a UI issue, so marking it as not a showstopper. The next step is to look at the debug logs to see whether there was any error with the sensing/FSM

https://github.com/e-mission/e-mission-phone#beta-testing-debugging

[Abby-Wheelis]

I just checked my LG phone again, and I have some recent trips, starting on Sept. 28th. However, it seems to have only recorded some of the travel I've done - there's a few trips that only a fragment of the whole trip I took is showing up on the label screen.

[Abby-Wheelis]

Another LG phone update, this morning I have recent trips loaded - saw the two that corresponded to my commute home yesterday (work -> errand, errand -> home) and then a draft trip for home -> work this morning.

I'll keep an eye out, but I think the previous gap may have been that this phone just never recorded a trip for a while as I don't always carry it with me or could have forgotten to charge it.