Unclear message about "unexpected" data downloaded

[chad-earthscope]

Often downloads result in this message: "The previous retrieval attempt (3 of 3) downloaded unexpected data. Error potentially caused because of inconsistent web services or data had a sample rate = 0."

It is not very clear what that means and it's not very useful in the sense that I'm not sure what a user is supposed to do with it. Retry? Can we suggest something useful?

For one. It says it got unexpected data, which makes me think data was download that was not requested and I find that hard to believe. Might be the wrong phrase to use?

For two, data having sample rate of 0 is not an error, per-say. Although it may need special handling.

For three, it says that occurred at retrieval attempt 3 of 3, but the preceding log entries strongly imply that no data was downloaded (because attempt 1 and 2 got no new data), and no errors occurred during attempt 3 of 3.

DEFAULT  2019-04-17 06:03:06,418: Successful retrieval, downloaded data so resetting retry count and verify.
DEFAULT  2019-04-17 06:03:06,418: Trying new retrieval attempt 1 of 3.
DEFAULT  2019-04-17 06:03:17,791: Downloading TA_540A 2010-143 (N_S 1/1; day 1/1)
DEFAULT  2019-04-17 06:03:18,604: Successful retrieval attempt 1 of 3, but no new data downloaded so try again.
DEFAULT  2019-04-17 06:03:18,605: Trying new retrieval attempt 2 of 3.
DEFAULT  2019-04-17 06:03:23,497: Downloading TA_540A 2010-143 (N_S 1/1; day 1/1)
DEFAULT  2019-04-17 06:03:24,132: Successful retrieval attempt 2 of 3, but no new data downloaded so try again.
DEFAULT  2019-04-17 06:03:24,132: Trying new retrieval attempt 3 of 3.
DEFAULT  2019-04-17 06:03:28,692: Downloading TA_540A 2010-143 (N_S 1/1; day 1/1)
DEFAULT  2019-04-17 06:03:29,433: The latest retrieval attempt had no errors.  Stopping at 3 retry attempts.

[chad-earthscope]

Test case: rover retrieve TA_540A_*_* 2010-5-23 2010-5-24

As an aside: the problem turns out to be a request for a two channels that only have a single sample for the entire day. So like the 0 sample rate channels these data have no coverage. This implies there is a problem with the extraction system, independent of ROVER. Until it's fixed this remains as a way to see the error in ROVER.

retrieve  DEFAULT: ROVER version 1.0.1 - starting retrieve
retrieve  DEFAULT: Status available at http://127.0.0.1:8000
retrieve  DEFAULT: Trying new retrieval attempt 1 of 3.
retrieve  DEFAULT: Downloading TA_540A 2010-143 (N_S 1/1; day 1/2)
retrieve  DEFAULT: Downloading TA_540A 2010-144 (N_S 1/1; day 2/2)
retrieve  DEFAULT: Successful retrieval, downloaded data so resetting retry count and verify.
retrieve  DEFAULT: Trying new retrieval attempt 1 of 3.
retrieve  DEFAULT: Downloading TA_540A 2010-143 (N_S 1/1; day 1/1)
retrieve  DEFAULT: Successful retrieval attempt 1 of 3, but no new data downloaded so try again.
retrieve  DEFAULT: Trying new retrieval attempt 2 of 3.
retrieve  DEFAULT: Downloading TA_540A 2010-143 (N_S 1/1; day 1/1)
retrieve  DEFAULT: Successful retrieval attempt 2 of 3, but no new data downloaded so try again.
retrieve  DEFAULT: Trying new retrieval attempt 3 of 3.
retrieve  DEFAULT: Downloading TA_540A 2010-143 (N_S 1/1; day 1/1)
retrieve  DEFAULT: The latest retrieval attempt had no errors.  Stopping at 3 retry attempts.
retrieve  DEFAULT: 
retrieve  DEFAULT: ----- Retrieval Finished -----
retrieve  DEFAULT: 
retrieve  DEFAULT: 
retrieve  DEFAULT: A ROVER retrieve task on mint.iris.washington.edu
retrieve  DEFAULT: started 2019-04-18T14:43:22 (2019-04-18T21:43:22 UTC)
retrieve  DEFAULT: has completed in 3.08 seconds
retrieve  DEFAULT: 
retrieve  DEFAULT: The download for 1 stations totaled 1.5 MiB,
retrieve  DEFAULT: with data covering 227605 seconds.
retrieve  DEFAULT: 
retrieve  DEFAULT: A total of 6 downloads were made, with 0 errors (0 on final pass of 3).
retrieve  DEFAULT: 
retrieve  DEFAULT: WARNING: Inconsistent retrieval was detected, most likely
retrieve  DEFAULT:          data indicated by the availability service was not
retrieve  DEFAULT:          avaialble from the dataselect service.
retrieve  DEFAULT: 
retrieve CRITICAL: The previous retrieval attempt (3 of 3) downloaded unexpected data. Error potentially caused because of inconsistent web services or data had a sample rate = 0.
retrieve  DEFAULT: See "rover help retrieve"

[timronan]

What about changing to the message below? I will update and push the code when we find a message that we like.

The previous %d retrieval attempts could not download all data specified by the availability service. Your local repository likely mimics the data center's repository. Error potentially caused because of inconsistent availability and web services, the data had a sample rate = 0, or data gaps occur in the repository

[chad-earthscope]

This would very likely be wrong:

Your local repository likely mimics the data center's repository

This should not cause and error, we fixed this right?

the data had a sample rate = 0

why would gaps cause this error message? It's very normal to have gaps.

data gaps occur in the repository

Go back to basics: when does this error actually happen, what triggers this? Then we can work on the words.

Also, we should include a suggestion if possible, such as running list-retrieve so the user can see what they are "missing" if anything and make a judgement on whether they care or not.

[timronan]

This error occurs because the availability service incorrectly indicates data that is not available in the data select service. Data select tries to grab the data and cannot. The local index is built, the indexes are compared and the local repository's index does not mimic the availability service, instead the local index is a copy of the data select service index.

This error used to happen when data with a sample rate of 0 were requested. It was fixed in either issue #47 and or #63. Because of these legacy cases, I think it is still worthy to add 0 sample rate case to the warnings.

This error can also occur when there is a gap in the requested data, then a few samples, and another gap (EX: rover retrieve TA540A_ 2010-5-23 2010-5-27). Or as noted above:

having a single sample for the entire day

is a gap if you request multiple days of data. I use the catch all word gaps to try and concisely describe the phenomenon above, but there may be a better word or phrase.

We should add something like:

Your local repository indicates that all requested data were not received. Consider running rover list-retrieve to determine why this inconsistency is occurring.

[timronan]

Message to be updated.

The previous %d retrieval attempts could not download some requested data available at the data center. Consider runningrover list-retrieveto determine missing data.

[timronan]

Updated by pull request #101