search

SCBI-ForestGEO / 2023census

Repository for the 2023 recensus of the SCBI ForestGEO plot
Creative Commons Attribution 4.0 International
3 stars 0 forks source link

Issues with measurement dates: date_measured column & EditDate column #62

Open llmorreale opened 5 months ago

llmorreale commented 5 months ago

There appears to be an issue where the date_measured column is being updated when errors are fixed/data is edited on the iPads instead of the EditDate being fixed. Because of this, there is not a fully reliable way to find the original measurement date within the census dataset at the moment.

@jess-shue I think this is an issue in the app that may need to be addressed before the SERC census starts.

For now, I will be addressing this problem for the 2023 census manuscript by replacing measurement dates of stems that have a measurement date after the edit date using the modal measurement date of the quadrat.

jess-shue commented 5 months ago

Hi @llmorreale, The 'date_measured' field must be physically changed by the person operating the iPad. It isn't autopopulated - the 'EditDate' is. I setup the second 'date_measured' field because David told us about this problem. I think because I set it to default to the current date, when they re-submit, it is changing the date. Otherwise, they would need to enter the date manually on every stem during the census.

I just opened the app and played with it a bit. If I click on a stem, the original date is listed in 'date_measured'. If I click edit, it does change the 'date_measured' to the now default date and time of the edit. I can do two things:

  1. I can make the date editable in the app (it is in the data itself, but I've made it read-only in the app).
  2. I can turn off the auto date, so it will stop updating.

Again, for the census itself I think it is important to have these settings as-is so folks aren't having to enter the date for every stem. But, for conducting field edits I completely understand why this would be a problem.

Would you like me to change those settings?

teixeirak commented 5 months ago

Does this mean date_measured is always automatically matching EditDate? Of course, this will be correct in some cases (where DBH wasn't originally taken or had to be fixed) and incorrect in others (where something else was fixed). With GitHub, we would have all the information needed to reconstruct when the measurements were actually taken, but I doubt it would be worth the effort. I think Luca's approach of using the modal measurement date makes sense, although the best approach would depend on the most common error type.

For the future, would there be a way to auto-populate date_measured when DBH is edited?

jess-shue commented 5 months ago

@teixeirak Currently, yes because it is being calculated when the 'team' field is filled. Again, this was setup so during the census the technicians weren't having to enter it for every stem. At the time, this was most convenient and made sense, but I see how it is causing trouble.

I can turn that off and make it editable. If I turn off the calculation, 'date_measured' would hold as the first date. The 'EditDate' field would reflect the most recent submission for that stem.

I may also be able to have it work on the DBH field rather than the 'Team' field, but because the DBH is already filled, we may have the same issue. I either need to make some changes and do some testing, and/or also contact David and Milton. I had access to BCI's recensus group, so I was going to check their date field, but I no longer have access or it has been deleted since the census is finished.

llmorreale commented 5 months ago

@jess-shue I totally understand and agree that the field crew should not have to fill out the date manually in the field! For the future, if there is a way to have the date_measured field auto-populate only on the first instance and not on subsequent edits I believe that that would be ideal.

In the meantime regarding the completed census - on your end, is there a record of edits to the database? I agree with Krista that while possible to reconstruct the actual date of measurement from Github, the effort would be substantial. If there were a single log of edits, that would make it easier to fix the data.

jess-shue commented 5 months ago

@llmorreale Currently, that functionality is not possible. At BCI, I'm pretty sure they entered it each time. I was being 'smart' and trying to reduce that issue, but one 'solution' sometimes creates other problems...

Let me take a look for the list of edits. I'm sure it is recoverable/kept, but I have not accessed it previously.

jess-shue commented 5 months ago

@llmorreale, @teixeirak

Forgot to get clarification: I can turn off the calculation for the population of the date. This would leave it 'as-is', but the 'EditDate' field would track any changes. If folks are in the field today, I can do that once they've returned to prevent any disruptions.

llmorreale commented 5 months ago

@jess-shue I think turning that off for now would be great! No one is in the field, but we are occasionally updating the data on the iPad as we find errors that need to be fixed. Thanks very much!

jess-shue commented 5 months ago

@llmorreale Thank you Luca, I have turned off the Arcade calculation code that was running in the background. I just checked it in the app, and when I opened up the editing section the date did not change.

I have left the 'date_measured' field as read only, so it cannot be edited. The census measurement date should now remain when making edits to a tree. I am looking into how to download the change log for the previous edits.