Data Quality Issues

[joachim-gassen]

Dear OxCRGT Team:

Thank you again for your tremendous data collection efforts and for putting this repo online. This is a very good idea. I maintain a related R package {tidycovid19}. Given timeliness and importance of this data, I have been looking into the data quality of this project in the past. See here for an admittedly dated blog post. I would like to mention three important issues that I believe addressing would improve the internal consistency of your data. All points are substantiated with some R code below.

1. In this repo you are not providing the notes to the data (the CSV link on your webpage does). As the notes provide the reference for a certain measurement, I think that they are absolutely essential for researchers to assess the quality of your data. I would definitely include them here.

2. Zero values compared to missing values. Based on the code below, only 7 % of your zero values are supported by references, while more than 90 % of the other measures are. Yet, zero measures are the most frequent data value in your analysis (44 % of the non-NA cases). I would strongly encourage you to reconsider all zero measures that you do not have references for. For me it seems as if your zeros are in some cases informed statements about certain measures not being present whereas in most cases they indicate nothing different from NA.

3. Organization of data by country-day observations. This is causing me a lot of headaches. Another high-quality repository for government intervention data provided by ACAPS provides its data in country-response structure, meaning that a government response characterizes an observation. This is also how regulatory intervention data is normally stored. It makes much more sense from a data collection standpoint and allows one to validate your data much quicker. For example, in order to assess how many of your interventions are actually supported by references, one first has to assess changes over time, assume that these are driven by government interventions, and then calculate the according statistics. See the code below for what I mean. Transforming your data to a country-response structure reduces your 143,874 country-response type-day observations to 8,184 country-response observations without any loss if information. When you focus on changes in the actual measures (and not in the references that are not included on Github), this number reduces even more to 4,310. For this focused sample it is much easier to provide data quality assurance and you can still produce any view/slice and cut of the data as you like.

Minor things that you might want to clean up

• There is a trailing comma in your CSV
• I would remove the confirmed cases or confirmed deaths data as these are not an integral part of your data collection. Alternatively, I would provide a reference to the data source or explain how you collected that data.

Thank you for listening and again for your contributions to open science!

Joachim

PS: Code follows

library(tidyverse)
library(lubridate)
github_url <- "https://raw.githubusercontent.com/OxCGRT/covid-policy-tracker/master/data/OxCGRT_latest.csv"
web_url <- "https://ocgptweb.azurewebsites.net/CSVDownload"

github_data <- read_csv(github_url) %>% select(-X27)
web_data <- read_csv(web_url) %>% select(-X40)

web_data %>% select(-ends_with("Notes")) %>% all_equal(github_data)

# [1] TRUE

# Web data is equal to Github data but contains Notes

# Reorganize to long to ease comparisons

df <- web_data
names(df)[c(seq(from = 4, by = 3, length.out = 7), 32, 34)] <- paste0("S", c(1:7, 12, 13), "_Measure")

long_dta <- df %>% select(1:23, 32:35) %>%
  # S7, S12, S13 have no "IsGeneral" value. I attach NA vars for consistency
  mutate(S7_IsGeneral = NA,
                S12_IsGeneral = NA,
                S13_IsGeneral = NA) %>%
  pivot_longer(4:30, names_pattern = "(.*)_(.*)", names_to = c("Type", ".value")) %>%
  mutate(Date = ymd(Date)) %>%
  arrange(CountryName, Type, Date)

nrow(long_dta)

# [1] 143874

# Give speaking names to Type

mat <- str_split_fixed(names(web_data)[4:35], "_", 2)
df <- tibble(
  Type = mat[, 1],
  ResponseType = mat[, 2]
) %>%
  filter(
    ResponseType != "Notes",
    ResponseType != "IsGeneral"
  )

long_dta <- long_dta %>% left_join(df, by = "Type") %>% select(-Type) %>%
  select(CountryName, CountryCode, ResponseType, Date, everything())

  # Focus on obs that are either the first or that contain changes from prior date

gov_resp <- long_dta %>%
  group_by(CountryCode, ResponseType) %>%
  filter(
    (row_number() == 1 &
       (!is.na(IsGeneral) | !is.na(Measure) | !is.na(Notes)))  |
      (is.na(lag(IsGeneral)) & !is.na(IsGeneral)) |
      (is.na(lag(Measure)) & !is.na(Measure)) |
      (is.na(lag(Notes)) & !is.na(Notes)) |
      (!is.na(lag(IsGeneral)) & is.na(IsGeneral)) |
      (!is.na(lag(Measure)) & is.na(Measure)) |
      (!is.na(lag(Notes)) & is.na(Notes)) |
      (lag(IsGeneral) != IsGeneral) |
      (lag(Measure) != Measure) |
      (lag(Notes) != Notes)
  ) %>%
  ungroup()

nrow(gov_resp)

# [1] 8184

# 8,184 observations that reflect changes in the data - impressive

# But, unfortunately, many of those only reflect changes that are just driven
# by notes (that are sometimes sticky, sometimes not) that are marginally
# changed or omitted after the initial day. See for example in the raw long data:

long_dta[72:76,]

# # A tibble: 5 x 7
# CountryName CountryCode ResponseType  Date       Measure IsGeneral Notes
# <chr>       <chr>       <chr>         <date>       <dbl>     <dbl> <chr>
# 1 Afghanistan AFG         School closi… 2020-03-12       0        NA  NA
# 2 Afghanistan AFG         School closi… 2020-03-13       0        NA  NA
# 3 Afghanistan AFG         School closi… 2020-03-14       2         1 "\n\nOn March 14, 2020, the…
# 4 Afghanistan AFG         School closi… 2020-03-15       2         1 "On March 14, 2020, the Afg…
# 5 Afghanistan AFG         School closi… 2020-03-16       2         1 "On March 14, 2020, the Afg…

# compared to

long_dta[161:165,]

# # A tibble: 5 x 7
# CountryName CountryCode ResponseType   Date       Measure IsGeneral Notes
# <chr>       <chr>       <chr>          <date>       <dbl>     <dbl> <chr>
# 1 Afghanistan AFG         Testing frame… 2020-02-21       0        NA  NA
# 2 Afghanistan AFG         Testing frame… 2020-02-22       0        NA  NA
# 3 Afghanistan AFG         Testing frame… 2020-02-23       1        NA "'The Ministry of Public H…
# 4 Afghanistan AFG         Testing frame… 2020-02-24       1        NA  NA
# 5 Afghanistan AFG         Testing frame… 2020-02-25       1        NA  NA

# To identify "real" government interventions, I focus on measure changes
# and discard changes that are just driven by the notes.

gov_resp %>%
  group_by(CountryCode, ResponseType) %>%
  filter((row_number() == 1 ) |
           (lag(Measure) != Measure) |
           (lag(IsGeneral) != IsGeneral) |
           (is.na(Measure) & !is.na(lag(Measure))) |
           (!is.na(Measure) & is.na(lag(Measure))) |
           (is.na(IsGeneral) & !is.na(lag(IsGeneral))) |
           (!is.na(IsGeneral) & is.na(lag(IsGeneral)))) %>%
  mutate(NotesThere = !is.na(Notes)) %>%
  ungroup() -> gov_resp

nrow(gov_resp)

# [1] 4310 - still impressive

gov_resp %>%
  group_by(ResponseType) %>%
  summarise(
    N = n(),
    PctNotes = sum(NotesThere)/n()
  ) %>% arrange(-N)

# A tibble: 9 x 3
# ResponseType                          N PctNotes
# <chr>                             <int>    <dbl>
# 1 International travel controls       527    0.488
# 2 Restrictions on internal movement   515    0.447
# 3 Workplace closing                   495    0.432
# 4 Cancel public events                493    0.426
# 5 School closing                      485    0.416
# 6 Public information campaigns        464    0.379
# 7 Testing framework                   463    0.404
# 8 Close public transport              453    0.342
# 9 Contact tracing                     415    0.345
# Compare this across type of measure changes

gov_resp %>%
  group_by(Measure) %>%
  summarise(
    N = n(),
    PctNotes = sum(NotesThere)/n()
  ) %>% arrange(Measure)

# A tibble: 5 x 3
# Measure     N PctNotes
# <dbl> <int>    <dbl>
# 1       0  1414   0.0686
# 2       1   642   0.924
# 3       2   973   0.941
# 4       3   169   0.923
# 5      NA  1112   0.0108

# While most of the non-zero measures are supported by refernces ony 6% of th
# zeros are. Yet they are by far the most frequent observation type in the data
# My hunch: Most of the zeros are actuall not backed by actual "events" or
# clear evidence that there is no such event.

# Share of zero measures

sum(gov_resp$Measure == 0, na.rm = TRUE)/sum(!is.na(gov_resp$Measure))

# [1] 0.4421513

[TobyPhillips]

Hi Joachim, great to hear from you!

Some quick responses to your points:

1. yep, this is next on the list of things to build into this repo! (should get to it this weekend)
2. this is a tougher one – zeros are intended to be a deliberate statement that there is no policy worthy of coding as a 1 or higher. But we don't require our data collectors to provide references to prove there is not a policy every time they code a 0. Indeed, our intention is that our data collectors only record a note when a policy changes, rather than every single day (as you refer to the 'stale' notes in your blog post), but this doesn't always happen.
3. we started recording data in the country-day format to make it as instantly usable as possible – for econometric analysis and for simple timeseries display for researchers and journalists. As you say, if we stored data at the level of country-response, we could still transform it into whatever format we wanted for output. We are thinking about ways around this; but again, it is not a very high priority compared to other system fixes we are looking at on the back-end. Could you describe more about the headaches it is causing? At the moment we see it is an inconvenience but not a serious problem – but happy to hear otherwise.

Cheers! -Toby

[joachim-gassen]

Hi Toby, thank you for coming back so quickly and great to hear that you will be including the notes going forward. About your other points:

2. this is a tougher one – zeros are intended to be a deliberate statement that there is no policy worthy of coding as a 1 or higher. But we don't require our data collectors to provide references to prove there is not a policy every time they code a 0. Indeed, our intention is that our data collectors only record a note when a policy changes, rather than every single day (as you refer to the 'stale' notes in your blog post), but this doesn't always happen.

For my analysis, I first excluded all daily observations where the measures are unchanged. So, I only count the zeros that reflect policy changes or initiations of the data. For these zeros you have only notes on file for 7% of the cases meaning that the vast majority of zeros are not backed by evidence. This seems also closely related to the next point.

3. we started recording data in the country-day format to make it as instantly usable as possible – for econometric analysis and for simple timeseries display for researchers and journalists. As you say, if we stored data at the level of country-response, we could still transform it into whatever format we wanted for output. We are thinking about ways around this; but again, it is not a very high priority compared to other system fixes we are looking at on the back-end. Could you describe more about the headaches it is causing? At the moment we see it is an inconvenience but not a serious problem – but happy to hear otherwise.

I think this issue has two aspects: A theoretical and a practical one.

Let me start with the theory: Data collection requires a clear understanding what a data point is. A data point needs to be factual and supported by evidence. A governmental intervention/response provides such a data point. It is observable, you can provide a reference. Another person can re-assess its existence.

Now let's assume that you want country-day levels of governmental responses as data points. A data collector facing this task would most likely start searching for governmental responses on various sources. Then, assuming that they do not find any evidence for a governmental response they will code the respective day as zero. But what is the evidence for this data point? "Tried real hard to find something but did not find anything"?

Conceptually, the country-day level assessment of government responses is an interpretation of the underlying data (governmental responses). The analyst evaluates the data (country-level responses that are still current for a given day) and produces an output (the overall measure for a given day). This can be done algorithmically, e.g., by simply adding up scores, or by more complex assessments but the underlying data remain the single documented responses.

Now to why all this matters from a practical standpoint: First, you want to make sure that you can efficiently collect data on changes in governmental interventions. As we are all hoping that all these governmental responses are temporary, and we are already seeing some of them being lifted, this is super important. When you code your data at the response level, this is easy to do. You can give each response a termination date. When a response is modified, you can terminate it and generate a new one.

If however, you code your data as country-day, this becomes impossible. Let's assume that a country has established three different types of social distancing measures and is lifting one of them. How would you reflect this in your data? You can reduce the overall measure for social distancing which lumps together all responses and add a note to explain what you did but these notes are not easily processable. And from the overall measure alone nobody will be able to assess which response was terminated. You are destroying information with this coding scheme.

Another, but less important, practical aspect that is already apparent in your data is that is much harder to verify information in the country-day structure. Let's assume that you want to impose basic data quality checks on your data. A straight-forward check would be to demand that every response change needs to supported by a note. To test this in a country-response data structure you simply impose that the note field cannot be missing and maybe test for basic robustness of the note content. In the country-day setting you would first have to limit your sample to the observations that actual change compared to the day before and then do the check.

This sounds as if it is a minor technical issue but it is not since there might be cases where an analyst decides that, while there is a change in the governmental response on a given day, this change is not substantial enough to warrant a change in the measure. They will add a note but not change the measure. How do you verify that this happens whenever there is a response change observed? For me, as an intermediary user of your data, this causes the issue that I do not know when to discard notes on not change days. Most notes on non-change days are "sticky stales" (unfortunately in several cases with white space differences or similar, making them technically different) and thus can be discarded but in some observations, notes on no-measure change days are actually indicative of some relevant governmental activities.

The good news is that you can still relatively easily change the format of your data. The code in my post above contains some pointers on how to do that. But my guess is that you need to take this decision relatively quickly as more and more governmental responses are being changed and/or lifted, raising the problem outlined above that you conceptually cannot address this in your existing data structure.

Sorry. This got much longer than I initially planed. Am I making sense to you?

Joachim

[TobyPhillips]

Hi Joachim,

Sorry for taking a while to pick this back up. We just did a marathon restructure of our database over the last couple of weeks.

The points you are making do make sense. We are still going with a country-day format, but I think our changes may have addressed some of your concerns. For instance, we have removed all redundant/stale notes, and put in place many more server-side validation and processing rules to make it a neater data set.

The issue about zeroes not being supported by notes is more worrying. I haven't had the time to look very closely at this one myself, but I wonder, if you exclude the initial state (eg periods of zero that start at 1 Jan) and if you exclude the columns of "sX_is_general" flags (now titled "c/e/hX_flag" in the new scheme) which do not have their own notes, is this still a problem?

-Toby

[TobyPhillips]

Hi Joachim,

Would you mind checking out data/OxCGRT_latest_allchanges.csvand telling me what you think?

At the moment it includes any change in the database, including if the policy value stays the same and only a new note is added on a later day. We are also working to produce a version that will only include a row when a policy value changes.

-Toby

[joachim-gassen]

Hi there,

sorry for the silence and thank you for pointing me to the new data format. I had a quick look. In principle, I like the data organisation much better (big surprise)!

However, I am not sure how exactly you define a 'change' as there are 43 observations in that file that are identical to directly prior ones, besides the Date variable. It could be that these are generated by the notes being different in your local encoding but this difference being lost when you are exporting to CSV but this is just a guess.

In any case it shows how hard it is to identify an actual "event" in your data if you do not discard the notes.

Any ideas what is causing these duplicates and how to interpret them going forward?

Joachim

Below is the code that I used and the duplicates for your reference

library(dplyr)
library(readr)

df_raw <- read_csv("https://raw.githubusercontent.com/OxCGRT/covid-policy-tracker/master/data/OxCGRT_latest_allchanges.csv")

df_sorted <- df_raw %>%
  arrange(CountryCode, PolicyType, Date) %>%
  group_by(CountryCode, PolicyType) %>% 
  mutate(
    dup_obs = row_number() > 1 &
      PolicyValue == lag(PolicyValue) &
      Flag == lag(Flag) &
      Notes == lag(Notes) 
  )

which(df_sorted$dup_obs)

[1]   30  166  220  265  353  367  681  789  901  905  906 1092 1308 1310 1312 1317 1725
[18] 1819 1838 1854 1949 2029 2211 2237 2252 2288 2330 2417 2483 2971 2972 2986 3466 3778
[35] 4151 5040 5063 5242 5428 5613 5762 5983 6166

CountryName	CountryCode	Date	PolicyType	PolicyValue	Flag	Notes	dup\_obs
Afghanistan	AFG	20200314	C1: School closing	3	1	On March 14, 2020, the Afghan government announced closure of schools for a month. Academic year in most of the country beings on March 21st. As of now, the schools will not start the new academic year until April 21, 2020:	FALSE
Afghanistan	AFG	20200315	C1: School closing	3	1	On March 14, 2020, the Afghan government announced closure of schools for a month. Academic year in most of the country beings on March 21st. As of now, the schools will not start the new academic year until April 21, 2020:	TRUE
United Arab Emirates	ARE	20200329	C3: Cancel public events	2	1	The National Emergency Crisis and Disaster Management Authority and the General Authority of Islamic Affairs and Endowments have decided to temporarily suspend prayers in all places of worship across the nation, effective today at 9 p.m. local time on the 16th of March, for a period of four weeks out of concern for public safety.	FALSE
United Arab Emirates	ARE	20200410	C3: Cancel public events	2	1	The National Emergency Crisis and Disaster Management Authority and the General Authority of Islamic Affairs and Endowments have decided to temporarily suspend prayers in all places of worship across the nation, effective today at 9 p.m. local time on the 16th of March, for a period of four weeks out of concern for public safety.	TRUE
Argentina	ARG	20200319	C6: Stay at home requirements	3	1	Measures were put in place 19/03, renewed 31/03 until 12/04 and renewed 11/04 until 26/04. ***///*** ***///***	FALSE
Argentina	ARG	20200418	C6: Stay at home requirements	3	1	Measures were put in place 19/03, renewed 31/03 until 12/04 and renewed 11/04 until 26/04. ***///*** ***///***	TRUE
Australia	AUS	20200403	C1: School closing	3	0	The Federal Education Minister of Australia has requested independent schools to reopen for term two. See the press report published on April 3: According to this press report published on April 8, Australian schools remain open:	FALSE
Australia	AUS	20200408	C1: School closing	3	0	The Federal Education Minister of Australia has requested independent schools to reopen for term two. See the press report published on April 3: According to this press report published on April 8, Australian schools remain open:	TRUE
Austria	AUT	20200423	C7: Restrictions on internal movement	1	1	Tyrolean authorities lift remaining lockdown in Paznaun, St. Anton and Sölden; no region of Austria under enforced cross-region lockdown, though government still recommends limiting movement;	FALSE
Austria	AUT	20200426	C7: Restrictions on internal movement	1	1	Tyrolean authorities lift remaining lockdown in Paznaun, St. Anton and Sölden; no region of Austria under enforced cross-region lockdown, though government still recommends limiting movement;	TRUE
Austria	AUT	20200403	H1: Public information campaigns	2	1		FALSE
Austria	AUT	20200418	H1: Public information campaigns	2	1		TRUE
Belize	BLZ	20200320	C4: Restrictions on gatherings	3	1	“Social gatherings will now be limited to 25 people. This includes wakes, funerals, weddings, and other social events.”	FALSE
Belize	BLZ	20200418	C4: Restrictions on gatherings	3	1	“Social gatherings will now be limited to 25 people. This includes wakes, funerals, weddings, and other social events.”	TRUE
Brazil	BRA	20200423	C1: School closing	3	0		FALSE
Brazil	BRA	20200426	C1: School closing	3	0		TRUE
Barbados	BRB	20200401	E1: Income support	1	0	Laid off workers to receive benefits for 6 months and those on short weeks to receive 60% for the days they are not working; government, through the welfare department, to provide minimum income of an amount up to USD300 per month to households where no person is left employed as a result of COVID-19. See: . See also:	FALSE
Barbados	BRB	20200507	E1: Income support	1	0	Laid off workers to receive benefits for 6 months and those on short weeks to receive 60% for the days they are not working; government, through the welfare department, to provide minimum income of an amount up to USD300 per month to households where no person is left employed as a result of COVID-19. See: . See also:	TRUE
Barbados	BRB	20200403	H1: Public information campaigns	2	1	Barbados has established a COVID-19 information hotline that can be reached at 246-536-4500. Source: Additionaly, information is being posted on the Government Information Service website with a special web page dedicated to Covid-19. Source:	FALSE
Barbados	BRB	20200419	H1: Public information campaigns	2	1	Barbados has established a COVID-19 information hotline that can be reached at 246-536-4500. Source: Additionaly, information is being posted on the Government Information Service website with a special web page dedicated to Covid-19. Source:	TRUE
Barbados	BRB	20200504	H1: Public information campaigns	2	1	Barbados has established a COVID-19 information hotline that can be reached at 246-536-4500. Source: Additionaly, information is being posted on the Government Information Service website with a special web page dedicated to Covid-19. Source:	TRUE
Chile	CHL	20200314	H1: Public information campaigns	2	1	Ministry of health announces COVID response has moved to Phase three, asks all citizens to take self-care measures, quarantines to be respected, handshakes, kisses and the most efficient measure, which is frequent hand washing, to be avoided. Launches government website;	FALSE
Chile	CHL	20200427	H1: Public information campaigns	2	1	Ministry of health announces COVID response has moved to Phase three, asks all citizens to take self-care measures, quarantines to be respected, handshakes, kisses and the most efficient measure, which is frequent hand washing, to be avoided. Launches government website;	TRUE
Congo	COG	20200328	C1: School closing	3	1	Places of worship, schools, higher education institutions, bars and nightclubs are closed.	FALSE
Congo	COG	20200330	C1: School closing	3	1	Places of worship, schools, higher education institutions, bars and nightclubs are closed.	TRUE
Congo	COG	20200328	C2: Workplace closing	3	1	There is only an exception for workers providing essential goods and services.	FALSE
Congo	COG	20200330	C2: Workplace closing	3	1	There is only an exception for workers providing essential goods and services.	TRUE
Congo	COG	20200328	C3: Cancel public events	2	1	Places of worship, schools, higher education institutions, bars and nightclubs are closed. Gatherings of more than 50 people as well as of family events such as weddings and funerals are prohibited.	FALSE
Congo	COG	20200330	C3: Cancel public events	2	1	Places of worship, schools, higher education institutions, bars and nightclubs are closed. Gatherings of more than 50 people as well as of family events such as weddings and funerals are prohibited.	TRUE
Congo	COG	20200328	C6: Stay at home requirements	3	1	Lockdown measures include prohibiting people from leaving home and a nationwide curfew from 8 pm until 5 am.	FALSE
Congo	COG	20200330	C6: Stay at home requirements	3	1	Lockdown measures include prohibiting people from leaving home and a nationwide curfew from 8 pm until 5 am.	TRUE
Denmark	DNK	20200311	C2: Workplace closing	1	1	Recommendation on March 11 that all employees in non-critical functions should be sent home to work from home. Sending home all non-essential public sector employees occurs on 13th March, schools close on 16th March. They should work from home if possible, but if not should be given paid leave. However employees in the health sector, the elderly care sector and the police have to stay at their posts.	FALSE
Denmark	DNK	20200313	C2: Workplace closing	1	1	Recommendation on March 11 that all employees in non-critical functions should be sent home to work from home. Sending home all non-essential public sector employees occurs on 13th March, schools close on 16th March. They should work from home if possible, but if not should be given paid leave. However employees in the health sector, the elderly care sector and the police have to stay at their posts.	TRUE
Ecuador	ECU	20200416	C1: School closing	3	1		FALSE
Ecuador	ECU	20200424	C1: School closing	3	1		TRUE
Ecuador	ECU	20200410	H1: Public information campaigns	2	1	Official website up to date on all COVID-19 related information. Source:	FALSE
Ecuador	ECU	20200416	H1: Public information campaigns	2	1	Official website up to date on all COVID-19 related information. Source:	TRUE
Egypt	EGY	20200401	C5: Close public transport	2	1	The Egyptian government has announced the suspension of public transport across the country for two weeks starting from the 25th of March 2020. Source: Al Jazeera Arabic	FALSE
Egypt	EGY	20200406	C5: Close public transport	2	1	The Egyptian government has announced the suspension of public transport across the country for two weeks starting from the 25th of March 2020. Source: Al Jazeera Arabic	TRUE
Estonia	EST	20200327	H1: Public information campaigns	2	1	Government of Estonia has a dedicated website, FAQ sheets, call centre, informational material and an A.I bot answering questions on COVID. There are coordinated communication materials and FAQs from all sectors available online / news media.	FALSE
Estonia	EST	20200427	H1: Public information campaigns	2	1	Government of Estonia has a dedicated website, FAQ sheets, call centre, informational material and an A.I bot answering questions on COVID. There are coordinated communication materials and FAQs from all sectors available online / news media.	TRUE
France	FRA	20200317	C2: Workplace closing	3	1		FALSE
France	FRA	20200319	C2: Workplace closing	3	1		TRUE
Greece	GRC	20200410	C1: School closing	3	1	ATHENS - According to an informal briefing by the Ministry of Education, the temporary ban of the operation of all educational institutions, such as schools, universities and others, will be extended until May 10, 2020.	FALSE
Greece	GRC	20200504	C1: School closing	3	1	ATHENS - According to an informal briefing by the Ministry of Education, the temporary ban of the operation of all educational institutions, such as schools, universities and others, will be extended until May 10, 2020.	TRUE
Greece	GRC	20200504	C7: Restrictions on internal movement	2	0	In order for the first phase of easing the restrictive measures to have the expected results, the Government decided to limit the movement of citizens within the limits of the relevant Regional Unit. In the case of Attica, the traffic restriction is done within the limits of the Region. For the whole country, travel to the islands located in the same Region is excluded. The restriction is valid for the period from 4.5.2020 to 18.5.2020. JMC No. ?1a / G?.???. 27818, Government Gazette ??1648 / 03.05.2020	FALSE
Greece	GRC	20200511	C7: Restrictions on internal movement	2	0	In order for the first phase of easing the restrictive measures to have the expected results, the Government decided to limit the movement of citizens within the limits of the relevant Regional Unit. In the case of Attica, the traffic restriction is done within the limits of the Region. For the whole country, travel to the islands located in the same Region is excluded. The restriction is valid for the period from 4.5.2020 to 18.5.2020. JMC No. ?1a / G?.???. 27818, Government Gazette ??1648 / 03.05.2020	TRUE
Greece	GRC	20200301	H1: Public information campaigns	2	1	“Greece’s Health Ministry is stepping up a public information plan about how people can protect themselves from the coronavirus, with seven cases in the country and the European Union raising its alert level to high for the disease, now called Covid-19. The Health Ministry is putting messages on TV and social media aimed at ensuring that people observe basic hygiene rules, especially washing hands regularly and the right way, to ward off spreading or contracting the disease.”	FALSE
Greece	GRC	20200505	H1: Public information campaigns	2	1	“Greece’s Health Ministry is stepping up a public information plan about how people can protect themselves from the coronavirus, with seven cases in the country and the European Union raising its alert level to high for the disease, now called Covid-19. The Health Ministry is putting messages on TV and social media aimed at ensuring that people observe basic hygiene rules, especially washing hands regularly and the right way, to ward off spreading or contracting the disease.”	TRUE
Guatemala	GTM	20200413	C1: School closing	3	1	Measures were renewed on 13/04. For schools, until 30/04 (see article 7).	FALSE
Guatemala	GTM	20200418	C1: School closing	3	1	Measures were renewed on 13/04. For schools, until 30/04 (see article 7).	TRUE
Guatemala	GTM	20200413	H1: Public information campaigns	2	1	Measures were renewed on 13/04 (see article 15). ***///*** Presidential Decree formally institutes information campaigns (see article 5)	NA
Guatemala	GTM	20200420	H1: Public information campaigns	2	1	Measures were renewed on 13/04 (see article 15). ***///*** Presidential Decree formally institutes information campaigns (see article 5)	TRUE
Hong Kong	HKG	20200126	C3: Cancel public events	2	1	The HKSAR Government will cancel large-scale events in the coming future which it organises or arranges and are expected to be attended by many people. Apart from the International Chinese New Year Carnival and the Lunar New Year Cup football tournament which have been cancelled earlier, lantern carnivals organised by the Leisure and Cultural Services Department will also be cancelled. The HKSAR Government has reached a consensus with the organiser of the Hong Kong Marathon that the event originally scheduled for February 9 will also be cancelled. In addition, as the HKSAR Government has to focus on fighting the disease, various types of Chinese New Year receptions hosted by the HKSAR Government, including the one hosted by the Chief Executive in the Government House, will be cancelled.. Source:	FALSE
Hong Kong	HKG	20200127	C3: Cancel public events	2	1	The HKSAR Government will cancel large-scale events in the coming future which it organises or arranges and are expected to be attended by many people. Apart from the International Chinese New Year Carnival and the Lunar New Year Cup football tournament which have been cancelled earlier, lantern carnivals organised by the Leisure and Cultural Services Department will also be cancelled. The HKSAR Government has reached a consensus with the organiser of the Hong Kong Marathon that the event originally scheduled for February 9 will also be cancelled. In addition, as the HKSAR Government has to focus on fighting the disease, various types of Chinese New Year receptions hosted by the HKSAR Government, including the one hosted by the Chief Executive in the Government House, will be cancelled.. Source:	TRUE
Honduras	HND	20200316	C6: Stay at home requirements	3	1	General curfew ordered. There is a schedule for individuals to be outside their homes based on the number of national ID card. This schedule is not the same for the provinces of the provinces of Cortés, Colón and a city in Yoro. Orders of curfew are listed below from most recent (26/04) to oldest starting from the 16/03 extension: ***///*** ***///*** ***///*** ***///***	FALSE
Honduras	HND	20200418	C6: Stay at home requirements	3	1	General curfew ordered. There is a schedule for individuals to be outside their homes based on the number of national ID card. This schedule is not the same for the provinces of the provinces of Cortés, Colón and a city in Yoro. Orders of curfew are listed below from most recent (26/04) to oldest starting from the 16/03 extension: ***///*** ***///*** ***///*** ***///***	TRUE
Italy	ITA	20200312	H1: Public information campaigns	2	1		FALSE
Italy	ITA	20200314	H1: Public information campaigns	2	1		TRUE
Italy	ITA	20200317	H1: Public information campaigns	2	1		TRUE
Jamaica	JAM	20200323	C1: School closing	3	1	Source:	FALSE
Jamaica	JAM	20200426	C1: School closing	3	1	Source:	TRUE
Libya	LBY	20200322	C7: Restrictions on internal movement	2	1	15/03/2020 - The education ministries of the Tripoli-based unity government and its eastern-based rival announced in separate statements that all the country’s schools, both public and private, will be closed from Sunday. 24/03/2020 - The Government of National Accords (GNA) based in Tripoli implemented an overnight curfew starting on Sunday, March 22. All movement between 18:00 and 06:00 (local time) is restricted within territory controlled by the GNA. Security, health workers, and other people involved in critical services are excluded from the curfew. Land and sea border crossing points within GNA territory were closed by Prime Minister Fayez al-Sarraj on Monday, March 16. All mosques, schools, restaurants, wedding halls, parks, and shops are also closed as a precaution. Food stores, bakeries, and fuel stations will remain open while some state institutions may reduce operations. 16/04/2020 - The besieged Tripoli administration ordered the shutdown of all large markets and non-essential shops in its territory, and banned cars from the roads. Citizens wearing masks may venture out by foot from 8am to 2pm, the statement said. Banks, a main source of crowding in recent weeks, would also close. 27/04/2020 – The Government of National Accord (GNA) announced on Thursday, April 23, that the ongoing coronavirus disease (COVID-19) curfew will be modified. The new curfew will be implemented on Monday, April 27, and will run from 18:00 to 06:00 (local time), and individuals will be permitted to operate vehicles, banned during the previous curfew. It is unclear as to how long the measure will remain in place.	FALSE
Libya	LBY	20200416	C7: Restrictions on internal movement	2	1	15/03/2020 - The education ministries of the Tripoli-based unity government and its eastern-based rival announced in separate statements that all the country’s schools, both public and private, will be closed from Sunday. 24/03/2020 - The Government of National Accords (GNA) based in Tripoli implemented an overnight curfew starting on Sunday, March 22. All movement between 18:00 and 06:00 (local time) is restricted within territory controlled by the GNA. Security, health workers, and other people involved in critical services are excluded from the curfew. Land and sea border crossing points within GNA territory were closed by Prime Minister Fayez al-Sarraj on Monday, March 16. All mosques, schools, restaurants, wedding halls, parks, and shops are also closed as a precaution. Food stores, bakeries, and fuel stations will remain open while some state institutions may reduce operations. 16/04/2020 - The besieged Tripoli administration ordered the shutdown of all large markets and non-essential shops in its territory, and banned cars from the roads. Citizens wearing masks may venture out by foot from 8am to 2pm, the statement said. Banks, a main source of crowding in recent weeks, would also close. 27/04/2020 – The Government of National Accord (GNA) announced on Thursday, April 23, that the ongoing coronavirus disease (COVID-19) curfew will be modified. The new curfew will be implemented on Monday, April 27, and will run from 18:00 to 06:00 (local time), and individuals will be permitted to operate vehicles, banned during the previous curfew. It is unclear as to how long the measure will remain in place.	TRUE
Mexico	MEX	20200330	C2: Workplace closing	3	1	Immediate suspension, from March 30 to April 30, 2020, of non-essential activities in the public, private and social sectors	FALSE
Mexico	MEX	20200415	C2: Workplace closing	3	1	Immediate suspension, from March 30 to April 30, 2020, of non-essential activities in the public, private and social sectors	TRUE
Namibia	NAM	20200327	C6: Stay at home requirements	2	0	For Khomas and Erdongo regions: “All residents in the restricted areas, including Okahandja and Rehoboth local authority areas are not allowed to leave their homes during the lockdown period, unless such movement by any person in the restricted area is for the following reasons: a. Travel by persons listed in Section 2 above; b. Visits to pharmacies, food supply stores, courts, banks and for medical reasons; c. Physical exercise in groups of not more than three (3) persons.”	FALSE
Namibia	NAM	20200418	C6: Stay at home requirements	2	0	For Khomas and Erdongo regions: “All residents in the restricted areas, including Okahandja and Rehoboth local authority areas are not allowed to leave their homes during the lockdown period, unless such movement by any person in the restricted area is for the following reasons: a. Travel by persons listed in Section 2 above; b. Visits to pharmacies, food supply stores, courts, banks and for medical reasons; c. Physical exercise in groups of not more than three (3) persons.”	TRUE
Sudan	SDN	20200314	C2: Workplace closing	1	1	Quotation: On 14 March, the Government of Sudan announced that government and private institutions should take measures to reduce congestion in the workplace. Link:	FALSE
Sudan	SDN	20200417	C2: Workplace closing	1	1	Quotation: On 14 March, the Government of Sudan announced that government and private institutions should take measures to reduce congestion in the workplace. Link:	TRUE
Sudan	SDN	20200313	H1: Public information campaigns	2	1	Quotation (bio): Sudanese Government Coved Outreach Initiative 19 - Coordination initiative to promote health awareness and education on Covid 19 and the new Coronavirus Link:	FALSE
Sudan	SDN	20200417	H1: Public information campaigns	2	1	Quotation (bio): Sudanese Government Coved Outreach Initiative 19 - Coordination initiative to promote health awareness and education on Covid 19 and the new Coronavirus Link:	TRUE
San Marino	SMR	20200225	H1: Public information campaigns	2	1	Covid-19 hotline active since February 25. Source: // Official information also available in	FALSE
San Marino	SMR	20200505	H1: Public information campaigns	2	1	Covid-19 hotline active since February 25. Source: // Official information also available in	TRUE
Slovenia	SVN	20200404	H1: Public information campaigns	2	1	COVID-19 Website Link: Coronavirus: Preparedness and measures in Slovenia and COVID-19 Helpline Link: Coronavirus: Preventing infections Link: Coronavirus: Symptoms of infection and treatment Link: Government measures Link: Recommendations for washing hands cough etiquette Link: Quotation: So far Slovenia has prepared hospital isolation rooms and urged citizens to be particularly careful with hygiene. Many hospitals have prohibited or limited visitations in recent days. (4th March 2020) Link: COVID-19 Press Conference: Official updates on the occurrence of the coronavirus infection (7 March 2020) Link: gov.si/en/news/2020-03-07-official-updates-on-the-occurrence-of-the-coronavirus-infection/ Latest information about coronavirus (Covid-19) in Slovenia (9 March 2020) Link: Quotation: The National Institute of Public Health (NIJZ) has drafted a leaflet with key information about the coronavirus, which will be distributed to all Slovenian households via the Post of Slovenia (10 March 2020) Link:	FALSE
Slovenia	SVN	20200418	H1: Public information campaigns	2	1	COVID-19 Website Link: Coronavirus: Preparedness and measures in Slovenia and COVID-19 Helpline Link: Coronavirus: Preventing infections Link: Coronavirus: Symptoms of infection and treatment Link: Government measures Link: Recommendations for washing hands cough etiquette Link: Quotation: So far Slovenia has prepared hospital isolation rooms and urged citizens to be particularly careful with hygiene. Many hospitals have prohibited or limited visitations in recent days. (4th March 2020) Link: COVID-19 Press Conference: Official updates on the occurrence of the coronavirus infection (7 March 2020) Link: gov.si/en/news/2020-03-07-official-updates-on-the-occurrence-of-the-coronavirus-infection/ Latest information about coronavirus (Covid-19) in Slovenia (9 March 2020) Link: Quotation: The National Institute of Public Health (NIJZ) has drafted a leaflet with key information about the coronavirus, which will be distributed to all Slovenian households via the Post of Slovenia (10 March 2020) Link:	TRUE
Chad	TCD	20200423	E1: Income support	2	1	CFAF 15 billion (0.3 percent of non-oil GDP) of health-related expenditures have been approved and are being implemented under a national contingency plan. Key measures include: (i) training of medical and technical staff, (ii) purchase of necessary medical equipment, (iii) construction of seven health centers in remote areas, (iv) construction of three mobile hospitals, and (v) securely managing entry points. The authorities have also decided on a package of fiscal measures to help households and businesses weather the shock. For small and medium-sized enterprises, the authorities will, among other things, reduce by 50 percent the business license fees and the presumptive tax for 2020. Tax breaks such as carryforward losses and delays in tax payments will also be examined on a case-by-case basis.	FALSE
Chad	TCD	20200424	E1: Income support	2	1	CFAF 15 billion (0.3 percent of non-oil GDP) of health-related expenditures have been approved and are being implemented under a national contingency plan. Key measures include: (i) training of medical and technical staff, (ii) purchase of necessary medical equipment, (iii) construction of seven health centers in remote areas, (iv) construction of three mobile hospitals, and (v) securely managing entry points. The authorities have also decided on a package of fiscal measures to help households and businesses weather the shock. For small and medium-sized enterprises, the authorities will, among other things, reduce by 50 percent the business license fees and the presumptive tax for 2020. Tax breaks such as carryforward losses and delays in tax payments will also be examined on a case-by-case basis.	TRUE
Tunisia	TUN	20200320	C7: Restrictions on internal movement	2	1	In a televised address, Saied announed new restrictions on public movement, saying city-to-city travel had been banned.https://web.archive.org/save/	FALSE
Tunisia	TUN	20200418	C7: Restrictions on internal movement	2	1	In a televised address, Saied announed new restrictions on public movement, saying city-to-city travel had been banned.https://web.archive.org/save/	TRUE
Ukraine	UKR	20200303	H1: Public information campaigns	2	1	The Ukrainian Government has created a special information page with all relevant information in addition to daily briefings by the chief medical officer and government officials:	FALSE
Ukraine	UKR	20200403	H1: Public information campaigns	2	1	The Ukrainian Government has created a special information page with all relevant information in addition to daily briefings by the chief medical officer and government officials:	TRUE
Venezuela	VEN	20200410	H1: Public information campaigns	2	1	Establishment of a national committee for corona, and public campaign to raise awareness	FALSE
Venezuela	VEN	20200428	H1: Public information campaigns	2	1	Establishment of a national committee for corona, and public campaign to raise awareness	TRUE

[TobyPhillips]

Thanks Joachim, that's really helpful. We've looked into the duplicates and there are a couple of reasons here:

1. odd characters in the raw database
2. a couple that seem to have been entered in a way that over-rides the redundant note check
3. duplicate notes that are not on the immediate-consecutive-day which are slipping through our redundant note check

We're working through these. We are also working on producing our own "event"/"response" level csv.

Regardless of the duplicate observations, you will still find there are many many cases where we add a note without changing the PolicyValue or Flag. For your purposes, I would suggest simply ignore the notes. And for that matter, ignore any "change" that has the same CountryCode, PolicyType, PolicyValue, Flag and Notes as the immediate most recent change for that same CountryCode/PolicyType.

Cheers, Toby

[TobyPhillips]

Just a heads up: we're now publishing a csv of policy responses... let me know what you think!

Cheers, Toby