Migrate WP category functionality to Grapher

[ikesau]

Our site navigation relies on WordPress to generate a hierarchical category structure. These categories are somewhat mirrored in Grapher, but the hierarchy is not.

In order to populate the header with links to GDocs-based topic pages, which are tagged with Grapher tags, we'll need to harmonize these two structures somehow.

renderMenuJson will need to be rewritten to depend on the Grapher version of the hierarchy, and pre-existing WP posts will need to be merged with it.

We will need to make sure that all functionality that depends on WP categories, e.g. getRelatedCharts migrates smoothly as well. I don't yet have an exhaustive list of all the features that we'll need to keep an eye on.

WordPress entry hierarchy

Entries:
- Energy and Environment:
  - Air and Climate
  - Energy
  - Land and Ecosystems
  - Waste and Pollution
  - Water
- Health:
  - Life and Death
  - Infectious Diseases
  - Health Risks
  - Health Institutions and Interventions
- Population and Demographic Change:
  - Population Change
  - Life and Death
  - Distribution of the World Population
- Food and Agriculture:
  - Nutrition
  - Food Production
  - Agricultural Inputs
- Education and Knowledge:
  - Educational Access and Outcomes
  - Inequality in Education
  - Inputs to Education
  - Media
- Innovation and Technological Change:
  - Technological Change
- Poverty and Economic Development:
  - Poverty
  - Poverty and Prosperity
  - Economic Inequality
  - Trade and Migration
  - Labor
  - Public Sector
  - Corruption
- Living Conditions Community and Wellbeing:
  - Happiness and Wellbeing
  - Relationships
  - Time Use
  - Culture
  - Housing
- Human Rights and Democracy:
  - Democracy and Corruption
  - Human Rights
- Violence and War:
  - Violence
  - War and Peace

Grapher entry hierachy

Entries:
  - Abstract:
    - Geographic
  - Culture, Values & Society:
    - Marriage & Relationships
    - Optimism & Pessimism
    - Personality Traits & Social Preferences
    - Quantitative View of the World
    - Sports
    - Teenage Pregnancy
    - Trust
    - Values
  - Economic Development, Work & Standard of Living:
    - Capital Formation
    - Child Labor
    - Consumption Baskets
    - Female Labor Supply
    - Happiness and Life Satisfaction
    - Homelessness
    - Housing
    - Human Development Index (HDI)
    - Industrialization
    - Informal Sector
    - Labor Force Participation & Employment
    - Labor Market Policies
    - Light
    - Quality of Life
    - Slavery
    - Start-Ups & Entrepreneurialism
    - Unemployment
    - Urbanization
    - Work Conditions & Safety
    - Working Hours
  - Education & Knowledge:
    - Educational Mobility & Inequality
    - Global Rise of Education
    - Intelligence
    - Literacy
    - Pre-Primary Education
    - Primary Education
    - Projections of Future Education
    - Quality of Education
    - Returns to Education
    - Science & Research
    - Secondary Education
    - Teachers & Professors
    - Tertiary Education
  - Environmental Change:
    - Air Pollution
    - Biodiversity
    - Climate Change
    - CO2 and Greenhouse Gas Emissions
    - Environmental Protection & Regulation
    - Fish and overfishing
    - Forests
    - Indoor Air Pollution
    - Irrigation
    - Land Use
    - Natural Disasters
    - Oil Spills
    - Ozone Layer
    - Plastic Pollution
    - Sanitation
    - Waste
    - Water Quality in Rivers and Oceans
    - Water Use
  - Food & Agriculture:
    - Agricultural Employment
    - Agricultural Production
    - Agricultural Regulation & Policy
    - Crop Yields
    - Diet Compositions
    - Environmental impacts of food production
    - Famines
    - Farm Size
    - Fertilizers
    - Food per Person
    - Food Prices & Expenditure
    - Food Trade
    - Food Waste
    - Human Height
    - Hunger & Undernourishment
    - Land Use and Yields in Agriculture
    - Meat and Seafood Production & Consumption
    - Micronutrient Deficiency
    - Non-food Crops
    - Obesity & BMI
    - Pesticides
    - Soil
  - Global Interconnections:
    - Foreign Aid
    - Foreign Direct Investment
    - Human Trafficking
    - Intergovernmental Organizations & Diplomacy
    - International Trade
    - Migration
    - Political Asylum & Refugees
    - Remittances
    - Tourism
  - Growth & Distribution of Prosperity:
    - Consumer Credit, Debt & Saving
    - Consumption Inequality
    - Economic Complexity
    - Economic Growth
    - Economic Inequality
    - Economic Inequality by Gender
    - Financial Inclusion
    - Financial Markets, Banks & Financial Institutions
    - Global Economic Inequality
    - Global Extreme Poverty
    - Incomes across the Distribution
    - Interest Rates & Credit Market
    - Labor & Capital Share
    - Multidimensional Poverty
    - Poverty
    - Poverty Relative to Poverty Lines Higher than the International Poverty Line
    - Prices
    - Productivity
    - Risk and Insurance
    - Wages
    - Wealth Inequality
  - Health:
    - Alcohol Consumption
    - Antibiotics
    - Burden of Disease
    - Cancer
    - Cardiovascular Diseases
    - Causes of Death
    - Child Health
    - Contraception & Abortion
    - Coronavirus disease [COVID-19]
    - Diarrheal Diseases
    - Doctors & Health Workers
    - Eradication of Diseases
    - Health Inequality
    - Healthcare capacity
    - HIV / AIDS
    - Hygiene
    - Infectious Diseases
    - Influenza
    - Lead Pollution
    - Malaria
    - Maternal Health
    - Measles
    - Medical Research & Innovation
    - Meningitis
    - Mental Health
    - Neglected Tropical Diseases
    - Neurological Disorders
    - Non-communicable Diseases
    - Polio
    - Respiratory Diseases
    - Smallpox
    - Smoking
    - Substance Use
    - Suicide
    - Tetanus
    - Trials
    - Tuberculosis
    - Vaccination
  - Media:
    - Books
    - News
    - Social Media
  - Political Regime:
    - Accountability & Transparency
    - Colonialism
    - Democracy
    - Franchise, Voting Rights & Electoral Systems
    - Political Engagement
    - Political Regimes
    - Press Freedom
    - Prisons
    - Quality of Institutions
    - Trust in Political Regime & Institutions
    - Voter Turnout
    - Women in Politics
  - Population Growth & Vital Statistics:
    - Age Structure
    - Child Mortality
    - Fertility Rate
    - Future Population Growth
    - Gender Ratio
    - Infant Mortality
    - Life Expectancy
    - Mortality by Age
    - Population Growth
  - Resources & Energy:
    - Access to Energy
    - Electricity
    - Emission Intensity
    - Energy
    - Energy Efficiency
    - Energy Prices
    - Energy Trade
    - Fossil Fuels
    - Metals & Minerals
    - Nuclear Energy
    - Renewable Energy
  - Technology & Infrastructure:
    - Artificial Intelligence
    - Communication Technology
    - Infrastructure Projects
    - Internet
    - Medical & Biotechnologies
    - Patents
    - Productivity of Technologies
    - Research & Development
    - Space Travel & Satellites
    - Technological Progress
    - Transport
    - Transport Safety & Accidents
    - Transportation Infrastructure
    - Transportation Technologies
  - The Public Sector & Economic System:
    - Business Environment
    - Corruption
    - Economic Freedom
    - Financing Education
    - Financing Healthcare
    - Government Revenue
    - Government Spending
    - Income Tax
    - Property Rights
    - Public Sector Employment
    - Redistribution
    - Social Spending
    - Taxation
  - Violence & Rights:
    - Animal Rights
    - Crime
    - Ethnographic and Archaeological Evidence on Violent Deaths
    - Homicides
    - Human Rights
    - LGBT+ Rights
    - Rights of Marginalized Ethnic Groups
    - Violence against & Rights for Children
    - Women’s Rights
  - War & Peace:
    - Biological & Chemical Weapons
    - Civil Wars
    - Genocides
    - Military Personnel
    - Military Spending
    - Nuclear Weapons
    - Peacekeeping
    - Terrorism
    - War and Peace
    - Weapons

[ikesau]

Here are the header entries that don't currently have a matching Grapher tag.

If we want to convert one of these old-style entries/WP topic pages to use Gdocs, we'll be blocked until this issue is fixed.

Biological and Chemical Weapons
Child and Infant Mortality
Clean Water
Clean Water and Sanitation
Co₂ and Greenhouse Gas Emissions
Coronavirus Pandemic (covid-19)
Education Spending
Employment In Agriculture
Farm Size and Productivity
Food Prices
Food Supply
Forests and Deforestation
Global Education
Global Health
Healthcare Spending
Hunger and Undernourishment
Light At Night
Loneliness and Social Connections
Marriages and Divorces
Maternal Mortality
Meat and Dairy Production
Military Personnel and Spending
Mpox (monkeypox)
Obesity
Opioids, Cocaine, Cannabis and Other Illicit Drugs
Outdoor Air Pollution
Pneumonia
Primary and Secondary Education
Research and Development
Space Exploration and Satellites
Suicides
Teachers and Professors
Technological Change
Time Use
Trade and Globalization
Violence Against Children and Children’s Rights
Water Use and Stress
Women’s Employment
World Population Growth

[JoeHasell]

To get a sense of the mismatch I looked through each of these now.

There are four cases I think:

• The most common case is just small differences in wording between the topic page title and the grapher tag. This should be easy to solve (though we should decide whether we impost that tags and titles must always exactly agree – for very long topic page titles, this might be annoying).

• There are some very low-hanging fruit cases where the page is currently not tagged (so no 'All charts' block shows on the page), but a very obvious matching grapher tag exists. For example Obesity. cc @edomt – see below for these cases. To my mind we should just tag these pages now with the relevant tags, but maybe I am missing something.

• There are cases where there is a proper mismatch between our grapher tags and topic pages – e.g. Water Use/Clean Water/Sanitation is especially messy. Military Personnel is another example. There are also some topic pages where no obvious grapher tag exists at all (e.g. Time Use). To solve this would require a bit of manual time from a data manager to re-tag. But there are not lots of these cases, so I think we are looking at a job that would take one or two days.

• There are cases that relate to sub-topics and meta-topics. This, to my mind, is the hardest case, because I don't think we are even clear what we even want to present to users. In trying to solve this problem, we are confronted with bigger problems concerning the structure of our website. Note, for example, that you can't currently navigate from the Energy topic page to the Energy Access (or other) subtopic. The problem is also bigger than the list below suggests, because this uncertainty is there even for pages where there is a matching grapher tag (e.g. Energy Access page and Access to Energy tag). This whole area is a bit of a mess.

Topic pages/old-style entries without matching grapher tags (taken from Ike's list above)

• Biological and Chemical Weapons (This is tagged with the 'Biological & Chemical Weapons' grapher tag – i.e. '&' not 'and'.))
• Child and Infant Mortality (This is tagged with the 'Child Mortality' grapher tag – i.e. just needs rewording)
• Clean Water (A subtopic. The page is not tagged and so has no All charts block. The Water Use tag looks like it is being used both to tag Clean Water-related charts and Water Use and Stress-related charts.)
• Clean Water and Sanitation (A topic-with-subtopics. The page is not tagged and so has no All charts block. There is a Sanitation tag. The Water Use tag looks like it is being used both to tag Clean Water-related charts and Water Use and Stress-related charts.)
• Co₂ and Greenhouse Gas Emissions (A topic-with-subtopics. The page is not tagged and so has no All charts block. There is a CO2 & Greenhouse Gases tag that appears to be being used for this purpose.)
• Coronavirus Pandemic (covid-19) (A topic-with-subtopics. The page is not tagged and so has no All charts block. There is a Coronavirus Disease tag that appears to be being used for this purpose.)
• Education Spending (This is tagged with the 'Financing Education' grapher tag – i.e. just needs rewording)
• Employment In Agriculture (This is tagged with the 'Agricultural Employment' grapher tag – i.e. just needs rewording)
• Farm Size and Productivity (This is tagged with the'Farm Size' grapher tag – i.e. just needs rewording)
• Food Prices (This is tagged with the 'Food prices & expenditure' grapher tag – i.e. just needs rewording)
• Food Supply (This is tagged with the 'Food per person' grapher tag –i.e. just needs rewording)
• Forests and Deforestation (This is tagged with the 'Forests' grapher tag – i.e. just needs rewording)
• Global Education (This is a meta-topic – i.e. sort of hierarchical. There is a 'Global Education' tag, but this does not inherit all charts of (what in principle are) children tags. Tagging is done with a view to which subset of those charts should also be shown on the Global Education All Charts block)
• Global Health (A meta-topic. The page is not tagged and so has no All charts block. There is no obvious tag serving a function similar to 'Global Education' (see above))
• Healthcare Spending (This is tagged with the 'Financing Healthcare' grapher tag – i.e. just needs rewording)
• Hunger and Undernourishment (This is tagged with the 'Hunger & Undernourishment' grapher tag – i.e. '&' not 'and')
• Light At Night (This is tagged with the 'Light' grapher tag – i.e. just needs rewording)
• Loneliness and Social Connections (The page is not tagged and so has no All charts block. No obvious tag exists)
• Marriages and Divorces (The page is not tagged and so has no All charts block. There is a Marriages & Relationships tag that appears to be being used for this purpose.)
• Maternal Mortality (This is tagged with the 'Maternal health' grapher tag – i.e. just needs rewording. In addition, the page is tagged as 'health' – redundantly, I think?)
• Meat and Dairy Production (This is tagged with the 'Meat and Seafood Production & Consumption' grapher tag – i.e. just needs rewording.)
• Military Personnel and Spending (This is tagged with the 'Military spending' grapher tag. However, there is also a Military Personnel tag which includes some charts (including a starred chart!), with no corresponding topic page. In addition, the page is tagged as 'War & Peace' – redundantly, I think?)
• Mpox (monkeypox) (The page is not tagged and so has no All charts block. No obvious tag exists.)
• Obesity (The page is not tagged and so has no All charts block. There is a Obesity & BMI tag that appears to be being used for this purpose.)
• Opioids, Cocaine, Cannabis and Other Illicit Drugs (This is tagged with the 'Substance Use' grapher tag – i.e. just needs rewording. In addition, the page is tagged as 'health' – redundantly, I think?)
• Outdoor Air Pollution (The page is not tagged and so has no All charts block. There are, however, 'Air Pollution and 'Indoor Air Pollution tags (and corresponding old-style entries with All charts blocks)).
• Pneumonia (The page is not tagged and so has no All charts block. There is a Respiratory Diseases tag, but this includes other charts besides pneumonia-related ones)
• Primary and Secondary Education (This page is tagged with both 'Primary Education' and 'Secondary Education').
• Research and Development (This is tagged with the 'Research & Development' grapher tag – i.e. '&' not 'and')
• Space Exploration and Satellites (This is tagged with the 'Space Travel & Satelites' grapher tag – i.e. just needs rewording.)
• Suicides (This is tagged with the 'Suicide' grapher tag – i.e. no 's'.)
• Teachers and Professors (The page is not tagged and so has no All charts block. There is a Teachers & Professors that appears to be being used for this purpose.)
• Technological Change (This is tagged with the 'Technological Progress' grapher tag – i.e. just needs rewording.)
• Time Use (The page is not tagged and so has no All charts block. No obvious tag exists)
• Trade and Globalization (This is tagged with the 'International Trade' grapher tag – i.e. just needs rewording.)
• Violence Against Children and Children’s Rights (This is tagged with the 'Violence against & Right for Children' grapher tag – i.e. just needs rewording.)
• Water Use and Stress (The page is not tagged and so has no All charts block. The Water Use tag looks like it is being used both to tag Clean Water-related charts and Water Use and Stress-related charts.)
• Women’s Employment (This is tagged with the 'Female Labour Supply' grapher tag – i.e. just needs rewording.)
• World Population Growth (This is now unpublished, replaced by Population Growth (for which there is a corresponding grapher tag))

[edomt]

There are some very low-hanging fruit cases where the page is currently not tagged (so no 'All charts' block shows on the page), but a very obvious matching grapher tag exists. For example Obesity. cc @edomt – see below for these cases. To my mind we should just tag these pages now with the relevant tags, but maybe I am missing something.

I agree with you, I don't see any reason not to do this. Though we should make sure that all charts embedded in the topic page are correctly tagged (charts embedded in the page may be untagged given that the all-charts block isn't currently used).

[danyx23]

We have since done this