This repository contains source code for the Environment and Health Data Portal. You can view a staged development version here and the live production version here.
This repository contains the website's source code. To get the raw data that the site displays, visit our data repo.
Ths site is an ongoing work in progress. We are always interested in people willing to do user testing and co-design work, so if you're interested in helping develop the site, you can:
This Readme file documents the basics of site structure and functioning.
You will need the following things properly installed on your computer.
To browse a local version of the site, in your local development environment, start the server by typing hugo serve --environment development --disableFastRender into the terminal.
hugo serve starts the server - you can then browse the site at http://localhost:1313/EH-dataportal--environment development specifies that it will serve the site for the development environment, using content from /config/development/config.toml--disableFastRender turns off fast render mode, so more small changes are rapidly served.You can browse the site at http://localhost:1313/EH-dataportal.
To build the source code, simply enter the command hugo. This assembles the site’s files, building to /docs (this build-to destination can be specified in config.toml).
To begin work:
After committing, working branches can be merged into development for testing then merged into production for deployment.
A run-down of main branches, actions, and purposes are:
| Deploy to | EH-dataportal branch: |
EHDP-data branch: |
Action on merge: | Used for: |
|---|---|---|---|---|
| Production servers | production |
production |
Builds to builds/prod-prod |
Live site |
| GitHub Pages | development |
production |
Builds to builds/dev-prod |
General development |
| 307 (internal) | development |
staging |
Builds to builds/dev-stage |
Demoing data & content |
On merge, these branches are automatically [built](https://github.com/peaceiris/actions-hugo and served to other branches using Github Actions (triggerd by a merged pull request). (Note that this requires a workflow YAML file in both main and development.)
When changed are merged into development or production, in addition to automated builds, these actions are triggered:
development, a Github Action builds and commits the site files to builds/dev-prod. On merge to production, an Action bulids and commits the site files to builds/prod-prod. We deploy this branch to our server to serve up the production site. Gruntfile.js runs to create /static/js/lunr/PagesIndex.json, which powers the search function (search-results.js uses Lunr functions to search PagesIndex.json and display results on the search-results.html template).The /config folder includes subfolders with environment-specific configuration. Specifically, there are different configuration files for development, staging, and production envirnoments. You serve or build the site by specifying the environment (eg, hugo serve --environment development or hugo --environment production). This merges the contents of that environment's config file (in /config/ENVIRONMENT/config.toml' with /config/_default/config.toml. You may find it useful to create aliases for these functions (in Powershell, or Bash).
Some key uses of environment-specific variables in the config are:
data_repo, which tells the site to read data from staging or production branches of EHDP-data.To deploy to a new environment, update the baseURL in config.toml. Update the path, if necessary, in the environment-specific config.toml file. And, you may need to update paths in other files, like search-results.js. Crtl-F is your friend.
Most of the data used by the site is stored in the separate EHDP-data repository. This setup allows us to update the site's data without needing to re-build the entire site. Look there for descriptions of the data files, and for the code used to generate the them.
Note that any file required to build the site should remain with the source code, but anything required only for display can be stored in the remote data repo, EHDP-data.
Generally, Hugo works by combining content (in markdown, located in /content) with templates (located in the /themes) - you'll notice that these two directories have identical structures, because Hugo combines content in /content/data-stories, for example, with templates in /themes/layouts/data-stories.
_index.md will get section.html layout index.md will, by default, receive the single.html layoutname.md, will receive single.html layout layout: custom in the frontmatter will get a layout called custom.html (all in the corresponding layouts folder).Templates can include Hugo code (which you can identify by {{ curly brackets }}. When Hugo serves or builds the site, it runs code, inserts content into the HTML, and produces static HTML pages. Any template is actually an assembly of other templates.
hugo new data-stories/TITLE/index.md. title, date, and draft seo_title and seo_descriptioncategories: this determines what Key Topics this data story is associated withkeywords to support search functionsimage to associate with the image filenamemenu.main.identifier to highlight the correct button in the nav menudraft: false. The data story will be a part of the site when you serve or build it, and it will appear on the related pages if it's been tagged properly via categories.Key Topics associate different content types by theme, and they also host their own child pages (such as the Air Quality Explorer, or the interactive Heat Vulnerability Index). Each Key Topic page is an _index.md file in a titled folder.
To create a new Key Topic:
hugo new key-topics/TITLE/_index.mdkeyTopic (for example, keyTopic: airquality). This associates this Key Topic with any other content that has airquality as one of its categories.layout: single to give it the correct templateTo create a child page, create a subfolder within the keytopic - for example, see the folder structure under /content/key-topics/airquality.
The data explorer includes markdown files for each topic (previously called subtopics). The associated indicators are specified in an array (with headers) in the frontmatter. Extensive Javascript powers the rest of the functions, with the javascript for each display (summary, map, trend, and links) in discrete files.
To publish a new neighborhood report, you'd need:
EHDP-data/neighborhood-reports/reports/data/globalsEHDP-data/neighborhood-reports/imagesEHDP-data/neighborhood-reports/dataTemplates are stored in themes/dohmh/layouts, in the folder for their corresponding content area. A template includes:
layouts/_default/baseof.html, and components referenced in that file (like header.html, footer.html, etc)layouts/data-stories/single.html)/partials. These can be called from any other templates.Shortcodes can be called from content files (markdown). Essentially, the shortcode is called and arguments are passed into it and inserted into the corresponding HTML code in layouts/shortcodes. There are shortcodes for a few different visualization embeds for Data Stories, and more can be written as needed.
Data accessible throughout the site can be stored in the data folder. This can be referenced by site templates. For example, featured_data.yml is referenced by partials/featured-data.html and displayed on the Home Page and the Data Explorer landing page. You can update "featured datasets" by updating this file.
Other content in data are SEO variables and Neighborhood Reports core content.
We use Hugo's integrity function; this adds hashes to JS filenames and tells the pages to fetch the files with the hashed names. This is a way of improving security by ensuring the integrity of the JS files. This might not work on production if the server's DigiCert is expired.
Dependencies (The JS libraries the site uses: D3, Arquero, Vega-Lite, Accessible Autocomplete, etc) are served by the site rather than linked from CDNs. When you run npm install to configure your local repo, they are stored in /node_modules. When you run a build (hugo or via merging to development or production, per Github Actions), Hugo grabs these dependencies, applies the Integrity hash (see above), and references these 'local' versions.
We use Hugo to automatically resize images. Where you put the source path of an image, there's additional code - Hugo resizes the image, generates a different size (puts it in the /resources/_gen/images), and automatically points to the resized image.
We use a variety of environment-specific code to produce:
data-index.html, on site build, assembles a json file of topics and indicators. It ranges over DE topic frontmatter and produces a cross-reference of topics and indicators (file. This is used on data-index.html as well as on the Neighborhood Reports: when an indicator is clicked, it runs getURL() to find the parent topic for the indicator, generates a URL, and produces the Get The Dataset button.
The repo includes some files to integrate with Cloudcannon, an online CMS provider. Specifically:
cloudcannon.config.yaml sets up how the site appears in the CC CMS, what the editor reveals, what shortcodes are easily accessible, etc. .cloudcannon/prebuild is code that runs when Cloudcannon builds/serves the site..cloudcannon/schemas include frontmatter templates for when CC works with frontmatter.Resources used in a build (like a Neighborhood Report json spec, for example) may be cached by whatever machine is running the build. Updates to resources might not be reflected in a build if Hugo is using cached versions. In config.toml, setting the cache to have a maxAge = 0 effectively turns it off, ensuring that Hugo will use the original, non-cached resources. Caching in Hugo is explained more here.
You can comment on issues and we'll follow up as soon as we can. In the spirit of free software, everyone is encouraged to help improve this project. Here are some ways you can contribute.
With regard to GitHub platform communications, staff from the New York City Department of Health & Mental Hygiene are authorized to answer specific questions of a technical nature with regard to this repository. Staff may not disclose private or sensitive data.