move English pages to /en directory

[acrymble]

As we're getting ready for a third language, does it make sense to shift all English pages to an /en directory in the same fashion as /es or /fr? This obviously means building in redirects and updating links. But it would mean ending the assumption that English is the default or most important language.

@mdlincoln you are the one who knows how complex this would be, so I'll defer to you.

[mdlincoln]

This is a very important question that I'd been thinking about during the recent language refactoring work. I agree completely that structuring every language within a sub-directory would be an important way to underline that the journal is language independent, and doesn't structurally privilege English.

On the technical side of things, this does mean a fair amount of work, though not necessarily prohibitive. All existing English pages would need to be given redirect_from tags so that their old links remain valid; many of the snippets links would need to be revised; we'd also need to very carefully plan which pages would still be available at the root domain (index? the rss feeds? what about the blog?) The validation plugin script would also need to be revised, as would the defaults set in _config.yml.

For the next month or two my bandwidth is going to be quite limited, so if someone would like to take a crack at this, I am very happy to review their work. However, I won't be able to take it on entirely myself right now.

[drjwbaker]

This sounds like it needs a team and a clear plan. Can we acknowledge at this stage that we want to do this in order to not privilege English, but accept that we need capacity to do it which we don't have right now?

[acrymble]

Part of that process might need to include separating the English editorial "Managing Editor" from a language neutral first point of contact for queries about the whole project (such as wanting to add a new sub team). At the minute this is bundled inherently in the English language representative.

[acrymble]

I've started working on this on a new branch: https://github.com/programminghistorian/jekyll/tree/issue-826

Just to help this along. Here is a list of the pages that would have to be moved:

/lessons

• [x] applied-archival-downloading-with-wget.md
• [x] automated-downloading-with-wget.md
• [x] basic-text-processing-in-r.md
• [x] building-static-sites-with-jekyll-github-pages.md
• [x] cleaning-data-with-openrefine.md
• [x] cleaning-ocrd-text-with-regular-expressions.md
• [x] code-reuse-and-modularity.md
• [x] corpus-analysis-with-antconc.md
• [x] correspondence-analysis-in-R.md
• [x] counting-frequencies.md
• [x] creating-an-omeka-exhibit.md
• [x] creating-and-viewing-html-files-with-python.md
• [x] creating-apis-with-python-and-flask.md
• [x] creating-network-diagrams-from-historical-sources.md
• [x] data-mining-the-internet-archive.md
• [x] data_wrangling_and_management_in_R.md
• [x] dealing-with-big-data-and-network-analysis-using-neo4j.md
• [x] downloading-multiple-records-using-query-strings.md
• [x] editing-audio-with-audacity.md
• [x] exploring-and-analyzing-network-data-with-python.md

---

• [x] extracting-keywords.md
• [x] fetch-and-parse-data-with-openrefine.md
• [x] from-html-to-list-of-words-1.md
• [ ] from-html-to-list-of-words-2.md
• [ ] generating-an-ordered-data-set-from-an-OCR-text-file.md
• [ ] geocoding-qgis.md
• [ ] geoparsing-text-with-edinburgh.md
• [ ] georeferencing-qgis.md
• [ ] getting-started-with-github-desktop.md
• [ ] getting-started-with-markdown.md
• [ ] getting-started-with-mysql-using-r.md
• [ ] googlemaps-googleearth.md
• [ ] graph-databases-and-SPARQL.md
• [ ] installing-omeka.md
• [ ] installing-python-modules-pip.md
• [ ] intro-to-bash.md
• [ ] intro-to-beautiful-soup.md
• [ ] intro-to-linked-data.md
• [ ] intro-to-powershell.md
• [ ] intro-to-twitterbots.md
• [ ] introduction-and-installation.md
• [ ] introduction-to-stylometry-with-python.md
• [ ] json-and-jq.md
• [ ] keywords-in-context-using-n-grams.md
• [ ] linux-installation.md
• [ ] mac-installation.md
• [ ] manipulating-strings-in-python.md
• [ ] mapping-with-python-leaflet.md
• [ ] naive-bayesian.md
• [ ] normalizing-data.md
• [ ] ocr-tutorial.md
• [ ] output-data-as-html-file.md
• [ ] output-keywords-in-context-in-html-file.md
• [ ] preserving-your-research-data.md
• [ ] qgis-layers.md
• [ ] r-basics-with-tabular-data.md
• [ ] research-data-with-unix.md
• [ ] sentiment-analysis.md
• [ ] sonification.md
• [ ] sustainable-authorship-in-plain-text-using-pandoc-and-markdown.md
• [ ] text-mining-with-extracted-features.md
• [ ] topic-modeling-and-mallet.md
• [ ] transforming-xml-with-xsl.md
• [ ] transliterating.md
• [ ] understanding-regular-expressions.md
• [ ] up-and-running-with-omeka.md
• [ ] using-javascript-to-create-maps.md
• [ ] vector-layers-qgis.md
• [ ] viewing-html-files.md
• [ ] windows-installation.md
• [ ] working-with-text-files.md
• [ ] working-with-web-pages.md ~/images (except for blog post stuff)~ ~/assets~
• [x] about.md
• [x] author-guidelines.md
• [x] contribute.md
• [x] editor-guidelines.md
• [x] feedback.md
• [ ] index.md (would need to be adjusted so there was a language neutral page here)
• [x] lesson-retirement-policy.md
• [x] lessons.md
• [x] project-team.md
• [x] research.md
• [x] reviewer-guidelines.md

These pages also seem to have things related to language and the URL:

• [ ] https://github.com/programminghistorian/jekyll/blob/gh-pages/_data/snippets.yml
• [ ] https://github.com/programminghistorian/jekyll/blob/gh-pages/_config.yml

[drjwbaker]

I think we need a list of all the activities that need to happen to deAnglicise our infrastructure (tech and social), give them separate tickets, divide up the work.

So alongside this, I can think of:

• clarity and consistency on contact points for feedback/enquiries by language
• whether https://programminghistorian.org/ should land on a language selection page
• the ISSN work that kinds sparked this #788
• a list of pages that should be translated consistently between EN/ES/FR https://programminghistorian.org/translation-concordance
• refactoring of https://programminghistorian.org/project-team and team roles/titles to fit new team structure https://github.com/programminghistorian/jekyll/wiki/Additional-Language-Sub-Teams-Policy
• a sub-mailing list for the EN team (presuming that the ES team have one of their own)

Anyone, I think this needs to go on a fresh ticket (I'm just keen not to do open yet another ticket if we can help it..)

[mdlincoln]

a list of pages that should be translated consistently between EN/ES/FR

@acrymble already did this: https://github.com/programminghistorian/jekyll/wiki/Additional-Language-Sub-Teams-Policy (also as https://programminghistorian.org/translation-concordance notes, that is a page of actually-translated pages, not should-be-translated pages.

[drjwbaker]

@mdlincoln My bad. Forgive me :)

[acrymble]

is there someone on the English editorial team that would be willing to work on some of these matters?

[mdlincoln]

One correction, Adam - given the way our architecture works, both images and assets must stay exactly where they are. Many (like lesson avatars or our team member profile pics) are referenced across lessons.

[mdlincoln]

@acrymble I notice you are repeating a lot of commits to add the issue urls back on to lessons. This work is done and now merged onto the gh-pages branch, so you shouldn't re-do it, otherwise there will be a huge number of merge conflicts.

[acrymble]

@mdlincoln ya I had made the mistake of updating files that were being updated elsewhere, so I had changed internal links to /en/lessons/..

It seemed faster to add in the review-ticket metadata for the entries I'd already done than to go through looking for links to change, which takes longer.

[mdlincoln]

OK, I think we erred in not setting down a general strategy for this move. It's important to fully sketch out necessary changes, and to rule out unnecessary changes. I'm sorry for not doing that.

1. I'd restart this work by creating a new branch off of the current gh-pages, which contains all the changes to lesson metadata from the review ticket PR.
2. I strongly suggest moving entire directories by doing this work on your local computer using GitHub Desktop, rather than trying to use the GitHub web interface. This way, you won't need to move each lesson individually. This will vastly simplify the commit history and make it easier for me to review.
3. We should not be manually updating existing internal links to point to new URLs. Rather, we need to be adding redirect_from to every page being moved. That's the only metadata that ought to be changing right now. That means we won't need to update e.g. any hardcoded links when one lesson refers to another - the redirects will handle that work for us.

[acrymble]

Ok. Does the redirect_from just go in the yml of the file that's moved? I'm not totally clear how the server finds it. Is there a master redirects file somewhere that acts like a map?

[mdlincoln]

No central action is needed. You just add a field in the YAML metadata like so:

redirect_from: /lessons/getting-started-with-mysql-using-r

It should contain the path of the old location of the lesson, no https://programminghistorian.org required.

[acrymble]

Ok, I've done this ,but it's on my desktop computer. So I just have to figure out how to do a pull request.

[acrymble]

Tihs was merged with #846. Other than the /index.md, everything should now be moved. Hopefully nothing broken.

[drjwbaker]

I clicked around for a few minutes and nothing broken that I've noticed. Only thing I did notice was slight hang/redirect when going into EN menu items

[acrymble]

Thanks James. I noted that too. I think we should change the menu items to directly point to the new pages. There's really no need to have redirecting.

[drjwbaker]

Want me to do anything?

[acrymble]

Would be awesome if you could do a pull request just updating the menu links (the redirects on the actual pages should still work I think).

[drjwbaker]

Nope. No idea how to do this. Presume we are looking to change https://github.com/programminghistorian/jekyll/blob/gh-pages/_includes/menu.html ?

[mdlincoln]

@drjwbaker look in the _data/snippets.yml file, you'll see all the menu labels and links there

[drjwbaker]

Thanks @mdlincoln. PR ready.