In testing vs live features

[acrymble]

I want to start a discussion about the need to have one-language only testing capabilities for new features and new ideas. This is something I've personally found challenging the last few months because we've committed to translating or making everything available in all of our working languages. I agree that's an important commitment, but it's also important (especially for things like figuring out how to raise funds) for us to be able to try lots of things, sometimes for a few days only, to tweak them regularly, and refine until we get something that works.

At present, I think our multilingual focus means that it's cumbersome to try new things because it involves at a minimum 3 people (to translate) every time a change is made.

Thinking about how we can resolve this practically, I'd like to propose an informal idea that some features are "in testing" and others are "live". Items in testing don't need to be fully rolled out across all languages until we're happy that we've got something that is effective.

This will let us try new strategies quickly, gather data and results, and refine, without having to pester lots of people to make frequent translations on text that may not stick around very long.

I remain committed to our multilingual approach, so I hope this suggestion will be see as a proposal for a practical approach as we explore new ideas, rather than turning our back on multilingualism.

Please share views.

[jenniferisasi]

@acrymble, as much as I am also committed to our multilingual approach, I agree with you on this one. Maybe we can create a preliminary list of actions that could fall under this idea?

I believe we need the input of the @programminghistorian/technical-team to see whether featuring actions in only one language affects/doubles/simplifies their work; ie. if the banner is only tested in English first, will @mdlincoln later have to work again on the code to add a banner in the Spanish and French sites?

[drjwbaker]

Knowing the issue you have in mind https://github.com/programminghistorian/jekyll/pull/1585 I'm guessing you have in mind using a single language to test socio-technical things (e.g. do banners attract donors?) rather than purely technical issues (e.g. can we implement banners and why do they look like?). Right? (I suspect English is the defacto language of our technical dev anyway). If so, I'm happy to support this if language editions are. On which, note that this also frees ES and FR teams to experiment with ideas in languages other than English.

[walshbr]

I'm not as familiar with how @mdlincoln was planning to implement this, so I can't speak to how much extra work would be needed to make independent banners for each language that can be turned on separately.

[mdlincoln]

Firstly, this conflates a few different things, so I want to address the specific case first before talking about the more general ones.

I think it's a great idea for the different language teams to write their own messaging for much of our site content, rather than needing to do literal translations from one originating language.

This assumes, however, that all teams generally agree on the need to have something like a "donations" banner (or whatever the feature is). If the project-wide team agrees some new bit of HTML is warranted, then each language team contributes their own content to it (same in spirit, if different in exact wording), that works very well within our current architecture.

On the other hand, any feature that needs to appear on some subset of pages but not on others means that we need to write some set of if/else logic around that feature. Like most things, this is technically possible. However, it also increases the burden on everyone (not just the technical team). We would all need to keep track of what bits of HTML are supposed to have these kinds of logic gates vs. which are supposed to appear everywhere. This means keeping a whole lot of temporary exceptions in mind in our mental model of the site. For example, a maintainer would need to know that the footer should be universal for all pages, and doesn't need extra if/else logic, while the banners are somehow special and DO need that logic - or at least for one month / quarter / year etc?

"Documenting" what things are appearing one place and not appearing elsewhere doesn't erase that burden - it just means we now have to keep code in sync with some wiki page that an editor is (hopefully) keeping up to date. Then we would need to track which features are now going to be rolled out live across all the site, so we can go in and remove all those logic gates. Each of those steps is going to require eyes from the technical team. Which means increasing time spent to assess, respond to, and justify every single "exceptional" feature, rather than committing that time instead to working out what a site-wide rollout of that feature would look like.

My greatest fear is that something put up "just for testing" on the English site is then never adopted or modified for the other languages of the site. Instead, that "just for testing" bit becomes permanent on one set of pages, with no plans for it ever to appear elsewhere. I've made clear since joining this team almost 3 years ago now that the only way multilingual PH works under our current technical architecture is if each language commits to using the same templates - the same functional bits of HTML. I won't donate time to support PH if it is going to gradually diverge in to three different websites - that'd mean fundamentally re-doing the setup of everything, and that's not a project I will volunteer for.

Finally, outside the technical problems, I would caution that "just for testing" not be used as a way to toss a feature on to the site without really considering how to ensure that it could eventually be globalized. (See, for example, Patreon still only having room for 2 languages of our donation text 👎 ) Part of the commitment to a growing organization is being comfortable with not moving as fast (and breaking things) as would be possible in the earlier, smaller generation of that organization.

[drjwbaker]

I won't donate time to support PH if it is going to gradually diverge in to three different websites - that'd mean fundamentally re-doing the setup of everything, and that's not a project I will volunteer for.

@mdlincoln Thanks for reiterating this. It is a very strong incentive not to single-language 'test' anything that directly changes our core technical infrastructure. It also makes me realise that when I said..

I'm happy to support this if language editions are

.. I didn't acknowledge the burden on the Tech Team, for which I apologise.

[jenniferisasi]

Thanks @mdlincoln for such a detailed explanation. I anticipated there would be technical challenges, thus requested your input https://github.com/programminghistorian/jekyll/issues/1620#issuecomment-573334708 but it is worse -if I may- than I though.

[mdlincoln]

I may have sounded overly pessimistic about it, but I do want to emphasize why and it has to do with future scaling.

For just one feature, it's a pretty light lift to add in the right liquid filters and take them out later. It adds a little burden, and time from all the editors to track what gets changed and when things are ready, but the burden could be worth it. What really concerns me is when we start trying to do this with two features or five at a time, losing track of what feature is in what testing workflow, and eventually having some things in permanent "testing" phase.

I recognize it's in my blood as a technical strategist to be inherently conservative. I really don't want to be a curmudgeon about everything! But I worry in particular about this because the "solution" - testing out things on subsets of pages - will only work with closely coordinated team communication. And the "problem" - things taking too long to get translated - is because our team coordination clearly has limits on how fast it can move and how smoothly it can work!

[mdlincoln]

I do really want to emphasize though that

I think it's a great idea for the different language teams to write their own messaging for much of our site content, rather than needing to do literal translations from one originating language.

This helps remove the dependency problem (english -> spanish -> french) without introducing really difficult asynchronous editing across the site. As long as all the teams are on board with the concept of a feature, then I for one love the idea of a greater flexibility in content.

[walshbr]

@mdlincoln's response confirms my suspicion, for what it's worth. Just in a more coherent and informed way than I was capable of mustering not being as familiar with the code base as he is.

[drjwbaker]

Thanks for your clarity and thoroughness @mdlincoln. I have a feeling we'll be citing / refering back to this thread for a long time!

[acrymble]

I appreciate that we want to keep everything syncronized. But that's the function tickets serve. Keeping track of work in progress. This is a social change, not a technical one. At the moment, the tendency is to say WAIT, we have to translate that first.

That's no one's job, so it means asking, and asking again, and waiting. And because you've gone through that process (and felt like you were nagging people), you don't bother to do it again when you realise that what you put the first time wasn't getting the message out effectively.

[drjwbaker]

@acrymble I'm not sure what your point is here, but I'm with @mdlincoln when he writes:

Part of the commitment to a growing organization is being comfortable with not moving as fast (and breaking things) as would be possible in the earlier, smaller generation of that organization.

We wait because we are a voluntary org, because we have processes, and because we have a responsibility to those processes that we've constructed and believe in (which is not to say we never question them).

If the Tech Team envisage a large burden being placed on them by unpicking our syncronized infrastructure, we should surely - without major investment in the Tech Team - accept waiting on developments (include social that impinges on technical) as the trade off for making their lives better and their contributions to the project more enjoyable?

[acrymble]

Sorry for being unclear. I was just trying to suggest that we already have processes (tickets) for keeping track of work under development, and that the suggestion that we would need some sort of ad hoc wiki was not necessary.

It seems like we do have some agreement on the idea that the three publications need to use the same architecture, but not necessarily always the same messaging within all parts of that architecture. That would mean it would be possible for us to develop text related to marketing and have marketing campaigns that diverge across languages (while encouraging each publication to develop its own voice rather than a translation of the English one). I think that's a good enough compromise if we're happy to go ahead with that.

[drjwbaker]

Thanks for clarifying @acrymble. This compromise seems sensible to me. Can we enshrine this in the Sub-Teams policy? https://github.com/programminghistorian/jekyll/wiki/Additional-Language-Sub-Teams-Policy

[mdlincoln]

I want to make sure we're all on the same page.

I believe it makes sense, for certain contexts like the sponsorship banners, for each language team to develop their text independently. This is a very good solution in my eyes.

This still means that we will expect to have that content ready for each language before a feature goes live, that way the technical team does not have to create "partial" features with lots of additional code.

Agreed?

[acrymble]

Yes. That's fine with me.

[drjwbaker]

Fab. Who can add a note on this to the sub-teams wiki?

[acrymble]

I've updated that sub-teams wiki page to reflect this conversation.

Thanks everyone.

[drjwbaker]

Thanks @acrymble. Wording captures this ticket really well.