Regression: no rich text links work inside Streamfields on service page steps or guide pages.

[easherma]

Here's a nice regression that i discovered while working on #3682 https://github.com/cityofaustin/techstack/issues/3053 ( https://github.com/cityofaustin/joplin/pull/420 ) Did its thing, QA'ed, merged https://github.com/cityofaustin/techstack/issues/3504 (https://github.com/cityofaustin/joplin/pull/499) Did its thing, code change merged in actually undoes the fix of the previous commit

So you might notice that internal links inside rich text fields in streamfields don't work anymore, but I don't think we noticed at the time that the new code merged in was undoing a previous fix and the new code wasn't incorporating the old code :tada:

Hoping to sort this out as part of the changes for #3682, but we should probably also have a regression test to cover cases like this (in progress here: https://github.com/cityofaustin/joplin/pull/545)

Two main goals:

• fix the bug
• have a test plan for making sure the bug doesn't happen again in the future

[easherma]

Testing resources: https://github.com/syrusakbary/snapshottest/ https://docs.graphene-python.org/en/latest/testing/ https://docs.graphene-python.org/projects/django/en/latest/testing/

[desigonz]

@desigonz to write up issue for the regression + test case with acceptance criteria

[easherma]

Braindump for places to test:

Unit tests: For model/page:

• Does the rich text widget have the expected list of features? ( not super worth testing)
• Do the links inside the author interface go to the right place on janis ( they do not, we could write a failing test, making this actually work would be its own work )

For the API (kinda systems test?):

• Test cases for our Streamfields interface ( would love input on fleshing this out in useful ways)
• Test if an expected query matches output
  • For all instances that have links to expand, are they expanded? (can check the structure)
  • If expanded to a link, does the link work (this might be better off as a janis test)
  • Can we have a test that checks if we are checking in new code that changes the graphql output? (i.e. how could we have spotted the regression or at least noted that we were making a change)

[easherma]

changed the estimate as the original estimate was just to document the regression test, but we really need to be able to fix the problem to unblock #3682 and future features

[easherma]

Ideal test case: Every rich text field has an internal or doc link ServicePage with steps steps include a steps_with_location block There is also at least once component included from 'dynamicContent' There are also several steps with options, which have nested options (which are rich text and should have a test internal link field or document link field)

It passes if: All links are expanded html All data shows up in graphql in the structure expected Janis still has everything it needs to render related information, etc

[easherma]

We've got three main options I can see:

1) https://github.com/cityofaustin/joplin/tree/3567-api-rep-error <--- strips out the custom service page stuff, fixes some errors, has related location info in pretty much the same structure

2) https://github.com/cityofaustin/joplin/pull/541 Leaves more code in place, sorta works. Dosen't work (i.e. links are not expanded in rich text blocks) for nested streamfields (such as steps with options), This was the first option I tried to go with to fix the bug, but it wasn't as much a path of least resistance as I'd hoped.

As is, it's going to require a good chunk more boilerplate code to be added to make all the Streamfield options work for service pages ( we'll basically need to write resolvers for all kinds of Streamfields + make sure they work when arbitrarily nested and still return expanded links, etc).

3) Third option would be to really give wagtail-grapple a solid try, as most of the above boilerplate would be already in place, we'd have a more solid foundation to build off of for the API vs refactoring our stuff completely from 'scratch'. I think this is the best long term option. Ideally we'd be able to introduce grapple in place alongside our current API. I haven't gotten this to work yet, but haven't sunk a ton of time into it yet.

I want to do more testing on the first option, especially with Janis, and see what @briaguya thinks. But at the moment if I had a pick a best course of action it would be #1 to restore links working for service page streamfields, followed by #3 once we have some breathing room

[easherma]

Ready-ish to test @: https://janis-3567-richtext-links.netlify.com

https://joplin-pr-3567-api-rep-error.herokuapp.com/

Service page with a bunch of kinds of steps and links everywhere: https://janis-3567-richtext-links.netlify.com/en/health-safety/health-prevention/disease-prevention/get-immunized/

https://joplin-pr-3567-api-rep-error.herokuapp.com/admin/pages/9/edit/#tab-content

The testing I've done is still manual, haven't added unit or systems tests (still thinking about what the best way to do that might be)

I've mostly focused on the functionality of this bug, but since we changed some of the ways Janis is querying data, it's possible we might also want to check other contextual nav stuff? ( I'd defer to @briaguya on other potential places to check for other regressions)

Test criteria: Any kind of link(internal, external, document link, etc) added to a rich text widget should work as the correct janis link, even if its in a streamfield on a service page. ( more on that here: https://github.com/cityofaustin/techstack/issues/3852#issuecomment-582638760 )

[desigonz]

Test cases

Service page

• [x] In Joplin, user adds internal link within basic step and publishes page; on Janis, the link opens to the right page
• [x] In Joplin, user adds internal link within step with options and publishes page; on Janis, the link opens to the right page
• [x] In Joplin, user adds internal link within step with location and publishes page; on Janis, the link opens to the right page
• [x] In Joplin, user adds internal link within additional content rich text block and publishes page; on Janis, the link opens to the right page

  Information page

• [x] In Joplin, user adds internal link within rich text box and publishes page; on Janis, the link opens to the right page

  Guide page

• [x] In Joplin, user publishes a guide with the service page and information page tested above (with internal links); on Janis, the links open to the right page

[desigonz]

I added links to and published the follow pages on Joplin:

• Service page
• Info page
• Guide But when I click "view live" in Joplin, the pages 404. I logged into Netlify and the last build was at 9:50 this morning:

Can you look into how the Janis and Joplin branches are hooked up?

[desigonz]

I was able to trigger the builds via Netlify and test out the cases above.

QA passed for the test cases! Looks and works great. We can spin out writing unit tests into a different issue, but use those test cases as the basis for those unit tests.

[desigonz]

@easherma Content has been sharing a lot of internal links that aren't working on published pages, such as this one on the mobile food vendor guide. Do you have an estimate on when this will be merged? Weighing whether to edit the pages with external links or waiting on this fix to be in production.