Making eRegs friendly to new developers

[theresaanna]

Since our workshop, I've had a lot of thoughts bouncing around in my brain wrt what eRegs needs to be in order to catch up to the demands being placed on it now and in the future. I want to continue conversations we started during the workshop around the technical direction of eRegs.

Goals:

• new back end developers, who are likely inexperienced in the exact pieces that comprise the parser, can reasonably quickly get up to speed on core parser concepts, its interface(s) and the pertinent information around the regulations ecosystem that will make clearer why the parser does things that it does.
• all new developers can understand how the different pieces of eRegs fit together - how -site and the back ends work as well as the ability to find out where all related data comes from (GPO, Federal Register, etc) and how it relates to the parsed content.
• front end developers can understand how to create a theme or otherwise extend the UI of eRegs
• ?

Why?

• hopefully, longer term, this work will help enable us to move both 18F and CFPB to shared core instances of eRegs
• on the 18F side, it should speed up development of new client instances
• more people involved increases the reach, potential and capacity of eRegs
• more brains = more ideas
• can get more meaningful open source contributions
• ?

How?

• thorough, well-designed and organized documentation
• implementing some kind of theme/skin system
• thinking about the parser interfaces as public ones, documenting and potentially restructuring code to bring to the surface clear touch points into the process
• ?

Any revisions/thoughts/additions?

[ascott1]

How?

I'm going to go into the weeds a little. My apologies in advance.

1. @theresaanna touched on this in the "Why," but restructuring the parser so that is has a (reasonably) simple core that can be built upon using a well documented plugin ecosystem.

In my mind that parsing core takes Federal Register/eCFR XML (maybe a full reg only?) input and outputs the minimal version of the regulation that could be displayed in regulations-site. This could be without many of the layers (no need for Interps, SxS, Definitions, diffs) and would greatly simplify the need to understand the inner-workings of the parser to get a working instance of things.

2. Something else that has come up and I may not be able to fully articulate it, but currently we take JSON output, put it into the reg-core database and then use that to serve up the JSON api. Is there any way to cut reg-core out of the (essential) equation?

[theresaanna]

@ascott1 Getting into the weeds is good! I have a vague idea of what the plugin ecosystem looks like; y'all have contextual knowledge that I don't (yet) that will determine what it will actually look like. That's one of the key things I'm hoping we'll come to a common understanding about here, so that we can start coming up with some actionable items.

[cmc333333]

More under "How?":

• Docs - I think moving into and greatly expanding content into readthedocs is a step here. As it stands, the README is super daunting; I can completely understand why it's ignored
• Easier environment setup - @micahsaul created and I believe CFPB updated a vagrant image. We should revisit this
• Better error messages - while reducing the complexity of the interface helps, there's a big class of errors around "the parser never expected this", which pops up whenever encountering a new regulation. Those sort of explosions should be easier to diagnose

[theresaanna]

Where is the best place to keep and work from an updated version of this information? Google Docs are opaque. Github is unwieldy. Trello has really grown on me, and I think could work for this, but I don't want to stick things in a tool that perhaps no one else wants to use. What do folks think I should do with this info?

[cmc333333]

Sorry, what would trello help with? I don't see project management tooling as am impediment to implementing any of the above. Seems more like time/effort/priority is the stumbling block?

[theresaanna]

@cmc333333 Updating the original list here messes with the history of the thread, but I'd like to keep them collected in a form we can easily update and act on when we get there.

[theresaanna]

@cmc333333 My hope is that, if we use a common organization platform (and this can be super informal), then we build a clear history of the work we've done and why. With teams I've been on that use Trello, we would bring a card from a concept level to implementation tasks, so the entire history was right there.

[willbarton]

I think I'd feel most comfortable keeping as much information as we can in GitHub. (Any documentation that goes on readthedocs is generated from files stored in GitHub, for example.) What we've done and why is vitally important, and as much as I love Trello, I don't see it capturing that kind of information in a way that's easily discoverable by new developers looking for historical context.

[theresaanna]

:+1:

I'll update the original post instead.