Define Development Architecture

[jag3773]

Story

As a developer I need to know the layout of the tX code architecture for Door43 so that I can contribute.

Notes

I see a few options, but maybe there are others too?

1. Keep all our conversion code in https://github.com/unfoldingWord-dev/tX . This means a monolithic code base that may quickly become difficult to maintain and test.
2. Split each converter out into its own Github repo and distribute them as Python Packages (pip).
   • This would mean that the main conversion Lambda function would include the respective packages in a requirements.txt file and conversion script would import them as 3rd party modules.
   • Travis CI can be setup to automatically deploy to the Python package index (see https://docs.travis-ci.com/user/deployment/pypi).
3. Split each converter out into its own Github repo and deploy them to individual API endpoints.
   • We will want to do this eventually anyway, so that any dev can get endpoint/convert/obs2html (or something like that) and get their desired output.
   • Our main conversion Lambda function would then call these individual API endpoints to actually do the conversion work from a Gogs webhook notification.
   • Here again, Travis CI can be setup to automatically deploy to an AWS Lambda function (see https://docs.travis-ci.com/user/deployment/lambda). Alternately, we could still use apex to do these deployments.

Although option 2 is enticing, it does box us into Python unnecessarily (Lambda supports other types of code too). Perhaps going with option 3 would allow the most amount of flexibility going forward.

[richmahn]

Probably should be even more specific, such as obs_md2html, obs_json2html (just an example...for tW we have it as a json), and then obs_html2pdf.

Then we should also have a package that handles the full conversion, e.g. obs_conv.py, which takes and input and output file and knows what to do based on the file extensions, kind of like pandoc does.

Will each converter/package expect file(s) for input and output a file, or should we do what USFM-Tools does where you can import the transform.py file and call the transform.singeHtmlRenderer() method with the content already loaded in memory and get a string back? Then we don't have to keep making files, such as .md -> obs.html -> obs.pdf but just .md -> obs.pdf?

[jag3773]

I'm beginning to record the spec in https://github.com/unfoldingWord-dev/door43.org/wiki/tX-Development-Architecture.

[jag3773]

After the proposal is accepted, there are code samples at https://developers.zamzar.com/docs which could pretty easily be modified to provide examples.