Serverless Bots

[djfurman]

note This may be more akin to an epic than an issue and likely will have multiple issues tied to it, but I wanted to float the idea by the community.

Recently I've been somewhat dark on GitHub working on day job projects, but I've been working more and more with AWS Serverless technology and I'm seeing a use case for Bots doing that.

---

Scenario: As a trading partner, I store my incoming EDI files of whatever format in S3. On placing the file in S3, I need the file to be translated via bots running inside an AWS Lambda function and place it in another S3 location. This may then trigger another Lambda function, upload the data to a trading partner, or any other action including long-term archival of the transaction for later use in testing or data analytics.

In order to realize this state, our bots framework needs some architectural overhaul. These changes essentially decouple the application from its Django UI.

---

Here are my thoughts on the needed steps.

• Extract the grammar extraction to their own python package
• Extract the engine to its own python package
• Provide a simple way to handle AWS Lambda triggered by S3 and abstract so it can be used on Google or Azure cloud platforms as well
• Provide connection to RDS for database features
• Ensure logging can help identify RAM bottlenecking
• Ensure we can output files to S3 and abstract it so it can be used on Google or Azure platforms as well
• Provide an API (flask? django rest?) to interact with bots
• Provide a CLI scaffolding tool to build maps and/or grammar
• Provide a CLI command to package Lambda function(s)
• Support for Python 3.6.x

Please let me know what your thoughts are and if this would add elsewhere!

[josh-demuth]

Did you end up doing any work on this? I'm interested in helping with this and was about to start researching when I saw your post.

[djfurman]

Hard to believe its been a year! I'm actually still in the early phases of ddoing this, but I would welcome the collaboration! I'm hoping to get som real progress on it this holiday season.

[djfurman]

I still want to interoperate very much with bots but to keep things clean especially for black box redesign check out https://github.com/nanites

[josh-demuth]

Cool, that was my plan as well. I just started using bots recently but for translations only since the majority of my setup is all serverless. Would love to get this working.

I haven't looked at the codebase too closely though so not sure what we're in for 😁.

I should have time over the next week to do a deeper dive and offer my input.

[djfurman]

Especially with S3 offering sftp access as of last month it seems time to revive ☺

[jedrus2000]

Cheers everyone ! I wrote post on Bots EDI lobby about it month ago, but none responded.

I've started work on it, nothing finished yet, but to not make same work twice please check code here.

Please treat it is only as a base for discussion about project shape, we are talking about here.

There is in tests - test_translator.py which currently translates one X12 file into XML and writes translated files with names as numbers, into current directory.

You can launch this test from main folder like: python -m unittest tests.test_translator.TestTranslator.test_translate_one_file

I am using Python 3.7, but should work on little lower version (3.6,3.5)

[djfurman]

Cool @jedrus2000 - thanks for the note! Sorry about the lobby; I'm thinking some update broke my phone's copy of gitter (no excuse, just my prelim cause analysis). Looking at the code, it looks nice. I'll raise any questions there.