[CWM] Whiteboarding session for creating the MM backend

[lin-d-hop]

Track under CWM in Clockify

Description

Let's create the tickets for the first steps in creating the MM backend.

This whiteboarding session is to create tickets that will get us to the point that we have:

• Created server in node with existing MM code
• Strip out all of the FE code
• Created an initial mock fake data API response from BE. This is more about testing infrastructure than finalising the response format, to enable this not to be blocked by API design.

Acceptance Criteria

• Whiteboard meeting
• Issues created

[rogup]

In a comment from #265:

@wu-lee: I think the main question left now is what goes on the back-end. (To think about this briefly, I would suggest something which works smoothly with Swagger/OpenAPI and is small and simple and aimed at rapid prototyping. I see that NocoDB says it supports Swagger, and we can run this on our Cloudron server, which simplifies deployment massively)

Are you suggesting that we don't have a backend server at all and use a 'serverless' architecture that just uses a database and a script for each API. Or was it a comment on the DB solution to use with a backend server?

• If the former, I think that could be a cool thing to aim for but I don't have much experience in serverless architecture, and it feels like quite a complicated thing to set up. Would we need to get our data factory to output data in a form that can be used with serverless scripts?
• If the latter, do we need a DB at all? I was imagining the simplest version of a backend would just be a Node server that reads the CSVs on startup and holds the initiatives in memory (like what the front-end currently does for existing maps). Since only read operations are needed, maybe this would also perform better than reading from a database?

Anyway, this is stuff to discuss in our whiteboard 😊

[rogup]

Hapi/Swagger https://github.com/krakenjs/hapi-openapi https://stackoverflow.com/questions/35990191/how-should-i-use-swagger-with-hapi

[rogup]

@wu-lee Discussion from whiteboard

• Nick [edit: somewhat] in favour of making mocks and prototype in NocodDB before making backend [edit: because probably a a very small amount of config and little or no deployment in order to get a service]. Rohit in favour of not doing extra work since we have confidence from previous demos that it will be ok.

Path forwards that we've decided

• As a prototype, just have mock files on a server for each response. eg. A big data file plus others with lists of IDs, one list for search result. FE "search" then amounts to a choice of which of these to consume.
• Get going with making the BE.
  1. Use Swagger to define (and create?) API. This may be the only part required between back and front end.
  2. Add a "hello world" NodeJS server app and dry-run the deployment process on dev-2
  3. Generate Hapi routes in this app (since we know Hapi works reliably in LX, or choose other common framework if not possible with Hapi. Check for Swagger code generation support.)

Target:

• set up the mock files and Swagger APIs by the end of Monday
• create simple BE server that serves mock responses by Thurs

[rogup]

@wu-lee has done some more investigation into frameworks e.g. Hapi or something else.

SwaggerHub can generate a stub app in NodeJS, but this doesn't work with Hapi. Hapi doesn't seem to have a tool for generating server-side code. There is only a tool for auto-generating openapi specs from Hapi code, which would be useful if the code becomes the "source of truth" later.

Other frameworks which seems to have better support for openapi are:

• NestJS (although this seems to be developer marmite)
• Fastify https://www.npmjs.com/package/fastify-openapi-glue
• Express https://openapi-generator.tech/docs/generators/nodejs-express-server

Fastify seems to do very well in benchmarks compared to other frameworks, it looks like it is fairly simple to use, and development looks like it will be fast if we use fastify-openapi-glue.

So let's go with this. I'll make a ticket for this work.