graphfoundation / ongdb

ONgDB is an independent fork of Neo4j® Enterprise Edition version 3.4.0.rc02 licensed under AGPLv3 and/or Community Edition licensed under GPLv3
https://www.graphfoundation.org/projects/ongdb/
380 stars 57 forks source link

Please add Infrastructure as Code template to get started easily #10

Closed bionicles closed 4 years ago

bionicles commented 4 years ago

Dev Ops Feature Request

This seems like a compelling project but it's not clear how to get started. As an engineer I like to get started quickly with infrastructure as code like Terraform, Pulumi, Cloudformation, AWS CDK, or just Makefiles to deploy a cloud solution

Describe the solution you'd like Do you think it would be possible to add an infrastructure as code script to deploy an autoscaling OngDB causal cluster on AWS? That would accelerate this community a lot because you could run an idempotent / modular / declarative template to make a cluster and get an API endpoint for your app

Describe alternatives you've considered I gave up on Neo4j because of the GPL license, closed-source, pricing hidden behind NDA, no autoscaling on Aura

Additional context Could use this: https://www.terraform.io/docs/providers/aws/r/autoscaling_group.html

Here's a template which makes an autoscaling group with a load balancer: https://github.com/terraform-aws-modules/terraform-aws-autoscaling/tree/master/examples/asg_elb

Another option would be to use Pulumi or ECS to code this infrastructure in a programming language rather than a template languate like YAML (cloudformation) or HCL (terraform). Could just make an autoscaling ECS cluster with a load balancer like this:

https://www.pulumi.com/docs/guides/crosswalk/aws/ecs/

Here's a CDK example which would generate a bajillion lines of Cloudformation to deploy ONGdb on AWS:

const loadBalancedEcsService = new ecsPatterns.ApplicationLoadBalancedEc2Service(stack, 'Service', {
  cluster,
  memoryLimitMiB: 1024,
  taskImageOptions: {
    image: ecs.ContainerImage.fromRegistry('test'),
    environment: {
      TEST_ENVIRONMENT_VARIABLE1: "test environment variable 1 value",
      TEST_ENVIRONMENT_VARIABLE2: "test environment variable 2 value"
    },
  },
  desiredCount: 2,
});

https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-ecs-patterns.ApplicationLoadBalancedEc2Service.html

here are a ton of examples of AWS CDK doing docker cluster services with load balancers, logging, networking etc

https://github.com/aws-samples/aws-cdk-examples/tree/master/typescript/ecs

TLDR: For this project to blow up, it needs to be easy to deploy an autoscaling / load balanced DB cluster and get an endpoint. If you write a document that says, "click here, type this, copy this here" it's subject to interpretation and will probably get screwed up, and it's slow. If you make Infrastructure as Code to deploy an autoscaling / load balanced causal cluster, then this project becomes 100x easier to get new developers and users, because they know they can deploy in a reproducible way with 1 line of code. If something goes wrong, they can easily re-deploy. If they need a dev cluster in their CI/CD pipeline, they can do that with 1 line and destroy it at the end to save money no problem.

I'm super bullish on infrastructure as code because it lets us automate the complicated setup process of cloud resources. Do you want to manually spend 1-2 hours deploying the database, or do you want to run 1 line of code?

Happy to help make this happen, let me know

bradnussbaum commented 4 years ago

@bionicles Thank you for the request. We are working on delivering core graph functionality and providing cloud support through our docker hub image (https://hub.docker.com/r/graphfoundation/ongdb). It would be great to see additional solutions for deployments in cloud.

As I'm sure you're aware there is a lot to do! As an open community we rely on the contributions from everyone. This project is for the community and the community advances it. When we all contribute some useful piece it lifts everyone else up. I hope your request inspires others to contribute as I think you're right - when we make it easy to deploy everywhere it helps advance the reach of the project. Many thanks.