Add development-mode for local and CI set ups

farost commented 1 month ago

Usage and product changes

development-mode

We add a new server's config parameter called development-mode.enable. This parameter can be used to separate real "release" application runs from our testing.

In the future, this parameter can influence many different sides of the server (logging settings, networking behavior?.., anything we want), but the only current purpose of this field is to completely turn off diagnostics and error reporting:

sentry;
diagnostics.

development-mode.enable is an optional parameter, and the absence of it in the config is equivalent to development-mode.enable=false. It is expected to be set to true for all the config.yml files used by local environments, tests, and snapshots.

--//server:config=[development/release] Bazel flag

We add a new Bazel build flag that helps choose the server configuration for all the build jobs, addressing the newly added development-mode.enable option.

Example of a release configuration build:

bazel build --//server:config=release //server:assemble-mac-x86_64-zip

Example of a development configuration build:

bazel build //server:assemble-mac-x86_64-zip
bazel build --//server:config=development //server:assemble-mac-x86_64-zip

deployment-id

We remove deployment-id from the typedb config as it makes sense only for Cloud servers. Suppose it was set in anyone's config before. In that case, the deployment-id value will be forcefully set to server-id, changing its previous manually set value.

It means that all the previous versions of TypeDB Core diagnostics submissions with different server-id and deployment-id (not expected to be higher than 0 - 10 servers) will possibly want to change the deployment-id in the database to avoid misleading query results.

Implementation

development-mode

We add new Optional KeyValues for YAMLParser, acting exactly like Predefined config options, but letting the configuration miss the key completely, setting its values to defaults inside the code. This way, declaring

KeyValue.Optional<CoreConfig.DevelopmentMode> developmentMode = optional(...);

lets us set development-mode.enable to true this way:

vaticle-factory: # any other option
  enable: false # any value

development-mode:
  enable: true 
# end of config

and set development-mode.enable to false in these two ways:

vaticle-factory: # any other option
  enable: false # any value

development-mode:
  enable: false 
# end of config

vaticle-factory: # any other option
  enable: false # any value

# end of config

This flag is still available for CLI manual overrides.

For TypeDBCoreRunner uses, this flag is forcefully set to true, not respecting the provided config.yml. However, it's possible to override this flag by passing it in the constructor if the runner user really wants to do it.

--//server:config=[development/release] Bazel flag

This flag affects all the targets that depend on the configuration file, including transitive dependencies (e.g. creating an archive of the server's library). It only accepts two values. Providing a wrong value leads to an explicit and specific build error.

NOTE: This is a Bazel flag, so it needs to be specified before the -- for application flags! For example, this works:

bazel run --//server:config=release --define version=$(cat VERSION) //:deploy-linux-x86_64-targz -- release

This does not:

bazel run --define version=$(cat VERSION) //:deploy-linux-x86_64-targz -- release --//server:config=release

vaticle-bot commented 1 month ago

PR Review Checklist

Do not edit the content of this comment. The PR reviewer should simply update this comment by ticking each review item below, as they get completed.

Trivial Change

[ ] This change is trivial and does not require a code or architecture review.

Code

[x] Packages, classes, and methods have a single domain of responsibility.
[x] Packages, classes, and methods are grouped into cohesive and consistent domain model.
[x] The code is canonical and the minimum required to achieve the goal.
[x] Modules, libraries, and APIs are easy to use, robust (foolproof and not errorprone), and tested.
[x] Logic and naming has clear narrative that communicates the accurate intent and responsibility of each module (e.g. method, class, etc.).
[x] The code is algorithmically efficient and scalable for the whole application.

Architecture

[x] Any required refactoring is completed, and the architecture does not introduce technical debt incidentally.
[x] Any required build and release automations are updated and/or implemented.
[x] Any new components follows a consistent style with respect to the pre-existing codebase.
[x] The architecture intuitively reflects the application domain, and is easy to understand.
[x] The architecture has a well-defined hierarchy of encapsulated components.
[x] The architecture is extensible and scalable.

farost commented 1 month ago

I'd still think about a correctness job to check that the only difference between config_release and config_development are the development-mode options. Not sure if we want to add it here, not sure if we'll add it later if we don't right now...

vaticle / typedb