Add docs for building and running the oss build of Mononoke

[Matthew-Benson]

I want to run Mononoke to try it out, but I can't seem to figure out how to run it correctly.

By following the CI actions in .github/workflows/mononoke_linux.yml, I was able to build it on Ubuntu 20.04, but the binary output I got from following

python3 build/fbcode_builder/getdeps.py --allow-system-packages build \
  --src-dir=. mononoke  --project-install-prefix mononoke:/usr/local

python3 build/fbcode_builder/getdeps.py --allow-system-packages fixup-dyn-deps \
  --strip --src-dir=. mononoke _artifacts/linux  --project-install-prefix mononoke:/usr/local \
  --final-install-prefix /usr/local

Seems to be

/tmp/fbcode_builder_getdeps-ZhomeZrunnerZsaplingZbuildZfbcode_builder/installed/mononoke/bin/mononoke

and if I run the following (pieced together from mononoke --help)

/tmp/fbcode_builder_getdeps-ZhomeZrunnerZsaplingZbuildZfbcode_builder/installed/mononoke/bin/mononoke \
  --listening-host-port 0.0.0.0:80 --cert cert.pem --private-key key.pem --ca-pem ca.pem --prod

I get an error message that's isn't clear to me how to proceed:

PANIC: not implemented: This is implemented only for fbcode_build!
from /tmp/fbcode_builder_getdeps-ZhomeZrunnerZsaplingZbuildZfbcode_builder/installed/rust-shed/source/shed/cached_config/src/oss.rs:59:9
    @ 0000000000000000 panichandler::set_panichandler::{{closure}}
    @ 00000000036fe097 <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call
                       /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/alloc/src/boxed.rs:2021
                       -> /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:783
    @ 00000000036fddb8 std::panicking::begin_panic_handler::{{closure}}
                       /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:649
    @ 00000000036fc975 std::sys_common::backtrace::__rust_end_short_backtrace
                       /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/sys_common/backtrace.rs:170
    @ 00000000036fdb51 rust_begin_unwind
                       /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:645
    @ 00000000004675a4 core::panicking::panic_fmt
                       /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/panicking.rs:72
    @ 0000000000000000 cached_config::oss::<impl cached_config::store::ConfigStore>::regex_signed_configerator
    @ 0000000000000000 mononoke_app::builder::create_config_store
    @ 0000000000000000 mononoke_app::builder::MononokeAppBuilder::build_with_subcommands
    @ 0000000000000000 mononoke::main
    @ 0000000000000000 std::sys_common::backtrace::__rust_begin_short_backtrace
    @ 0000000000000000 std::rt::lang_start::{{closure}}
    @ 00000000036f2016 core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
                       /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ops/function.rs:284
                       -> /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:552
                       -> /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516
                       -> /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142
                       -> /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148
                       -> /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:552
                       -> /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516
                       -> /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142
                       -> /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148
    @ 0000000000000000 std::rt::lang_start
    @ 0000000000000000 __libc_start_main
    @ 0000000000000000 _start
Aborted

The error seems to be related to caching, so I tried running with the flag --cache-mode disabled, but that didn't seem to have any impact.

From build command steps, I had guessed that I would get a working install somewhere under /usr/local but that doesn't seem to be the case.

When I get a working build, I'd be happy to contribute docs for building/running and was planning on creating a repo with a simple container image that runs Mononoke as an example, but I may need a little bit of assistance configuring the oss build to run correctly.

Any assistance is greatly appreciated!

[markbt]

Hi @Matthew-Benson, thanks for your interest in Sapling.

Unfortunately we don't currently have a supported workflow for running Mononoke externally. As a scalable server, Mononoke still has some strong dependencies on internal Meta infrastructure in order to provide that scalability. Many of these can be stubbed out for testing purposes, but the stubs are not necessarily fully featured. For now, the code is mostly available for information only.

That said, in this case it looks like the particular component you are missing is our distributed configuration store (called Configerator). You can override the use of this with the --local-configerator-path command line argument, however you will need to point this at a directory containing valid configuration. You may have some luck looking at the setup for some of our basic integration tests which should contain enough to run a basic Mononoke instance.

[Matthew-Benson]

Thanks for the help @markbt! I think I'm very close to getting it to start, but I'm getting an error about config parsing. I suspect I may have some toolchain difference from the CI environment? In short, I followed the build steps in .github/workflows/mononoke_linux.yml and then followed something akin to the integration tests you linked.

export TESTTMP="/tmp/test"
export HGTEST_CERTDIR="/tmp/cert"
export TEST_FIXTURES="$HOME/sapling/eden/mononoke/tests/integration"
export URLENCODE="$(which urlencode)"
export HGRCPATH="$HOME/.hgrc"
export DUMMYSSH="/usr/local/bin/dummyssh"
export MONONOKE_SERVER="/tmp/fbcode_builder_getdeps-ZhomeZrunnerZsaplingZbuildZfbcode_builder/installed/mononoke/bin/mononoke"

mkdir -p $TESTTMP
mkdir -p $HGTEST_CERTDIR

source "${TEST_FIXTURES}/library.sh"

setup_common_config
setup_mononoke_config
cd $TESTTMP
start_and_wait_for_mononoke_server

But I get an error from the config files generated for the default repo "repo":

V0128 03:33:58.065978 42285 [main] tunables/src/lib.rs:165] Initializing tunables: {"killswitches":{},"ints":{},"strings":{},"vec_of_strings":{},"killswitches_by_repo":null,"ints_by_repo":null,"strings_by_rep
o":null,"vec_of_strings_by_repo":null}
V0128 03:33:58.066056 42285 [main] /tmp/fbcode_builder_getdeps-ZhomeZrunnerZsaplingZbuildZfbcode_builder/installed/rust-shed/source/shed/justknobs_stub/src/cached_config.rs:125] Initializing JustKnobs: {"ints
":{"scm/mononoke_timeouts:repo_client_getbundle_timeout_secs":1800,"scm/mononoke_timeouts:repo_client_bookmarks_timeout_secs":180,"scm/mononoke_timeouts:remote_derivation_client_retry_delay_ms":100,"scm/monon
oke:derived_data_service_batch_size":3,"scm/mononoke:repo_client_gettreepack_buffer_size":1000,"scm/mononoke_timeouts:derived_data_slow_derivation_threshold_secs":15,"scm/mononoke_timeouts:repo_client_default
_timeout_secs":900,"scm/mononoke:repo_client_max_nodes_in_known_method":100000,"scm/mononoke:repo_client_concurrent_blob_uploads":1000,"scm/mononoke:scs_other_methods_sampling_rate":1,"scm/mononoke:commit_gra
ph_prefetch_step_limit":4,"scm/mononoke:scs_popular_methods_sampling_rate":1000,"scm/mononoke:warm_bookmark_cache_poll_interval_ms":1000,"scm/mononoke_timeouts:derived_data_service_request_timeout_ms":1000,"s
cm/mononoke_timeouts:remote_derivation_client_timeout_ms":15000,"scm/mononoke:derivation_worker_sleep_duration_ms":250,"scm/mononoke_timeouts:repo_client_clone_timeout_secs":14400,"scm/mononoke_timeouts:repo_
client_getpack_timeout_secs":18000},"bools":{"scm/mononoke:lfs_k_consistent_hash_routing":false,"scm/mononoke:derived_data_enable_remote_derivation_local_fallback":false,"scm/mononoke:derived_data_disable_rem
ote_derivation":false,"scm/mononoke:scs_commit_find_files_disable_bssm":false,"scm/mononoke:scs_commit_find_files_use_bssm_for_suffix_queries":false,"scm/mononoke:derived_data_disable_derivation_workers":fals
e,"scm/mononoke:sql_disable_auto_cache":false,"scm/mononoke:sql_disable_auto_retries":false,"scm/mononoke:disable_running_hooks_in_pushredirected_repo":false,"scm/mononoke:static-backend":false,"scm/mononoke:
bssm_enable_optimized_derivation":false,"scm/mononoke:mononoke_force_local_pushrebase":false,"scm/mononoke:disable_hooks_on_plain_push":false}}
Error: invalid config options: no named_repo_config "repo" for repo "Some("repo")".

Immediately assumed there was a problem with the config, but I can't tell from inspecting it that it's incorrect:

cat mononoke-config/repo_definitions/encode%20repo/server.toml

repo_name="repo"
repo_config="repo"
enabled=true
hipster_acl="default"

cat mononoke-config/repos/encode%20repo/server.toml

hash_validation_percentage=100
storage_config = "blobstore"

[pushrebase]
forbid_p2_root_rebases=false
rewritedates=false

[hook_manager_params]
disable_acl_checker=true
[push]
pure_push_allowed = true
[infinitepush]
  [derived_data_config]
  enabled_config_name = "default"
  scuba_table = "file:///tmp/test/derived_data_scuba.json"
[derived_data_config.available_configs.default]
types=["blame", "changeset_info", "deleted_manifest", "fastlog", "filenodes", "fsnodes", "unodes", "hgchangesets", "skeleton_manifests", "bssm", "bssm_v3", "testmanifest", "testshardedmanifest"]

So I suspect I may have just built with a different Rust toolchain that's making some parsing difference? I'll have to dig into the GitHub runner for their Ubuntu 20.04 image and see what they have - I used cargo 1.75.0 (1d8b05cdd 2023-11-20). EDIT: I double checked toolchain versions from the GitHub actions runner here and rebuilt on Ubuntu 20.04 and still ran into the same issue.