search

livebook-dev / livebook

Automate code & data workflows with interactive Elixir notebooks
https://livebook.dev
Apache License 2.0
4.87k stars 416 forks source link

Introduce Fly.io runtime #2708

Closed jonatanklosko closed 2 months ago

jonatanklosko commented 2 months ago

This adds a new runtime "Fly.io machine". In the runtime configuration UI, we ask the user for API token, which can be generated in the Fly dashboard. We create a new app (if needed) and the user can configure resources and storage. When connecting to the runtime, we create a new machine, which is automatically destroyed once the runtime is disconnected.

Changes

There are a few elements to this PR:

  1. Now that connecting is a longer operation, I made Runtime.connect async. Following this, there were a lot of changes around Livebook.Session.Data to account for the new "connecting" state during operations, and a fair amount of tests that needed adjusting. On a sidenote, this shifted some logic from Session and even the LV directly to Data, which is more consistent with other operations and easier to test.

  2. For the Fly runtime, we need a custom EPMD module, so I removed EPMD-less mode and now we always use the custom EPMD module. In Docker, Desktop and Escript we set the VM args automatically, however when running from source or in dev it is necessary to set ELIXIR_ERL_OPTIONS. I plan a patch to OTP, such that we can set it after boot in the future.

  3. The Fly runtime. The runtime is implemented in Livebook.Runtime.Fly and there much more details in the module docs. The UI flow is handled in FlyRuntimeComponent and FlyAPI.

Demo

https://github.com/user-attachments/assets/8541c7bd-9586-4029-b282-104f4a53ed24

github-actions[bot] commented 2 months ago

Uffizzi Preview deployment-54146 was deleted.

jonatanklosko commented 2 months ago

Note that this relies on Docker images, so if you want to try this PR locally, change this line:

https://github.com/livebook-dev/livebook/blob/2fe9be23b6cf8aac2358984cb5349cc106714381/lib/livebook/runtime/fly.ex#L163

to

image = "jonatanklosko/livebook-main:main"