microsoft / rats

Rats is a collection of tools to help researchers define and run experiments. It is designed to be a modular and extensible framework currently supporting building and running pipelines, integrating configs and services.
MIT License
13 stars 4 forks source link

improvements to the auto scoping mechanisms #163

Open ms-lolo opened 4 months ago

ms-lolo commented 4 months ago

we're creating more types of data structures that act as global identifiers across the project. ServiceId, CommandId, and others need to be globally unique, so we auto-prefix them with the module path in order to guarantee uniqueness and also give log messages useful metadata that shows authors the path to the definition of a given id.

Elon pointed out that using the full module path typically includes the private module file, which makes things fragile when we move the implementation across private files. We decided the prefix should probably be the path to the nearest public package where this id object is exposed. this allows us to move the class without breaking things like serialized json that might include these id objects. it's reasonable to expect things to break when we move an id class between public packages, but that matches the expectation of any other public api change.

so instead of ServiceId("rats.pipelines._something:some-service"), we want the prefixing to be something along the lines of ServiceId("rats.pipelines:Something.some-service").

ms-lolo commented 1 month ago

used the current api for a few months and have some thoughts.

this is our current pattern for defining service ids:

from rats import apps

@apps.autoscope
class PluginServices:
    THING_ONE = ServiceId[apps.Executable]("thing-one")
    THING_TWO = ServiceId[apps.Executable]("thing-two")

but this pattern doesn't provide an obvious enough api:

maybe a similar direction along these lines could work and looks less foreign to me:

from rats import apps
from typing import NamedTuple

@apps.autoscope
class PluginServices(NamedTuple):
    thing_one: ServiceId[apps.Executable]
    thing_two: ServiceId[apps.Executable]

SERVICES = PluginServices(
    thing_one=ServiceId[apps.Executable]("thing-one"),
    thing_two=ServiceId[apps.Executable]("thing-two"),
)

… still noodling