Unable to validate Experiment Recipe schema in Python

[jaredlockhart]

Using 0.0.3:

In [2]: check_schema('experiments/Experiment', {})                                                                                                                                    
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-2-8a057b79bf54> in <module>
----> 1 check_schema('experiments/Experiment', {})

/usr/local/lib/python3.8/site-packages/mozilla_nimbus_shared/__init__.py in check_schema(schema_name, instance)
     19     if schema_name not in _schema_cache:
     20         schema_path = "schemas/" + schema_name + ".json"
---> 21         schema_text = pkgutil.get_data("mozilla_nimbus_shared", schema_path)
     22         _schema_cache[schema_name] = json.loads(schema_text)
     23 

/usr/local/lib/python3.8/pkgutil.py in get_data(package, resource)
    635     parts.insert(0, os.path.dirname(mod.__file__))
    636     resource_name = os.path.join(*parts)
--> 637     return loader.get_data(resource_name)

/usr/local/lib/python3.8/importlib/_bootstrap_external.py in get_data(self, path)

FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/lib/python3.8/site-packages/mozilla_nimbus_shared/schemas/experiments/Experiment.json'

[mythmon]

Well that's no bueno. I'm guessing this is running in Docker? Is it something isolated, or is it Experimenter? As a sanity check, can you see if /usr/local/lib/python3.8/site-packages/mozilla_nimbus_shared/schemas/experiments/Experiment.json, or it's parents mozilla_nimbus_shared/schemas and mozilla_nimbus_shared/schemas/experiments exist?

[pdehaan]

I created a new venv w/ Python 3.7 locally in an empty directory and this is what seems to get installed in my venv site-packages' "mozilla_nimbus_shared" directory:

pip install mozilla-nimbus-shared
tree venv/lib/python3.7/site-packages/mozilla_nimbus_shared

venv/lib/python3.7/site-packages/mozilla_nimbus_shared
├── __init__.py
├── __pycache__
│   └── __init__.cpython-37.pyc
└── schemas
    ├── messaging
    │   └── SimpleCFRMessage.json
    └── normandy
        ├── AddonRollbackArguments.json
        ├── AddonRolloutArguments.json
        ├── BranchedAddonStudyArguments.json
        ├── ConsoleLogArguments.json
        ├── MessagingExperimentArguments.json
        ├── MultiPreferenceExperimentArguments.json
        ├── NormandyRecipe.json
        ├── OptOutStudyArguments.json
        ├── PreferenceExperimentArguments.json
        ├── PreferenceRollbackArguments.json
        ├── PreferenceRolloutArguments.json
        └── ShowHeartbeatArguments.json

4 directories, 15 files

[mythmon]

Oh, well then this is easy. experiments/Experiment wasn't a part of the 0.0.3 release: https://github.com/mozilla/nimbus-shared/tree/f8c5a09af64ef37a794b1a2870aad7d373efb030/types

It will be in 0.0.4.

[jaredlockhart]

I wonder if we should setup a regular release process or just keep it ad hoc? We should mention that in the next retro.

[mythmon]

With 0.0.4 released, this should be ready to go.