[Bug] Conflict using temporal-sdk with loguru

[northpowered]

What are you really trying to do?

I`m trying to use Loguru module with temporal-sdk code

OS: Ubuntu 22.04 LTS 5.15.0-33-generic

• python = 3.10.4
• temporalio = "0.1b3"
• loguru = "0.6.0"

from temporalio.client import Client
from temporalio.worker import Worker
from loguru import logger

#  Some code

or

from temporalio import workflow, activity
from loguru import logger

# Some code

Describe the bug

Imports like these raise this:

Long traceback, CLICK to expand

Traceback (most recent call last): File "/home/northstar/.cache/pypoetry/virtualenvs/alif-interactor-tmPkMBix-py3.10/lib/python3.10/site-packages/temporalio/worker/_workflow.py", line 113, in __init__ workflow_runner.prepare_workflow(defn) File "/home/northstar/.cache/pypoetry/virtualenvs/alif-interactor-tmPkMBix-py3.10/lib/python3.10/site-packages/temporalio/worker/workflow_sandbox/_runner.py", line 53, in prepare_workflow self.create_instance( File "/home/northstar/.cache/pypoetry/virtualenvs/alif-interactor-tmPkMBix-py3.10/lib/python3.10/site-packages/temporalio/worker/workflow_sandbox/_runner.py", line 87, in create_instance return _Instance(det, self._runner_class, self._restrictions) File "/home/northstar/.cache/pypoetry/virtualenvs/alif-interactor-tmPkMBix-py3.10/lib/python3.10/site-packages/temporalio/worker/workflow_sandbox/_runner.py", line 107, in __init__ self._create_instance() File "/home/northstar/.cache/pypoetry/virtualenvs/alif-interactor-tmPkMBix-py3.10/lib/python3.10/site-packages/temporalio/worker/workflow_sandbox/_runner.py", line 118, in _create_instance self._run_code( File "/home/northstar/.cache/pypoetry/virtualenvs/alif-interactor-tmPkMBix-py3.10/lib/python3.10/site-packages/temporalio/worker/workflow_sandbox/_runner.py", line 160, in _run_code exec(code, self.globals_and_locals, self.globals_and_locals) File "", line 2, in File "/home/northstar/.cache/pypoetry/virtualenvs/alif-interactor-tmPkMBix-py3.10/lib/python3.10/site-packages/temporalio/worker/workflow_sandbox/_importer.py", line 398, in __call__ return self.current(*args, **kwargs) File "/home/northstar/.cache/pypoetry/virtualenvs/alif-interactor-tmPkMBix-py3.10/lib/python3.10/site-packages/temporalio/worker/workflow_sandbox/_importer.py", line 201, in _import mod = importlib.__import__(name, globals, locals, fromlist, level) File "", line 1129, in __import__ File "", line 1050, in _gcd_import File "", line 1027, in _find_and_load File "", line 992, in _find_and_load_unlocked File "", line 241, in _call_with_frames_removed File "", line 1050, in _gcd_import File "", line 1027, in _find_and_load File "", line 1006, in _find_and_load_unlocked File "", line 688, in _load_unlocked File "", line 883, in exec_module File "", line 241, in _call_with_frames_removed File "/home/northstar/projects/alif-interactor/source/temporal_runner/__init__.py", line 1, in from .factory import Temporal File "/home/northstar/.cache/pypoetry/virtualenvs/alif-interactor-tmPkMBix-py3.10/lib/python3.10/site-packages/temporalio/worker/workflow_sandbox/_importer.py", line 398, in __call__ return self.current(*args, **kwargs) File "/home/northstar/.cache/pypoetry/virtualenvs/alif-interactor-tmPkMBix-py3.10/lib/python3.10/site-packages/temporalio/worker/workflow_sandbox/_importer.py", line 201, in _import mod = importlib.__import__(name, globals, locals, fromlist, level) File "", line 1133, in __import__ File "", line 1050, in _gcd_import File "", line 1027, in _find_and_load File "", line 1006, in _find_and_load_unlocked File "", line 688, in _load_unlocked File "", line 883, in exec_module File "", line 241, in _call_with_frames_removed File "/home/northstar/projects/alif-interactor/source/temporal_runner/factory.py", line 3, in from loguru import logger File "/home/northstar/.cache/pypoetry/virtualenvs/alif-interactor-tmPkMBix-py3.10/lib/python3.10/site-packages/temporalio/worker/workflow_sandbox/_importer.py", line 398, in __call__ return self.current(*args, **kwargs) File "/home/northstar/.cache/pypoetry/virtualenvs/alif-interactor-tmPkMBix-py3.10/lib/python3.10/site-packages/temporalio/worker/workflow_sandbox/_importer.py", line 201, in _import mod = importlib.__import__(name, globals, locals, fromlist, level) File "", line 1129, in __import__ File "", line 1050, in _gcd_import File "", line 1027, in _find_and_load File "", line 1006, in _find_and_load_unlocked File "", line 688, in _load_unlocked File "", line 883, in exec_module File "", line 241, in _call_with_frames_removed File "/home/northstar/.cache/pypoetry/virtualenvs/alif-interactor-tmPkMBix-py3.10/lib/python3.10/site-packages/loguru/__init__.py", line 10, in from ._logger import Core as _Core File "/home/northstar/.cache/pypoetry/virtualenvs/alif-interactor-tmPkMBix-py3.10/lib/python3.10/site-packages/temporalio/worker/workflow_sandbox/_importer.py", line 398, in __call__ return self.current(*args, **kwargs) File "/home/northstar/.cache/pypoetry/virtualenvs/alif-interactor-tmPkMBix-py3.10/lib/python3.10/site-packages/temporalio/worker/workflow_sandbox/_importer.py", line 201, in _import mod = importlib.__import__(name, globals, locals, fromlist, level) File "", line 1133, in __import__ File "", line 1050, in _gcd_import File "", line 1027, in _find_and_load File "", line 1006, in _find_and_load_unlocked File "", line 688, in _load_unlocked File "", line 883, in exec_module File "", line 241, in _call_with_frames_removed File "/home/northstar/.cache/pypoetry/virtualenvs/alif-interactor-tmPkMBix-py3.10/lib/python3.10/site-packages/loguru/_logger.py", line 92, in from ._datetime import aware_now File "/home/northstar/.cache/pypoetry/virtualenvs/alif-interactor-tmPkMBix-py3.10/lib/python3.10/site-packages/temporalio/worker/workflow_sandbox/_importer.py", line 398, in __call__ return self.current(*args, **kwargs) File "/home/northstar/.cache/pypoetry/virtualenvs/alif-interactor-tmPkMBix-py3.10/lib/python3.10/site-packages/temporalio/worker/workflow_sandbox/_importer.py", line 201, in _import mod = importlib.__import__(name, globals, locals, fromlist, level) File "", line 1133, in __import__ File "", line 1050, in _gcd_import File "", line 1027, in _find_and_load File "", line 1006, in _find_and_load_unlocked File "", line 688, in _load_unlocked File "", line 883, in exec_module File "", line 241, in _call_with_frames_removed File "/home/northstar/.cache/pypoetry/virtualenvs/alif-interactor-tmPkMBix-py3.10/lib/python3.10/site-packages/loguru/_datetime.py", line 12, in class datetime(datetime_): TypeError: _RestrictedProxy.__init__() missing 1 required positional argument: 'matcher' The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/northstar/projects/alif-interactor/source/main.py", line 13, in app() # pragma: no cover File "/home/northstar/projects/alif-interactor/source/cli/run.py", line 14, in workflows temporal.start_worker_sync( File "/home/northstar/projects/alif-interactor/source/temporal_runner/factory.py", line 39, in start_worker_sync w = asyncio.run(generator) File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run return loop.run_until_complete(main) File "/usr/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete return future.result() File "/home/northstar/projects/alif-interactor/source/temporal_runner/factory.py", line 66, in create_workflow_worker return Worker( File "/home/northstar/.cache/pypoetry/virtualenvs/alif-interactor-tmPkMBix-py3.10/lib/python3.10/site-packages/temporalio/worker/_worker.py", line 261, in __init__ self._workflow_worker = _WorkflowWorker( File "/home/northstar/.cache/pypoetry/virtualenvs/alif-interactor-tmPkMBix-py3.10/lib/python3.10/site-packages/temporalio/worker/_workflow.py", line 117, in __init__ raise RuntimeError(f"Failed validating workflow {defn.name}") from err RuntimeError: Failed validating workflow FooFlow

Some notes:

• I`d never seen conflicts like this
• Loguru fails only in code file with temporal-sdk imports
• This exception rises THIS validation exception
• Using Loguru in other modules WITHOUT temporal code, but in THIS virtualenv is ok

For example (this is working fine)

import typer
from cli import run_app  # Importing module with temporal code
from utils.logger import logger  # Importing loguru

app = typer.Typer(no_args_is_help=True, help="add help")
app.add_typer(run_app, name='run')

if __name__ == "__main__":
    logger.info('start')  
    app() 

[cretz]

This is due to a known issue with the sandbox when imports extend restricted classes like datetime. See https://github.com/temporalio/sdk-python/tree/0.1b3#extending-restricted-classes. Hopefully this will be fixed in a release coming soon, but in the meantime, you need to mark loguru as a "passthrough module" (https://github.com/temporalio/sdk-python/tree/0.1b3#passthrough-modules), e.g.:

my_restrictions = dataclasses.replace(
    SandboxRestrictions.default,
    passthrough_modules=SandboxRestrictions.passthrough_modules_default | SandboxMatcher(access={"loguru"}),
)
my_worker = Worker(..., workflow_runner=SandboxedWorkflowRunner(restrictions=my_restrictions))

We have made this easier in main which should be released shortly.

[northpowered]

@cretz This works great, tnx a lot! Adding some modules to custom restrictions should look like this?

access={"loguru","pydantic"}

[cretz]

Yes, but that is going to change in a couple of days :-) See the equivalent section on main at https://github.com/temporalio/sdk-python#passthrough-modules (though hopefully this bug won't surface unless you actually try to use the logger in the workflow file/class which you probably shouldn't).

[northpowered]

Thanx, we`re not using logging right in the workflows :-)

Closed