OSError: [Errno 30] Read-only file system for DSPy Caching

hello there,

when trying to run DSPy in AWS lambdas, we get this error:

[ERROR] OSError: [Errno 30] Read-only file system: '/home/sbx_user1051'
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/usr/src/app/langevals/server.py", line 76, in <module>
    for evaluator_cls in get_evaluator_classes(evaluator_package):
  File "/usr/src/app/langevals/utils.py", line 34, in get_evaluator_classes
    module = importlib.import_module(f"{evaluator_package.__name__}.{module_name}")
  File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/usr/src/app/evaluators/langevals/langevals_langevals/product_sentiment_polarity.py", line 11, in <module>
    import dspy
  File "/usr/src/app/.venv/lib/python3.11/site-packages/dspy/__init__.py", line 1, in <module>
    import dsp
  File "/usr/src/app/.venv/lib/python3.11/site-packages/dsp/__init__.py", line 1, in <module>
    from .modules import *
  File "/usr/src/app/.venv/lib/python3.11/site-packages/dsp/modules/__init__.py", line 7, in <module>
    from .azure_openai import AzureOpenAI
  File "/usr/src/app/.venv/lib/python3.11/site-packages/dsp/modules/azure_openai.py", line 9, in <module>
    from dsp.modules.cache_utils import CacheMemory, NotebookCacheMemory, cache_turn_on
  File "/usr/src/app/.venv/lib/python3.11/site-packages/dsp/modules/cache_utils.py", line 27, in <module>
    CacheMemory = Memory(location=cachedir, verbose=0)
  File "/usr/src/app/.venv/lib/python3.11/site-packages/joblib/memory.py", line 1020, in __init__
    self.store_backend = _store_backend_factory(
  File "/usr/src/app/.venv/lib/python3.11/site-packages/joblib/memory.py", line 132, in _store_backend_factory
    obj.configure(location, verbose=verbose,
  File "/usr/src/app/.venv/lib/python3.11/site-packages/joblib/_store_backends.py", line 452, in configure
    mkdirp(self.location)
  File "/usr/src/app/.venv/lib/python3.11/site-packages/joblib/disk.py", line 61, in mkdirp
    os.makedirs(d)
  File "<frozen os>", line 215, in makedirs
  File "<frozen os>", line 215, in makedirs
  File "<frozen os>", line 225, in makedirs

This is due to DSPy cache assuming home path as a default place to write caches. We can fix with this line before loading the module:

import os
from tempfile import mkdtemp

# Necessary for running DSPy on AWS lambdas
os.environ["DSP_CACHEDIR"] = mkdtemp()

however that is a bit unexpected, maybe default behaviour should be to fallback to mkdtemp already

cheers!

stanfordnlp / dspy

OSError: [Errno 30] Read-only file system for DSPy Caching #1115