Parsl / parsl

Parsl - a Python parallel scripting library
http://parsl-project.org
Apache License 2.0
508 stars 195 forks source link

Unable to get source code for app caching #2135

Open daheise opened 3 years ago

daheise commented 3 years ago

Describe the bug I receive the following warning on bash_app task startup:

serializing id_for_memo_function for function <function wrap_error.<locals>.wrapper at 0xHEXADDR>, type <class 'function'>
[WARNING] Unable to get source code for app caching. Recommend creating module. Exception was: functools.partial (<function hash_results at 0xHEXADDR>), <function APPNAME at 0xHEXADDR) is not module, class, method, function, traceback, frame, or code object

To Reproduce Steps to reproduce the behavior, for e.g:

  1. Setup Parsl 1.1.0 with Python 3.6 on cluster
  2. Run a task

Expected behavior Caching takes app source code into account.

Environment

Distributed Environment

benclifford commented 3 years ago

I think this a bug that affects any attempts by parsl to get the app source that was introduced by rearrangements I made to try to consolidate all the memoization code in one place - I also get it with a fairly trivial test on my laptop against 1.1.0