Open ncollier opened 1 year ago
Relevant points:
So, if we use keys, we are by-passing the proxying -- which may be fine given that we most likely resolve an object as soon as we use it. W/r to 2., given that a key is an object we would need to pickle it and go through the encoding / decoding that we do now when pickling the proxies themselves.
The proxystore API has some changes since the proxies.py code was written. Stores are now created with
# Store(name, Connector)
store = Store('my-store', FileConnector('/tmp/proxystore'))
When we sort out what to do in light of https://github.com/emews/EQ-SQL/issues/35#issuecomment-1601550630 we also need to update to the new API.
Options:
For context, proxy payload looks like:
func = proxies.dump_proxies(f=test_proxy_wf.task_func)['f']
proxy_map = proxies.dump_proxies(c=1.0)
params = pop_to_dict(pop, ('x', 'y'))
# use the required payload dict names
payload = json.dumps({'func': func, 'proxies': proxy_map, 'parameters': params})
And swift-t loads the proxies with:
f_str = r'%s'
// f_str in the proxied function
f = proxies.load_proxies({'f': json.loads(f_str)})['f']
proxy_str = r'%s'
params_str = r'%s'
r = f(proxy_str, params_str)
The function uses our decorator:
# this function is proxied in the ME, and proxies.app decorator
# unpacks the arguments.
@proxies.app
def task_func(c, x, y):
result = c * sin(4 * x) + sin(4 * y) + -2 * x + x**2 - 2 * y + y**2
return result
Rather than storing a representation of the proxy object in the DB use its key.
Create a simple proxy store example that exercises this and put in emews_examples repo.