Feature: Serialize Python Values, Add Job API, and create BF2 Work Function Wrapper

[Severn-J-Lortie]

This MR adds most APIs outlined for the Job Class

[wiwichips]

Awesome!!!!!!!!!!!!!!!! I'm super excited to see some of the refactors you mentioned and I'm happy to see them come in as separate tickets

I've only given the code a glance through, so I'll give it a test drive now and see if I can break it some more :weary:

Sidenote: I'm happy to merge this in once CI tests pass as long as there is a plan to address feedback in subsequent MRs

(looks CI tests are timing out - are you deploying a job in your test?)

[wiwichips]

Extremely excited to see us switch to dill sometime in the future for better Python inter version comapt - but I won't hold back this MR for that feature,

[wiwichips]

Fails in tests when I run locally

================================================= test session starts =================================================
platform linux -- Python 3.10.12, pytest-7.4.4, pluggy-1.5.0
rootdir: /home/will/git/bifrost2/tests
collected 23 items                                                                                                    

test_init.py F..
test_api/test_compute_for.py F.
test_api/test_job_fs.py ..-------------------------------------
..{'/bf2-test-cf9mycbn': 'test_add_files_relative_no_dest'}
..
test_api/test_misc_api_existence.py ....
test_api/test_wallet.py ..
test_js_wrappers/test_class_registry.py .
test_js_wrappers/test_js_object.py ....
test_js_wrappers/test_wrapping_on_proxy.py .

====================================================== FAILURES =======================================================
____________________________________________ TestDcpInitFunction.test_init ____________________________________________

self = <tests.test_init.TestDcpInitFunction testMethod=test_init>

    def test_init(self):
        ret_module = dcp.init()
        from dcp import wallet
        from dcp import job

        self.assertEqual(ret_module, dcp)

>       job = job.Job('x=>{progress();return x+1', [])

test_init.py:16: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../dcp/api/job.py:21: in __init__
    job_js = dry.aio.blockify(compute_for_js)(*args, **kwargs)
../dcp/dry/aio.py:33: in blocking_fn
    return loop.run_until_complete(asyncify(async_fn)(*args, **kwargs))
/usr/lib/python3.10/asyncio/base_events.py:649: in run_until_complete
    return future.result()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

args = ('x=>{progress();return x+1', []), kwargs = {}

    async def aio_fn(*args, **kwargs):
>       return_value = leaky_async_fn(*args, **kwargs)
E       pythonmonkey.SpiderMonkeyError: Error in file /home/will/git/bifrost2/dcp/js/node_modules/dcp-client/dist/dcp-client-bundle.js, on line 1, column 1097859:
E       TypeError: Work function arguments (the last parameter), should be an array.

../dcp/dry/aio.py:23: SpiderMonkeyError
___________________________________________ TestComputeFor.test_compute_for ___________________________________________

self = <test_compute_for.TestComputeFor testMethod=test_compute_for>

    def test_compute_for(self):
        dcp.init()
        job1 = dcp.compute_for('x=>{progress(); return x * 2}', [1,2,3])
>       job2 = dcp.compute.do(5, 'x=>{progress(); return x * 2}')

test_api/test_compute_for.py:10: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../dcp/dry/class_manager.py:46: in method
    return wrap_obj(ret_val)
../dcp/dry/class_manager.py:108: in wrap_obj
    return bfclass(js_val)
../dcp/api/job.py:21: in __init__
    job_js = dry.aio.blockify(compute_for_js)(*args, **kwargs)
../dcp/dry/aio.py:33: in blocking_fn
    return loop.run_until_complete(asyncify(async_fn)(*args, **kwargs))
/usr/lib/python3.10/asyncio/base_events.py:649: in run_until_complete
    return future.result()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

args = ({'__listeners': {'newlistener': [<pythonmonkey.JSFunctionProxy object at 0x71c5692c7970>]}, 'address': <class 'pythonmonkey.null'>, 'bank': {}, 'breakpointEvents': {}, ...},)
kwargs = {}

    async def aio_fn(*args, **kwargs):
>       return_value = leaky_async_fn(*args, **kwargs)
E       pythonmonkey.SpiderMonkeyError: Error in file /home/will/git/bifrost2/dcp/js/node_modules/dcp-client/dist/dcp-client-bundle.js, on line 1, column 1098140:
E       TypeError: Work parameter must be a function or a string that evaluates to a function, or a URL that points to the location of a function.
E       Received type undefined (undefined)

../dcp/dry/aio.py:23: SpiderMonkeyError
=============================================== short test summary info ===============================================
FAILED test_init.py::TestDcpInitFunction::test_init - pythonmonkey.SpiderMonkeyError: Error in file /home/will/git/bifrost2/dcp/js/node_modules/dcp-client/dist/dcp-clie...
a
============================================ 2 failed, 21 passed in 0.73s =============================================

---

Two errors

CHANGE THE TEST TO USE compute_for instead of Job

diff --git a/tests/test_init.py b/tests/test_init.py
index 3af5ad2..207ddb1 100644
--- a/tests/test_init.py
+++ b/tests/test_init.py
@@ -13,7 +13,8 @@ class TestDcpInitFunction(unittest.TestCase):

         self.assertEqual(ret_module, dcp)

-        job = job.Job('x=>{progress();return x+1', [])
+        job = dcp.compute_for([], 'x=>{progress();return x+1')
+        job.worktime = 'map-basic'
         job.on('readystatechange', print)

Find why compute.do is broken?

<Edit: make compute.do do compute >

[wiwichips]

Tests time out

[wiwichips]

Merging - even if there's bugs, we'll fix in subsequent MRs