jsiegle
commented
4 years ago This is definitely something we should fix. Right now the marshmallow version is pinned to 2.19.2, but I can change this to the latest release. After that, it should just be matter of changing create_input_json.py to return different dictionaries depending on the module name, as well as making sure the OutputParameters match what's expected.
Marshmallow 3.0 (and consequently, argschema 2.0) have changed things so that schemas are always strict, making it so that any extra keys or invalid data are passed, gets you a validation error.
I know there is a force downgrade parameter in the install file for this package, but newer versions of different packages (most relevantly, the Allen SDK), require the newest versions or Marshmallow/Argschema.
See Marshmallow upgrade documentation here: https://marshmallow.readthedocs.io/en/stable/upgrading.html
So far, only working in the NPX extraction module: This causes errors in making the input JSON file (which can be worked around by creating module-specific files and running them one at a time), and also in the output JSON file, due to mismatched key names in OutputParameters class and the keys returned by run_npx_extraction in main.py
extract_from_npx. main.py:
return {"execution_time" : execution_time, "npx_extractor_commit_date" : commit_date, "npx_extractor_commit_hash" : commit_hash }extract_from_npx._schemas.py:
class OutputParameters(OutputSchema): npx_extractor_execution_time = Float() settings_json = String() npx_extractor_commit_hash = String() npx_extractor_commit_date = String()Traceback of error:
Traceback (most recent call last): File "C:\Users\saharm\AppData\Local\Continuum\anaconda3\envs\py3\lib\runpy.py", line 193, in _run_module_as_main "__main__", mod_spec) File "C:\Users\saharm\AppData\Local\Continuum\anaconda3\envs\py3\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "c:\users\saharm\documents\code\code_package_downloads\ecephys_spike_sorting\ecephys_spike_sorting\modules\extract_from_npx\__main__.py", line 77, in <module> main() File "c:\users\saharm\documents\code\code_package_downloads\ecephys_spike_sorting\ecephys_spike_sorting\modules\extract_from_npx\__main__.py", line 71, in main mod.output(output, indent=2) File "C:\Users\saharm\AppData\Local\Continuum\anaconda3\envs\py3\lib\site-packages\argschema\argschema_parser.py", line 234, in output output_json = self.get_output_json(d) File "C:\Users\saharm\AppData\Local\Continuum\anaconda3\envs\py3\lib\site-packages\argschema\argschema_parser.py", line 204, in get_output_json raise(mm.ValidationError(errors)) marshmallow.exceptions.ValidationError: {'execution_time': ['Unknown field.']}