"glia analyze (filename) integrity" fails with "json.decoder.JSONDecodeError"

atthewmay commented 4 years ago

Issue Description

Running "glia analyze --configuration config.yaml R1_E3_AMES_540min_ffhq_BnW_6100_9200 integrity" gives a "json.decoder.JSONDecodeError" in the try-except block of create_epl_gen_v2 in glia/eyecandy.py.

In my eye-candy directory, I ran "docker-compose up" prior to running the above command.

I have performed "git pull" for the eye-candy directory and the glia directory. Tried while on the "master" branch of glia and the 3brain_units branch.

Steps to reproduce.

docker-compose up to start eyecandy, using the attached docker-config.yml.
Run "glia analyze --configuration config.yaml R1_E3_AMES_540min_ffhq_BnW_6100_9200 integrity", using the attached config.yaml file (for manual thresholding).

What's the Actual Result?

Copied below are 1. the error message from glia, and 2. The error message from eyecandy.

From Glia (glia) matthewhunt@Matthews-MacBook-Pro 200818 % glia analyze --configuration config.yaml R1_E3_AMES_540min_ffhq_BnW_6100_9200 integrity version 0.5.1 Analyzing R1_E3_AMES_540min_ffhq_BnW_6100_9200 No .stim file found. Creating from .analog file. couldn't parse json from <Response [500]> Traceback (most recent call last): File "/Users/matthewhunt/miniconda3/envs/glia/bin/glia", line 11, in load_entry_point('glia', 'console_scripts', 'glia')() File "/Users/matthewhunt/miniconda3/envs/glia/lib/python3.8/site-packages/click/core.py", line 829, in call return self.main(args, kwargs) File "/Users/matthewhunt/miniconda3/envs/glia/lib/python3.8/site-packages/click/core.py", line 782, in main rv = self.invoke(ctx) File "/Users/matthewhunt/miniconda3/envs/glia/lib/python3.8/site-packages/click/core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/Users/matthewhunt/miniconda3/envs/glia/lib/python3.8/site-packages/click/core.py", line 1268, in invoke Command.invoke(self, ctx) File "/Users/matthewhunt/miniconda3/envs/glia/lib/python3.8/site-packages/click/core.py", line 1066, in invoke return ctx.invoke(self.callback, ctx.params) File "/Users/matthewhunt/miniconda3/envs/glia/lib/python3.8/site-packages/click/core.py", line 610, in invoke return callback(args, *kwargs) File "/Users/matthewhunt/miniconda3/envs/glia/lib/python3.8/site-packages/click/decorators.py", line 21, in new_func return f(get_current_context(), args, kwargs) File "/Users/matthewhunt/Coding/Seattle_Research/MEA_Project/glia/glia_scripts/command_line.py", line 343, in analyze metadata, stimulus_list = glia.create_stimuli( File "/Users/matthewhunt/Coding/Seattle_Research/MEA_Project/glia/glia/eyecandy.py", line 220, in create_stimuli metadata, stimulus_gen = create_epl_gen_v2(program, epl, window_width, File "/Users/matthewhunt/Coding/Seattle_Research/MEA_Project/glia/glia/eyecandy.py", line 71, in create_epl_gen_v2 response = r.json() File "/Users/matthewhunt/miniconda3/envs/glia/lib/python3.8/site-packages/requests/models.py", line 898, in json return complexjson.loads(self.text, kwargs) File "/Users/matthewhunt/miniconda3/envs/glia/lib/python3.8/json/init.py", line 357, in loads return _default_decoder.decode(s) File "/Users/matthewhunt/miniconda3/envs/glia/lib/python3.8/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/Users/matthewhunt/miniconda3/envs/glia/lib/python3.8/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
The error message from eyecandy: nginx_1 | 172.17.0.1 - - [02/Sep/2020:22:24:17 +0000] "POST /analysis/run-program HTTP/1.1" 500 21 "-" "python-requests/2.24.0" "-" web_1 | <-- POST /analysis/run-program web_1 | analysis/run-program web_1 | compiling EPL. web_1 | xxx POST /analysis/run-program 500 540ms - web_1 | web_1 | ReferenceError: measureIntegrity is not defined web_1 | at vm.js:39:1 web_1 | at Script.runInContext (vm.js:107:20) web_1 | at VM.run (/www/node_modules/vm2/lib/main.js:204:72) web_1 | at compileJSProgram (/www/src/epl/eval.js:43:8) web_1 | at router.post.ctx (/www/src/app.js:268:13) web_1 | at dispatch (/www/node_modules/koa-router/node_modules/koa-compose/index.js:44:32) web_1 | at next (/www/node_modules/koa-router/node_modules/koa-compose/index.js:45:18) web_1 | at /www/node_modules/koa-router/lib/router.js:346:16 web_1 | at dispatch (/www/node_modules/koa-router/node_modules/koa-compose/index.js:44:32) web_1 | at /www/node_modules/koa-router/node_modules/koa-compose/index.js:36:12 web_1 |

atthewmay commented 4 years ago

The config.yml for the "glia analyze" looks like:

The docker-compose.yml looks like this

tbenst commented 4 years ago

Thanks for creating this issue! I’ll take a closer look tomorrow but in the meantime a couple issues:

the calibration file should have 6 values (2 for each stimulus_index % 3—the three “Eye of Sauron” modes in histogram)
In future please post the relevant lab notebook. You can upload a gist if you would like. Lab notebook for this experiment can be found by searching for the name “ R1_E3_AMES_540min_ffhq_BnW_6100_9200“ and contains important fields for troubleshooting, particularly the epl
your error is because the EPL is failing to compile. In particular, EyeCandy used to define measureIntegrity in every EPL file. This has now been added as a function in the global namespace of the language, per https://github.com/tbenst/eye-candy/pull/31/files. See generators.js. This suggests that your eyecandy is not the latest version. If you have not restarted your docker container after pulling and rebuilding, please do so

If that does not solve the issue, please let me know

atthewmay commented 4 years ago

Also, in case this is confusing, I had to rename the experiment to be "R1_E3_AMES_540min_ffhq_BnW_6100_9200" rather "than R1_E3_AMES_540min_ffhq_BnW_6100_9201", bc it seemed the lab_notebook.yml had 6100_9200.

From: Tyler Benster notifications@github.com Sent: Wednesday, September 2, 2020 8:40 PM To: tbenst/glia glia@noreply.github.com Cc: Hunt, Matthew huntm@wustl.edu; Author author@noreply.github.com Subject: Re: [tbenst/glia] "glia analyze (filename) integrity" fails with "json.decoder.JSONDecodeError" (#29)

Thanks for creating this issue! I’ll take a closer look tomorrow but in the meantime a couple issues:

the calibration file should have 6 values (2 for each stimulus_index % 3—the three “Eye of Sauron” modes in histogram)
In future please post the relevant lab notebook. You can upload a gist if you would like. Lab notebook for this experiment can be found by searching for the name “ R1_E3_AMES_540min_ffhq_BnW_6100_9200“ and contains important fields for troubleshooting, particularly the epl
your error is because the EPL is failing to compile. In particular, EyeCandy used to define measureIntegrity in every EPL file. This has now been added as a function in the global namespace of the language, per https://github.com/tbenst/eye-candy/pull/31/files. See generators.js. This suggests that your eyecandy is not the latest version. If you have not restarted your docker container after pulling and rebuilding, please do so

If that does not solve the issue, please let me know

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/tbenst/glia/issues/29#issuecomment-686180923, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AHTEQ2RJNLIO3HONDMFWGZTSD3X2RANCNFSM4QUAIFSA.

The materials in this message are private and may contain Protected Healthcare Information or other information of a sensitive nature. If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail.

tbenst / glia