Closed russelljjarvis closed 6 years ago
pwollstadt
commented
6 years ago Thanks for sharing @russelljjarvis. I will add this to the wiki if you don't mind.
You only need pyopencl if you intend to use the GPU/OpenCL estimators. That's why IDTxl can be installed even if pyopencl is not present on your system. You can use the JIDT/JAVA estimators for TE estimation. It will complain though when you try to use the OpenCL estimators.
russelljjarvis
commented
6 years ago I have been reading your wiki a lot, actually I will need to revise the Dockerbuild instructions a bit. Since the docker build above still breaks, but during later instructions.
I have been trying to do multivariate TE estimates in binned binary spike train data (from neo and elephant data types), that come a PyNN network simulation.
I found that I could wrangle the neo spike train into a idtxl.data object, but when I tried to run the code, I got errors about missing :
OpenCL.settings = {'cmi_estimator': 'JidtKraskovCMI',
'max_lag_sources': 3,
'max_lag_target': 3,
'min_lag_sources': 1}Do you think if I simply change n_procs from =2 to =1, the code will no longer depend on OpenCL, as it will no longer be regarded as parallel code?
Note about snippet below mdf1 is a neo segment object containing analog signals (membrane potentials) and spike trains. It's tempting to use the continuous membrane potentials as signals that are input into multivariate TE, as I suspect interactions between inhibitory to excitatory neurons could be better represented using the continuous membrane potential.
This code works:
import numpy as np
from idtxl.multivariate_te import MultivariateTE
from idtxl.data import Data
n_procs = 1
settings = {
'cmi_estimator': 'JidtDiscreteCMI',
'n_perm_max_stat': 21,
'max_lag_target': 5,
'max_lag_sources': 5,
'min_lag_sources': 4}
binary_trains = []
for spiketrain in mdf1.spiketrains:
x = conv.BinnedSpikeTrain(spiketrain, binsize=5 * pq.ms, t_start=0 * pq.s)
binary_trains.append(x.to_array())
dat = Data(np.array(binary_trains), dim_order='spr')This code breaks:
dat.n_procs = n_procs
mte = MultivariateTE()
res_full = mte.analyse_network(settings=settings, data=dat)Traceback
0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0,
0, 0, 0, 0]], dtype=int64)]
Adding data with properties: 1 processes, 122 samples, 400 replications
overwriting existing data
<idtxl.data.Data object at 0x7fc267b39a58>
####### analysing target with index 0 from list [0]
PyOpenCl is not available on this system. Install it using pip or the package manager to use OpenCL-powered CMI estimation.
File "/opt/conda/lib/python3.5/site-packages/idtxl/estimators_opencl.py", line 7, in <module>
import pyopencl as cl
Traceback (most recent call last):
File "sa.py", line 112, in <module>
te(mdfl)
File "sa.py", line 77, in te
res_full = mte.analyse_network(settings=settings, data=dat)
File "/opt/conda/lib/python3.5/site-packages/idtxl/multivariate_te.py", line 163, in analyse_network
sources[t])
File "/opt/conda/lib/python3.5/site-packages/idtxl/multivariate_te.py", line 273, in analyse_single_target
self._initialise(settings, data, sources, target)
File "/opt/conda/lib/python3.5/site-packages/idtxl/network_inference.py", line 51, in _initialise
self._cmi_estimator = EstimatorClass(settings)
TypeError: Can't instantiate abstract class JidtKraskov with abstract methods estimate
mwibral
commented
6 years ago Hi Russell,
I wonder where that opencl dependency creeps in. Sicne youi're using 'JIDTKraskovCMI' the code shouldn't need any oepncl. Could you post the full error message?
Best,
Michael Wibral
On 23.04.2018 05:22, Russell Jarvis wrote:
I have been reading your wiki a lot, actually I will need to revise the Dockerbuild instructions a bit.
I have been trying to do multivariate TE estimates in binned binary spike train data (from neo and elephant data types), that come a PyNN network simulation.
I found that I could wrangle the neo spike train into a |idtxl.data| object, but when I tried to run the code, I got errors about missing OpenCL.
Do you think if I simply change |n_procs| from |=2| to |=1|, the code will no longer depend on OpenCL, as it will no longer be regarded as parallel code?
Works:
import numpyas np from idtxl.multivariate_teimport MultivariateTE from idtxl.dataimport Data n_procs= 1 settings= { 'cmi_estimator':'JidtKraskovCMI', 'n_perm_max_stat':21, 'max_lag_target':5, 'max_lag_sources':5, 'min_lag_sources':4} settings['cmi_estimator']= 'JidtKraskovCMI' binary_trains= [] for spiketrainin mdf1.spiketrains: x= conv.BinnedSpikeTrain(spiketrain,binsize=5 * pq.ms,t_start=0 * pq.s) binary_trains.append(x.to_array()) dat= Data(np.array(binary_trains),dim_order='spr')breaks:
dat.n_procs= n_procs mte= MultivariateTE() res_full= mte.analyse_network(settings=settings,data=dat)— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/pwollstadt/IDTxl/issues/12#issuecomment-383443748, or mute the thread https://github.com/notifications/unsubscribe-auth/AIqYGtJN8XpRUyGsrE1veV1NovxKTLW5ks5trUkJgaJpZM4Td8as.
{"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/pwollstadt/IDTxl","title":"pwollstadt/IDTxl","subtitle":"GitHub repository","main_image_url":"https://cloud.githubusercontent.com/assets/143418/17495839/a5054eac-5d88-11e6-95fc-7290892c7bb5.png","avatar_image_url":"https://cloud.githubusercontent.com/assets/143418/15842166/7c72db34-2c0b-11e6-9aed-b52498112777.png","action":{"name":"Open in GitHub","url":"https://github.com/pwollstadt/IDTxl"}},"updates":{"snippets":[{"icon":"PERSON","message":"@russelljjarvis in #12: I have been reading your wiki a lot, actually I will need to revise the Dockerbuild instructions a bit.\r\n\r\nI have been trying to do multivariate TE estimates in binned binary spike train data (from neo and elephant data types), that come a PyNN network simulation.\r\n\r\nI found that I could wrangle the neo spike train into a
idtxl.dataobject, but when I tried to run the code, I got errors about missing OpenCL.\r\n\r\nDo you think if I simply changen_procsfrom=2to=1, the code will no longer depend on OpenCL, as it will no longer be regarded as parallel code?\r\n\r\nWorks:\r\npython\r\n import numpy as np\r\n from idtxl.multivariate_te import MultivariateTE\r\n from idtxl.data import Data\r\n n_procs = 1\r\n settings = {\r\n 'cmi_estimator': 'JidtKraskovCMI',\r\n 'n_perm_max_stat': 21,\r\n 'max_lag_target': 5,\r\n 'max_lag_sources': 5,\r\n 'min_lag_sources': 4}\r\n settings['cmi_estimator'] = 'JidtKraskovCMI'\r\n binary_trains = []\r\n for spiketrain in mdf1.spiketrains:\r\n x = conv.BinnedSpikeTrain(spiketrain, binsize=5 * pq.ms, t_start=0 * pq.s)\r\n binary_trains.append(x.to_array())\r\n dat = Data(np.array(binary_trains), dim_order='spr')\r\n\r\nbreaks:\r\npython\r\n dat.n_procs = n_procs\r\n\r\n mte = MultivariateTE()\r\n res_full = mte.analyse_network(settings=settings, data=dat)\r\n"}],"action":{"name":"View Issue","url":"https://github.com/pwollstadt/IDTxl/issues/12#issuecomment-383443748"}}}
Prof. Dr. rer. nat. Michael Wibral MEG Labor, Brain Imaging Center Goethe Universität
Heinrich Hoffmann Strasse 10 60528 Frankfurt am Main
russelljjarvis
commented
6 years ago Yes, that's why I am confused too. All of the work I am referencing resides in git repositories. I have done so in a seperate dedicated issue https://github.com/pwollstadt/IDTxl/issues/13
Russell.
pwollstadt
commented
6 years ago Hi Russel, I tried to replicate the issue about IDTxl unintentionally importing pyopencl even though your are using the JIDT estimator, but the below scripts runs fine (in a conda environment with JPype and without pyopencl). Could you run the script on your machine and tell me what happens?
![https://cloud.githubusercontent.com/assets/143418/17495839/a5054eac-5d88-11e6-95fc-7290892c7bb5.png","avatar_image_url":"https://cloud.githubusercontent.com/assets/143418/15842166/7c72db34-2c0b-11e6-9aed-b52498112777.png","action":{"name":"Open](https://cloud.githubusercontent.com/assets/143418/17495839/a5054eac-5d88-11e6-95fc-7290892c7bb5.png","avatar_image_url":"https://cloud.githubusercontent.com/assets/143418/15842166/7c72db34-2c0b-11e6-9aed-b52498112777.png","action":{"name":"Open){kind=link}
import numpy as np
from idtxl.multivariate_te import MultivariateTE
from idtxl.data import Data
dat = Data(np.random.rand(10,100,3), dim_order='psr')
settings = {
'cmi_estimator': 'JidtKraskovCMI',
'n_perm_max_stat': 21,
'max_lag_target': 5,
'max_lag_sources': 5,
'min_lag_sources': 4}
mte = MultivariateTE()
res_full = mte.analyse_network(settings=settings, data=dat, targets=[0])Regarding the other issues: As i mentioned in #13, you don't need to set the number of processes, the Data() object gets this from the dimension of your data array. Also, the error TypeError: Can't instantiate abstract class JidtKraskov with abstract methods estimate could result from setting settings['cmi_estimator']='JidtKraskov' instead of settings['cmi_estimator']='JidtKraskovCMI'. Can you make sure, the estimator name is specified correctly?
Thanks!
russelljjarvis
commented
6 years ago I agree with you, it will probably run on my end as described. I will get back to this issue in the next week. Russell.
pwollstadt
commented
6 years ago Hi all, I am closing this issue because it has been inactive for a few months. Please reopen it if you have any news on installation instructions or think about moving this discussion to the google group.
The following instructions were not sufficient to build in a Docker container for: "Debian GNU/Linux 8 (jessie)"
This line is necessary to to test for successful instalation, because
IDTxl.gitinstaller gives indication of succeeding otherwise, but TE() objects cannot be instantiated properly due to failed python3-pyopencl installation.Removing intermediate container b2b1a9f32c8e ---> df5260ec8370 Step 5/20 : RUN /opt/conda/bin/pip3 install dask psutil natsort pyspike pyNN lazyarray neo matplotlib pyopencl ---> Running in fec6bf15a424 Requirement already satisfied: dask in /opt/conda/lib/python3.5/site-packages Requirement already satisfied: psutil in /opt/conda/lib/python3.5/site-packages Collecting natsort Downloading https://files.pythonhosted.org/packages/c9/5d/64d443b303fc213bf7abe2512232d2b47acbc3e6f8e01ad19a9bf7d313cb/natsort-5.3.0-py2.py3-none-any.whl Collecting pyspike Downloading https://files.pythonhosted.org/packages/a2/bb/38f54ab8490ec342337a270dea8ef6e8300ddd5e56e58ab0c3fa52a7d74e/pyspike-0.5.3.tar.gz (1.4MB) Collecting pyNN Downloading https://files.pythonhosted.org/packages/da/e4/ab620b4730de4c08d0df6346b181eb5b9a9ffdab8aefed811452e547d1c9/PyNN-0.9.2.tar.gz (358kB) Collecting lazyarray Downloading https://files.pythonhosted.org/packages/a0/c5/4cac8d8749bea1e675aa97232e7eaa1e340b5bd56a041268eb48c8dc523c/lazyarray-0.3.2.tar.gz Collecting neo Downloading https://files.pythonhosted.org/packages/d2/8c/3a41152f82fd8b8ee2d1cbeb87872be1555f0beb47ddce9faebba02b9d9e/neo-0.6.1.tar.gz (1.5MB) Requirement already satisfied: matplotlib in /opt/conda/lib/python3.5/site-packages Collecting pyopencl Downloading https://files.pythonhosted.org/packages/d3/66/080ee1a9cc3f71f7491c7e3261b767c788be2aea0d694017cd25aa762b0e/pyopencl-2018.1.1.tar.gz (357kB) Complete output from command python setup.py egg_info: c/_cffi_backend.c:15:17: fatal error: ffi.h: No such file or directory
include
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-9we4w_vo/pyopencl/ You are using pip version 9.0.1, however version 10.0.1 is available. You should consider upgrading via the 'pip install --upgrade pip' command. The command '/bin/sh -c /opt/conda/bin/pip3 install dask psutil natsort pyspik