Problem in creating database

[anamika168]

pandeya@kw12521:~/git/mimclib$ python -c 'from mimclib.db import MIMCDatabase; print MIMCDatabase().DBCreationScript();' | mysql ERROR 1045 (28000): Access denied for user 'pandeya'@'localhost' (using password: NO) Traceback (most recent call last): File "", line 1, in File "mimclib/init.py", line 3, in from .mimc import MIMCRun, MIMCItrData File "mimclib/mimc.py", line 12, in from scipy.stats import norm ImportError: No module named scipy.stats

[Virtakuono]

While your problem is triggered by trying to create a database, I doubt it is database related. My guess is that you have missing dependencies, at least scipy. If you run ipython -i -c "import scipy" what happens?

[anamika168]

pandeya@kw12521:~/git/mimclib$ ipython -i -c "import scipy" Python 2.7.3 (default, Oct 26 2016, 21:01:49) Type "copyright", "credits" or "license" for more information.

IPython 0.12.1 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object', use 'object??' for extra details.

ImportError Traceback (most recent call last) /home/pandeya/git/mimclib/ in () ----> 1 import scipy

ImportError: No module named scipy

In [1]:

[anamika168]

what is the solution please ?

[anamika168]

This problem is solved now, I get the following on importing scipy

---

pandeya@kw12521:~/git/mimclib$ ipython -i -c "import scipy" Python 2.7.3 (default, Oct 26 2016, 21:01:49) Type "copyright", "credits" or "license" for more information.

IPython 0.12.1 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object', use 'object??' for extra details. /home/pandeya/.local/lib/python2.7/site-packages/scipy/init.py:111: UserWarning: Numpy 1.7.1 or above is recommended for this version of scipy (detected version 1.6.1) UserWarning)

---

[Virtakuono]

Once you have scipy installed scipy.stats should work too. To double check that it works, I would run ipython -i -c "from scipy.stats import norm" and if all runs smoothly, try to next run the command you started off with.

[anamika168]

it is giving the error ValueError: numpy.dtype has the wrong size, try recompiling. Expected 80, got 88

[Virtakuono]

In that case I would go and try update the Numpy version you have to 1.7.1. or greater, perhaps the warning you encountered earlier is causing all this trouble.

[anamika168]

yes, doing it. It is in the process

[anamika168]

So now on executing ipython -i -c "from scipy.stats import norm"

---

pandeya@kw12521:~$ ipython -i -c "from scipy.stats import norm" Python 2.7.3 (default, Oct 26 2016, 21:01:49) Type "copyright", "credits" or "license" for more information.

IPython 0.12.1 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object', use 'object??' for extra details.

---

[Virtakuono]

Now if you run the original command it should go through, I hope. Or at least it'll be broken in a different way ;)

[anamika168]

No, it is not executing properly. pandeya@kw12521:~/git/mimclib$ python -c 'from mimclib.db import MIMCDatabase; print MIMCDatabase().DBCreationScript();' | mysql ERROR 1045 (28000): Access denied for user 'pandeya'@'localhost' (using password: NO) Traceback (most recent call last): File "", line 1, in AttributeError: 'MIMCDatabase' object has no attribute 'DBCreationScript'

[Virtakuono]

There's clearly a python error and a MySQL error and I am not quite sure which one triggers the other... What happens if you command python -c 'from mimclib.db import MIMCDatabase; print MIMCDatabase().DBCreationScript();' (leave the mysql bit out...) ?

[anamika168]

ok, on doing so. I get the following

---

Traceback (most recent call last): File "", line 1, in AttributeError: 'MIMCDatabase' object has no attribute 'DBCreationScript'

---

[anamika168]

pandeya@kw12521:~/git/mimclib$ python -c 'from mimclib.db import MIMCDatabase; print MIMCDatabase().DBCreationScript();' Traceback (most recent call last): File "", line 1, in AttributeError: 'MIMCDatabase' object has no attribute 'DBCreationScript' pandeya@kw12521:~/git/mimclib$

[Virtakuono]

There is something tricky going on with the mimc installation. I would redo make and make pip, see if there are error messages and if there are none, retry the original command once more. If it does not still go through, I'm starting to be at my wit's end, but you might try to command python -c 'from mimclib.db import MIMCDatabase; print dir(MIMCDatabase())'

[anamika168]

So, I have not only redo make and make pip, even re-cloned and then successfully executed make and make pip with the message Successfully installed mimclib. Still on executing python -c 'from mimclib.db import MIMCDatabase; print MIMCDatabase().DBCreationScript();' | mysql -u pandeya -p the output is

---

Enter password: Traceback (most recent call last): File "", line 1, in AttributeError: 'MIMCDatabase' object has no attribute 'DBCreationScript'

---

and your suggested trial : python -c 'from mimclib.db import MIMCDatabase; print dir(MIMCDatabase());' gives

---

['DBConn', 'DBName', 'class', 'delattr', 'dict', 'doc', 'format', 'getattribute', 'hash', 'init', 'module', 'new', 'reduce', '__reduce_ex', 'repr', 'setattr', 'sizeof', 'str', 'subclasshook', 'weakref__', '_fetchArray', 'connArgs', 'createRun', 'deleteRuns', 'engine', 'getRunsIDs', 'markRunDone', 'markRunFailed', 'markRunSuccessful', 'readRuns', 'readRunsByID', 'writeRunData']

---

[anamika168]

Well, thanks @Virtakuono for your help. I hope someone response to my query.

[haji-ali]

That's the old command. You should do python -c 'from mimclib.db import MySQLDBConn; print MySQLDBConn.DBCreationScript();' | mysql

Or, if you are not planning on doing too many parallel runs, you can use the file based database python -c 'from mimclib.db import SQLiteDBConn; print SQLiteDBConn.DBCreationScript();' | sqlite3 data.sql which will create a database file called 'data.sql' in the current directory. Make sure that you pass '-db_engine sqlite -db_name data.sql' to you mimc commands.

[anamika168]

Hello Ali, thanks for your comment. I tried python -c 'from mimclib.db import MySQLDBConn; print MySQLDBConn.DBCreationScript();' | mysql -u pandeya -p

only mysql doesn't work for me. May be because of my setting. But the above gives me ERROR 1050 (42S01) at line 15: Table 'vw_runs' already exists

[Virtakuono]

@haji-ali for the long term, I think it'd be better to remove all references to such old instructions. Are there still some left in the repo that we know of? @anamika168 where did you get the old command?

[anamika168]

@Virtakuono I agree, it would be better to remove old instruction. I have got these instruction on github @Virtakuono and @haji-ali may someone of you direct me to solve my current problem what I am getting now ERROR 1050 (42S01) at line 15: Table 'vw_runs' already exists

[Virtakuono]

If I am not mistaken, you have already at some stage installed the database and the table already exists. I would try to store data into the database (if you want to use the database in the first place) and see if you can write into it. If yes, you're set to go. If not.... ....well... at least we have a new problem to debug.

[haji-ali]

Yes. You have already created the database. If you do not care about the data in that database then just pass True to DBCreationScript. So: python -c 'from mimclib.db import MySQLDBConn; print MySQLDBConn.DBCreationScript(True);' | mysql -u pandeya -p

WARNING: This will delete any run data that was saved in the database.

[anamika168]

@haji-ali even after passing the True value the same error occurs. Even I have also tried to drop the user pandeya in mysql and just tried by root as follows:

---

pandeya@kw12521:~/git/mimclib$ python -c 'from mimclib.db import MySQLDBConn; print MySQLDBConn.DBCreationScript(True);' | mysql -u root -p Enter password: pandeya@kw12521:~/git/mimclib$ python -c 'from mimclib.db import MySQLDBConn; print MySQLDBConn.DBCreationScript();' | mysql -u root -p Enter password: ERROR 1050 (42S01) at line 15: Table 'vw_runs' already exists pandeya@kw12521:~/git/mimclib$

---

[anamika168]

Moreover, the database is required for parallel run. I have some problem in single-run-example as well. It is not showing me anything other than the following

---

pandeya@kw12521:~/git/mimclib/tests/gbm$ ./single_run_example Doing a single run of mimclib using the GBM example. pandeya@kw12521:~/git/mimclib/tests/gbm$

---

[haji-ali]

It is succeeding after the first command. python -c 'from mimclib.db import MySQLDBConn; print MySQLDBConn.DBCreationScript(True);' | mysql -u root - (You don't need the root user, but that's a different issue). This command creates a new database in the mysql server (after deleting any existing ones). After this you are ready to do the parallel runs.

./single_run_example should run as usual without any database. Can you execute these commands in the gbm folder?

make ./echo_test_cmd.py ./echo_test_cmd.py | /bin/bash

[anamika168]

pandeya@kw12521:~/git/mimclib/tests/gbm$ make ./echo_test_cmd.py ./echo_test_cmd.py | /bin/bash /bin/bash: line 2: unexpected EOF while looking for matching `'' /bin/bash: line 3: syntax error: unexpected end of file

[haji-ali]

Sorry. Can you execute these commands one at a time?

[anamika168]

ok I did that too and got following:

---

pandeya@kw12521:~/git/mimclib/tests/gbm$ make ./echo_test_cmd.py make: Nothing to be done for `echo_test_cmd.py'. pandeya@kw12521:~/git/mimclib/tests/gbm$ ./echo_test_cmd.py | /bin/bash Traceback (most recent call last): File "mimc_run.py", line 71, in import mimclib File "/home/pandeya/git/mimclib/mimclib/init.py", line 3, in from .mimc import MIMCRun, MIMCItrData File "/home/pandeya/git/mimclib/mimclib/mimc.py", line 12, in from scipy.stats import norm File "/usr/local/lib/python2.7/dist-packages/scipy/stats/init.py", line 344, in from .stats import File "/usr/local/lib/python2.7/dist-packages/scipy/stats/stats.py", line 173, in import scipy.special as special File "/usr/local/lib/python2.7/dist-packages/scipy/special/init.py", line 636, in from ._ufuncs import File "init.pxd", line 155, in init scipy.special._ufuncs (scipy/special/_ufuncs.c:36300) RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 pandeya@kw12521:~/git/mimclib/tests/gbm$

---

[haji-ali]

See here http://stackoverflow.com/questions/40845304/runtimewarning-numpy-dtype-size-changed-may-indicate-binary-incompatibility

[anamika168]

ok, so now single run works, but parallel run still has some problem. The error message is sqlite3.OperationalError: table vw_runs already exists

[anamika168]

Hi Ali, any comments on parallel_run_example

[haji-ali]

You are using sqlite (or running the script as is, which uses sqlite by default). This error usually occurs because multiple instances are trying to create the database file (since it was not created before) and a race condition happens.

Long story: Assuming the file exists (I believe it's mimc.sql), running the command again should succeed.