davidljohnson
commented
7 years ago I also tried recreating my taxiiauth and taxiipersist databases after updating mariadb to the new version, but no dice...
Closed davidljohnson closed 7 years ago
davidljohnson
commented
7 years ago I also tried recreating my taxiiauth and taxiipersist databases after updating mariadb to the new version, but no dice...
FloatingGhost
commented
7 years ago You will probably chastise me for using python 2.7
you're damn right I will
use real python!
that's how my environment was initially set up when I installed MISP.
you can run multiple python versions at once y'know
Try installing under py3 and see if you get anything good - this error was certainly an old mysql thingy, idk why you're getting it under 10.1.22
davidljohnson
commented
7 years ago Thanks for the feedback -- unfortunately, I'm still receiving the same error. But at least now, it feels good to be using a version of python that's actually still supported. I went ahead and rebuilt the taxiiauth an taxiipersist databases and performed a 'sudo apt-get update && sudo apt-get upgrade' to ensure I'm using the latest versions of my software packages. Also, when I built this using python3, I had to install mysqldb via the command 'sudo apt-get install python3-mysqldb'. Not sure what is wrong. :(
/var/git/MISP-Taxii-Server/config$ opentaxii-create-services -c config/services.yaml Traceback (most recent call last): File "/usr/local/lib/python3.5/dist-packages/SQLAlchemy-1.1.9-py3.5-linux-x86_64.egg/sqlalchemy/engine/base.py", line 1182, in _execute_context context) File "/usr/local/lib/python3.5/dist-packages/SQLAlchemy-1.1.9-py3.5-linux-x86_64.egg/sqlalchemy/engine/default.py", line 470, in do_execute cursor.execute(statement, parameters) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 226, in execute self.errorhandler(self, exc, value) File "/usr/lib/python3/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler raise errorvalue File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 217, in execute res = self._query(query) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 378, in _query rowcount = self._do_query(q) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 341, in _do_query db.query(q) File "/usr/lib/python3/dist-packages/MySQLdb/connections.py", line 280, in query _mysql.connection.query(self, query) _mysql_exceptions.OperationalError: (1071, 'Specified key was too long; max key length is 767 bytes')
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/bin/opentaxii-create-services", line 9, in
FloatingGhost
commented
7 years ago It's a long shot, but can you try running
SET @@global.innodb_large_prefix = 1;
on your sql server?
davidljohnson
commented
7 years ago Here's some information on the character sets of my tables.
MariaDB [(none)]> SELECT default_character_set_name FROM information_schema.SCHEMATA S WHERE schema_name = "taxiipersist"; +----------------------------+ | default_character_set_name | +----------------------------+ | utf8mb4 | +----------------------------+ 1 row in set (0.00 sec)
MariaDB [(none)]> SELECT default_character_set_name FROM information_schema.SCHEMATA S WHERE schema_name = "taxiiauth"; +----------------------------+ | default_character_set_name | +----------------------------+ | utf8mb4 | +----------------------------+ 1 row in set (0.00 sec)
davidljohnson
commented
7 years ago It's a long shot, but can you try running
SET @@global.innodb_large_prefix = 1;
No dice :(
davidljohnson
commented
7 years ago Do I need to change this to a 3-byte character set? I think utf8mb4 is 4 bytes. I'll give it a shot and update this thread accordingly.
davidljohnson
commented
7 years ago Dang! That didn't work either. Both databases are in 'utf8', but still receiving the same error.
FloatingGhost
commented
7 years ago Do dee dooooo!
ALTER DATABASE taxiipersist CHARACTER SET latin1 COLLATE latin1_general_ci;
ALTER DATABASE taxiiauth CHARACTER SET latin1 COLLATE latin1_general_ci;run that!
davidljohnson
commented
7 years ago Thank God you figured that out. Now I'm getting a different error:
/var/git/MISP-Taxii-Server/config$ opentaxii-create-services -c config/services.yaml
2017-04-05T15:02:59.792149Z [opentaxii.utils] info: api.initialized {api=opentaxii.persistence.sqldb.SQLDatabaseAPI, event=api.initialized, logger=opentaxii.utils, level=info, timestamp=2017-04-05T15:02:59.792149Z}
2017-04-05T15:02:59.832473Z [opentaxii.utils] info: api.initialized {api=opentaxii.auth.sqldb.SQLDatabaseAPI, event=api.initialized, logger=opentaxii.utils, level=info, timestamp=2017-04-05T15:02:59.832473Z}
Traceback (most recent call last):
File "/usr/local/bin/opentaxii-create-services", line 9, in
FloatingGhost
commented
7 years ago ah, that one's easy! You didn't run sudo python3 setup.py install from the MISP-Taxii-Server folder!
davidljohnson
commented
7 years ago Great, that worked! I guess I was confused about the order of the instructions and thought I needed to run these commands before running sudo python3 setup.py install in the parent directory.
opentaxii-create-services -c config/services.yaml opentaxii-create-collections -c config/collections.yaml opentaxii-create-account -u root -p root.
Now I'm getting a certificate verification failure with my MISP instance, but that's a different issue that I should be able to resolve myself (hopefully).
Thanks very much!
FloatingGhost
commented
6 years ago pymisp.exceptions.PyMISPError: Unable to connect to MISP (http://192.168.1.106:9000). Please make sure the API key and the URL are correct (http/https is required): HTTPConnectionPool(host='192.168.1.106', port=9000):
Seriously!? You're asking for support on an issue that explains itself so very clearly?
Jesus merciful christ end my suffering.
It's saying that it can't connect to MISP on port 9000 on that address, with the API key you provided.
Config should be
misp:
url: WHEREVER YOUR MISP INSTANCE IS LOCATED
api: WHATEVER YOUR API KEY IS ON THAT INSTANCE
mcamachotw
commented
6 years ago @FloatingGhost I understand, thank you for your quickly reply.
I had no problems following the instructions until I tried the 'opentaxii-create-services -c services.yaml' command. You will probably chastise me for using python 2.7, but that's how my environment was initially set up when I installed MISP. I looked at the other closed topic on this issue and updated mariadb accordingly to the latest version (10.1.22). Unfortunately, the problem persists:
$ opentaxii-create-services -c services.yaml
/usr/local/lib/python2.7/dist-packages/SQLAlchemy-1.1.9-py2.7-linux-x86_64.egg/sqlalchemy/engine/default.py:470: Warning: Specified key was too long; max key length is 767 bytes 2017-04-04T20:49:53.864728Z [opentaxii.utils] info: api.initialized {timestamp=2017-04-04T20:49:53.864728Z, logger=opentaxii.utils, api=opentaxii.persistence.sqldb.SQLDatabaseAPI, event=api.initialized, level=info} Traceback (most recent call last): File "/usr/local/bin/opentaxii-create-services", line 9, in
load_entry_point('opentaxii==0.1.9a1', 'console_scripts', 'opentaxii-create-services')()
File "/usr/lib/python2.7/dist-packages/pkg_resources/init.py", line 542, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/usr/lib/python2.7/dist-packages/pkg_resources/init.py", line 2569, in load_entry_point
return ep.load()
File "/usr/lib/python2.7/dist-packages/pkg_resources/init.py", line 2229, in load
return self.resolve()
File "/usr/lib/python2.7/dist-packages/pkg_resources/init.py", line 2235, in resolve
module = import(self.module_name, fromlist=['name'], level=0)
File "/usr/local/lib/python2.7/dist-packages/opentaxii-0.1.9a1-py2.7.egg/opentaxii/cli/init.py", line 10, in
server = TAXIIServer(config)
File "/usr/local/lib/python2.7/dist-packages/opentaxii-0.1.9a1-py2.7.egg/opentaxii/server.py", line 41, in init
api=initialize_api(config['auth_api']))
File "/usr/local/lib/python2.7/dist-packages/opentaxii-0.1.9a1-py2.7.egg/opentaxii/utils.py", line 36, in initialize_api
instance = cls(**params)
File "/usr/local/lib/python2.7/dist-packages/opentaxii-0.1.9a1-py2.7.egg/opentaxii/auth/sqldb/api.py", line 41, in init
self.db.create_all_tables()
File "/usr/local/lib/python2.7/dist-packages/opentaxii-0.1.9a1-py2.7.egg/opentaxii/sqldb_helper.py", line 60, in create_all_tables
self.metadata.create_all(bind=self.engine)
File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/schema.py", line 3885, in create_all
File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1929, in _run_visitor
File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1538, in _run_visitor
File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/visitors.py", line 121, in traverse_single
File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/ddl.py", line 733, in visit_metadata
File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/visitors.py", line 121, in traverse_single
File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/ddl.py", line 767, in visit_table
File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 945, in execute
File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/ddl.py", line 68, in _execute_on_connection
File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1002, in _execute_ddl
File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1189, in _execute_context
File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1402, in _handle_dbapi_exception
File "build/bdist.linux-x86_64/egg/sqlalchemy/util/compat.py", line 203, in raise_from_cause
File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1182, in _execute_context
File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/default.py", line 470, in do_execute
File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 226, in execute
self.errorhandler(self, exc, value)
File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorvalue
sqlalchemy.exc.OperationalError: (_mysql_exceptions.OperationalError) (1071, 'Specified key was too long; max key length is 767 bytes') [SQL: u'\nCREATE TABLE accounts (\n\tid INTEGER NOT NULL AUTO_INCREMENT, \n\tusername VARCHAR(256), \n\tpassword_hash VARCHAR(256), \n\tPRIMARY KEY (id), \n\tUNIQUE (username)\n)\n\n']
The output of my services.yaml file is as follows:
$ cat config.yaml
domain: "localhost:9000" support_basic_auth: yes
persistence_api: class: opentaxii.persistence.sqldb.SQLDatabaseAPI parameters: db_connection: mysql://taxii:[PASSWORD]@localhost/taxiipersist create_tables: yes
auth_api: class: opentaxii.auth.sqldb.SQLDatabaseAPI parameters: db_connection: mysql://taxii:[PASSWORD]@localhost/taxiiauth create_tables: yes secret: [Random secret string]
logging: opentaxii: info root: info
hooks: misp_taxii_hooks.hooks
zmq: host: localhost port: 50000
misp: url: "https://[FQDN of local MISP instance]" api: [API key]