Closed forgettableEgg closed 5 years ago
FloatingGhost
commented
5 years ago This is probably because the version of opentaxii used before today is old now, and may have been buggy (it was an alpha release)
Try updating using the installation instructions in the README now and see if it fixes the issue
forgettableEgg
commented
5 years ago Gotcha will try again now thanks!
forgettableEgg
commented
5 years ago Not sure if I missed a step but I don't see the command opentaxii-sync-data on my system.
forgettableEgg
commented
5 years ago Good ole reboot fixed it sorry about that!
forgettableEgg
commented
5 years ago So the database is not populating after running the opentaxii-sync-data command. This is the output I get from the command
2019-02-13T21:12:43.870737Z [opentaxii.utils] info: api.initialized {api=opentax ii.persistence.sqldb.SQLDatabaseAPI} 2019-02-13T21:12:43.885873Z [opentaxii.utils] info: api.initialized {api=opentax ii.auth.sqldb.SQLDatabaseAPI} 2019-02-13T21:12:43.886072Z [opentaxii.server] info: opentaxii.server_configured 2019-02-13T21:12:43.891973Z [anyconfig] info: Loading: config/data-configuration .yaml 2019-02-13T21:12:43.911204Z [opentaxii.utils] info: sync_services.updated {id=in box} 2019-02-13T21:12:43.913105Z [opentaxii.utils] info: sync_services.updated {id=di scovery} 2019-02-13T21:12:43.915034Z [opentaxii.utils] info: sync_services.updated {id=co llection_management} 2019-02-13T21:12:43.916955Z [opentaxii.utils] info: sync_services.updated {id=po ll} 2019-02-13T21:12:43.917103Z [opentaxii.utils] info: sync_services.stats {updated =4, created=0, deleted=0} 2019-02-13T21:12:43.925468Z [opentaxii.utils] info: sync_collections.updated {na me=my_collection, id=1} 2019-02-13T21:12:43.925605Z [opentaxii.utils] info: sync_collections.stats {upda ted=1, created=0, disabled=0, deleted=0} 2019-02-13T21:12:43.975013Z [opentaxii.utils] info: sync_accounts.updated {usern ame=guest} 2019-02-13T21:12:44.038448Z [opentaxii.utils] info: sync_accounts.updated {usern ame=admin} 2019-02-13T21:12:44.038659Z [opentaxii.utils] info: sync_accounts.stats {updated =2, created=0, deleted=0}
FloatingGhost
commented
5 years ago That literally says that it worked
sync_collections.stats {updated=1, created=0, disabled=0, deleted=0}
sync_services.stats {updated =4, created=0, deleted=0}
sync_accounts.stats {updated =2, created=0, deleted=0}I know it says it worked but if you look below
MariaDB [(none)]> use taxiipersist; Database changed MariaDB [taxiipersist]> show tables; Empty set (0.00 sec) MariaDB [taxiipersist]>
I understand that the output means it worked but as I mentioned before it still doesn't actually create the tables. In the directions you mention using the database I created for MISP. I went back into the MISP documentation and copied this command to install the database.
sudo apt-get install mariadb-client mariadb-server sudo mysql_secure_installation
Then followed the rest of your guide, I am hoping I missed something really simple.
FloatingGhost
commented
5 years ago Gonna bet everything I have that you didn't re-export OPENTAXII_CONFIG after you rebooted, leading to opentaxii using a default sqlite setup
forgettableEgg
commented
5 years ago I did in fact have to re-export them because you were right after reboot they disappeared but now I went back to the original error. After running the commands from the beginning I get this again. When I run opentaxii-run-dev without sudo I get
Traceback (most recent call last):
File "/home/taxii/.local/lib/python3.6/site-packages/sqlalchemy/engine/base.py ", line 1236, in _execute_context
cursor, statement, parameters, context
File "/home/taxii/.local/lib/python3.6/site-packages/sqlalchemy/engine/default .py", line 536, in do_execute
cursor.execute(statement, parameters)
File "/home/taxii/.local/lib/python3.6/site-packages/MySQLdb/cursors.py", line 206, in execute
res = self._query(query)
File "/home/taxii/.local/lib/python3.6/site-packages/MySQLdb/cursors.py", line 312, in _query
db.query(q)
File "/home/taxii/.local/lib/python3.6/site-packages/MySQLdb/connections.py", line 224, in query
_mysql.connection.query(self, query)
MySQLdb._exceptions.OperationalError: (1005, 'Can\'t create table taxiipersist .service_to_collection (errno: 150 "Foreign key constraint is incorrectly form ed")')
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/taxii/.local/bin/opentaxii-run-dev", line 7, in taxiipersist.service_to_collection (errno: 150 "Foreign key constraint is incorrectly formed")') [SQL: '\nCREATE TABLE service_to_collec tion (\n\tservice_id VARCHAR(150) NOT NULL, \n\tcollection_id INTEGER NOT NULL, \n\tPRIMARY KEY (service_id, collection_id), \n\tFOREIGN KEY(service_id) REFEREN CES services (id) ON DELETE CASCADE, \n\tFOREIGN KEY(collection_id) REFERENCES d ata_collections (id) ON DELETE CASCADE\n)\n\n'] (Background on this error at: ht tp://sqlalche.me/e/e3q8)
`
cohenshimi
commented
4 years ago same issue . any solution ?
raitomx
commented
4 years ago same issue here, help please!!!
Followed the instructions to the letter on a new ubuntu 18.04 box. Got to the database table creation steps and I get this error for the first call. Tried debugging all day but got nothing please tell me its a stupid mistake so I can this working. I get the same error for all 3 commands right after each other.
My files are all default settings with the passwords and such changed. mysql version
mysql Ver 15.1 Distrib 10.1.34-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2Python 3.6 as shown below. If I run with sudo over a bunch of the commands it seems to work but the tables still aren't created.
2019-02-05T22:14:06.792359Z [opentaxii.utils] info: api.initialized {api=opentaxii.persistence.sqldb.SQLDatabaseAPI, event=api.initialized, logger=opentaxii.utils, level=info, timestamp=2019-02-05T22:14:06.792359Z} 2019-02-05T22:14:06.871060Z [opentaxii.utils] info: api.initialized {api=opentaxii.auth.sqldb.SQLDatabaseAPI, event=api.initialized, logger=opentaxii.utils, level=info, timestamp=2019-02-05T22:14:06.871060Z} 2019-02-05T22:14:06.871466Z [opentaxii.server] info: opentaxii.server_configured {event=opentaxii.server_configured, logger=opentaxii.server, level=info, timestamp=2019-02-05T22:14:06.871466Z} Loading: config/collections.yaml 2019-02-05T22:14:06.936437Z [opentaxii.persistence.manager] info: collection.created {collection=collection, event=collection.created, logger=opentaxii.persistence.manager, level=info, timestamp=2019-02-05T22:14:06.936437Z}Thanks for the assistanceTraceback (most recent call last): File "/usr/local/lib/python3.6/dist-packages/SQLAlchemy-1.3.0b2-py3.6-linux-x86_64.egg/sqlalchemy/engine/base.py", line 1244, in _execute_context cursor, statement, parameters, context File "/usr/local/lib/python3.6/dist-packages/SQLAlchemy-1.3.0b2-py3.6-linux-x86_64.egg/sqlalchemy/engine/default.py", line 546, in do_execute cursor.execute(statement, parameters) File "/home/sigmanj/.local/lib/python3.6/site-packages/MySQLdb/cursors.py", line 198, in execute res = self._query(query) File "/home/sigmanj/.local/lib/python3.6/site-packages/MySQLdb/cursors.py", line 304, in _query db.query(q) File "/home/sigmanj/.local/lib/python3.6/site-packages/MySQLdb/connections.py", line 217, in query _mysql.connection.query(self, query) MySQLdb._exceptions.OperationalError: (1005, 'Can\'t create tabletaxiipersist.service_to_collection` (errno: 150 "Foreign key constraint is incorrectly formed")')The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/usr/local/bin/opentaxii-create-services", line 11, in
load_entry_point('opentaxii==0.1.10a1', 'console_scripts', 'opentaxii-create-services')()
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 480, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 2693, in load_entry_point
return ep.load()
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 2324, in load
return self.resolve()
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 2330, in resolve
module = import(self.module_name, fromlist=['name'], level=0)
File "/usr/local/lib/python3.6/dist-packages/opentaxii-0.1.10a1-py3.6.egg/opentaxii/cli/init.py", line 10, in
server = TAXIIServer(config)
File "/usr/local/lib/python3.6/dist-packages/opentaxii-0.1.10a1-py3.6.egg/opentaxii/server.py", line 38, in init
server=self, api=initialize_api(config['persistence_api']))
File "/usr/local/lib/python3.6/dist-packages/opentaxii-0.1.10a1-py3.6.egg/opentaxii/utils.py", line 36, in initialize_api
instance = cls(params)
File "/usr/local/lib/python3.6/dist-packages/opentaxii-0.1.10a1-py3.6.egg/opentaxii/persistence/sqldb/api.py", line 43, in init
self.db.create_all_tables()
File "/usr/local/lib/python3.6/dist-packages/opentaxii-0.1.10a1-py3.6.egg/opentaxii/sqldb_helper.py", line 60, in create_all_tables
self.metadata.create_all(bind=self.engine)
File "/usr/local/lib/python3.6/dist-packages/SQLAlchemy-1.3.0b2-py3.6-linux-x86_64.egg/sqlalchemy/sql/schema.py", line 4230, in create_all
ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
File "/usr/local/lib/python3.6/dist-packages/SQLAlchemy-1.3.0b2-py3.6-linux-x86_64.egg/sqlalchemy/engine/base.py", line 2033, in _run_visitor
conn._run_visitor(visitorcallable, element, kwargs)
File "/usr/local/lib/python3.6/dist-packages/SQLAlchemy-1.3.0b2-py3.6-linux-x86_64.egg/sqlalchemy/engine/base.py", line 1607, in _run_visitor
visitorcallable(self.dialect, self, kwargs).traverse_single(element)
File "/usr/local/lib/python3.6/dist-packages/SQLAlchemy-1.3.0b2-py3.6-linux-x86_64.egg/sqlalchemy/sql/visitors.py", line 131, in traverse_single
return meth(obj, kw)
File "/usr/local/lib/python3.6/dist-packages/SQLAlchemy-1.3.0b2-py3.6-linux-x86_64.egg/sqlalchemy/sql/ddl.py", line 781, in visit_metadata
_is_metadata_operation=True,
File "/usr/local/lib/python3.6/dist-packages/SQLAlchemy-1.3.0b2-py3.6-linux-x86_64.egg/sqlalchemy/sql/visitors.py", line 131, in traverse_single
return meth(obj, **kw)
File "/usr/local/lib/python3.6/dist-packages/SQLAlchemy-1.3.0b2-py3.6-linux-x86_64.egg/sqlalchemy/sql/ddl.py", line 826, in visit_table
include_foreign_key_constraints,
File "/usr/local/lib/python3.6/dist-packages/SQLAlchemy-1.3.0b2-py3.6-linux-x86_64.egg/sqlalchemy/engine/base.py", line 988, in execute
return meth(self, multiparams, params)
File "/usr/local/lib/python3.6/dist-packages/SQLAlchemy-1.3.0b2-py3.6-linux-x86_64.egg/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
return connection._execute_ddl(self, multiparams, params)
File "/usr/local/lib/python3.6/dist-packages/SQLAlchemy-1.3.0b2-py3.6-linux-x86_64.egg/sqlalchemy/engine/base.py", line 1050, in _execute_ddl
compiled,
File "/usr/local/lib/python3.6/dist-packages/SQLAlchemy-1.3.0b2-py3.6-linux-x86_64.egg/sqlalchemy/engine/base.py", line 1248, in _execute_context
e, statement, parameters, cursor, context
File "/usr/local/lib/python3.6/dist-packages/SQLAlchemy-1.3.0b2-py3.6-linux-x86_64.egg/sqlalchemy/engine/base.py", line 1466, in _handle_dbapi_exception
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "/usr/local/lib/python3.6/dist-packages/SQLAlchemy-1.3.0b2-py3.6-linux-x86_64.egg/sqlalchemy/util/compat.py", line 374, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python3.6/dist-packages/SQLAlchemy-1.3.0b2-py3.6-linux-x86_64.egg/sqlalchemy/util/compat.py", line 128, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/dist-packages/SQLAlchemy-1.3.0b2-py3.6-linux-x86_64.egg/sqlalchemy/engine/base.py", line 1244, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.6/dist-packages/SQLAlchemy-1.3.0b2-py3.6-linux-x86_64.egg/sqlalchemy/engine/default.py", line 546, in do_execute
cursor.execute(statement, parameters)
File "/home/sigmanj/.local/lib/python3.6/site-packages/MySQLdb/cursors.py", line 198, in execute
res = self._query(query)
File "/home/sigmanj/.local/lib/python3.6/site-packages/MySQLdb/cursors.py", line 304, in _query
db.query(q)
File "/home/sigmanj/.local/lib/python3.6/site-packages/MySQLdb/connections.py", line 217, in query
_mysql.connection.query(self, query)
sqlalchemy.exc.OperationalError: (MySQLdb._exceptions.OperationalError) (1005, 'Can\'t create table
taxiipersist.service_to_collection(errno: 150 "Foreign key constraint is incorrectly formed")') [SQL: '\nCREATE TABLE service_to_collection (\n\tservice_id VARCHAR(150) NOT NULL, \n\tcollection_id INTEGER NOT NULL, \n\tPRIMARY KEY (service_id, collection_id), \n\tFOREIGN KEY(service_id) REFERENCES services (id), \n\tFOREIGN KEY(collection_id) REFERENCES data_collections (id)\n)\n\n'] (Background on this error at: http://sqlalche.me/e/e3q8) `