unable to import from keepass file (.kdb)

[hothouse-github]

Hi RatticDB Team,

Have tried importing one of our keepass databases (which is in a kbdx format). First task was to export to a kdb format (as it doesn't seem to support kdbx). Validated that the exported file was valid (using keepass 1.27).

However when importing into rattic the following error is thrown (via a 500 internal server error in apache2):

2014-09-12 12:58:00,002 [ERROR] Internal Server Error: /staff/import/keepass/ Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 199, in get_response response = middleware_method(request, response) File "/usr/local/lib/python2.7/dist-packages/user_sessions/middleware.py", line 46, in process_response request.session.save() File "/usr/local/lib/python2.7/dist-packages/user_sessions/backends/db.py", line 69, in save session_data=self.encode(self._get_session(no_load=must_create)), File "/usr/local/lib/python2.7/dist-packages/django/contrib/sessions/backends/base.py", line 87, in encode serialized = self.serializer().dumps(session_dict) File "/usr/local/lib/python2.7/dist-packages/django/core/signing.py", line 88, in dumps return json.dumps(obj, separators=(',', ':')).encode('latin-1') File "/usr/lib/python2.7/json/init.py", line 238, in dumps _kw).encode(obj) File "/usr/lib/python2.7/json/encoder.py", line 200, in encode chunks = self.iterencode(o, _one_shot=True) File "/usr/lib/python2.7/json/encoder.py", line 263, in iterencode return _iterencode(o, 0) UnicodeDecodeError: 'utf8' codec can't decode byte 0xb4 in position 0: invalid start byte [client 192.168.X.X] mod_wsgi (pid=1287): Exception occurred processing WSGI script '/home/ratticdb/application/RatticWeb/ratticweb/wsgi.py'. [client 192.168.X.X] Traceback (most recent call last): [client 192.168.X.X] File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/wsgi.py", line 206, in call [client 192.168.X.X] response = self.get_response(request) [client 192.168.X.X] File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 203, in get_response [client 192.168.X.X] response = self.handle_uncaught_exception(request, resolver, sys.exc_info()) [client 192.168.X.X] File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 236, in handle_uncaught_exception [client 192.168.X.X] return callback(request, _param_dict) [client 192.168.X.X] File "/home/ratticdb/application/RatticWeb/ratticweb/views.py", line 14, in handle500 [client 192.168.X.X] return render(request, '500.html', status=500) [client 192.168.X.X] File "/usr/local/lib/python2.7/dist-packages/django/shortcuts/init.py", line 53, in render [client 192.168.X.X] return HttpResponse(loader.render_to_string(_args, *_kwargs), [client 192.168.X.X] File "/usr/local/lib/python2.7/dist-packages/django/template/loader.py", line 162, in render_to_string [client 192.168.X.X] t = get_template(template_name) [client 192.168.X.X] File "/usr/local/lib/python2.7/dist-packages/django/template/loader.py", line 138, in get_template [client 192.168.X.X] template, origin = find_template(template_name) [client 192.168.X.X] File "/usr/local/lib/python2.7/dist-packages/django/template/loader.py", line 127, in find_template [client 192.168.X.X] source, display_name = loader(name, dirs) [client 192.168.X.X] File "/usr/local/lib/python2.7/dist-packages/django/template/loader.py", line 43, in call [client 192.168.X.X] return self.load_template(template_name, template_dirs) [client 192.168.X.X] File "/usr/local/lib/python2.7/dist-packages/django/template/loader.py", line 49, in load_template [client 192.168.X.X] template = get_template_from_string(source, origin, template_name) [client 192.168.X.X] File "/usr/local/lib/python2.7/dist-packages/django/template/loader.py", line 149, in get_template_from_string [client 192.168.X.X] return Template(source, origin, name) [client 192.168.X.X] File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 125, in init [client 192.168.X.X] self.nodelist = compile_string(template_string, origin) [client 192.168.X.X] File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 153, in compile_string [client 192.168.X.X] return parser.parse() [client 192.168.X.X] File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 278, in parse [client 192.168.X.X] compiled_result = compile_func(self, token) [client 192.168.X.X] File "/usr/local/lib/python2.7/dist-packages/django/template/loader_tags.py", line 215, in do_extends [client 192.168.X.X] nodelist = parser.parse() [client 192.168.X.X] File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 278, in parse [client 192.168.X.X] compiled_result = compile_func(self, token) [client 192.168.X.X] File "/usr/local/lib/python2.7/dist-packages/django/template/loader_tags.py", line 190, in do_block [client 192.168.X.X] nodelist = parser.parse(('endblock',)) [client 192.168.X.X] File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 276, in parse [client 192.168.X.X] self.invalid_block_tag(token, command, parse_until) [client 192.168.X.X] File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 331, in invalid_block_tag [client 192.168.X.X](command, get_text_list%28["'%s'" % p for p in parse_until]%29)) [client 192.168.X.X] TemplateSyntaxError: Invalid block tag: 'trans', expected 'endblock'

Thoughts ?

[smarthall]

So it looks like there are two issues here. The first is that RatticDB has had an issue with some unicode inside your Keepass file. This is probably a mistake in the code somewhere that assumes a string coming from Keepass is ascii not unicode.

There is also a second issue here. RatticDB tried to load an error page and it encountered a syntax error. I've found that and fixed it already. :smile:

What version of RatticDB are you running?

[hothouse-github]

Hi Dan,

We are running a copy of v1.2.1.

[smarthall]

Hrmmm, it seems that there is some unicode in the file that RatticDB doesn't like. It looks like there is an incorrectly encoded backtick. I'll take a shot at writing something that still imports, but throws away invalid characters. Hopefully this will resolve this issue.

[smarthall]

I've had a few reports of this, and most people seem to be using MySQL. Would you be send me the following:

• The MySQL version that you are running. Use SHOW VARIABLES LIKE "%version%";
• The output of SHOW CREATE TABLE cred_cred;

[hothouse-github]

Hey Dan,

mysql> SHOW VARIABLES LIKE "%version%"; 
+-------------------------+------------------+
| Variable_name           | Value            |
+-------------------------+------------------+
| innodb_version          | 5.5.38           |
| protocol_version        | 10               |
| slave_type_conversions  |                  |
| version                 | 5.5.38-0+wheezy1 |
| version_comment         | (Debian)         |
| version_compile_machine | x86_64           |
| version_compile_os      | debian-linux-gnu |
+-------------------------+------------------+
7 rows in set (0.00 sec)

And

SHOW CREATE TABLE cred_cred; 
+-----------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table     | Create Table                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
+-----------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| cred_cred | CREATE TABLE `cred_cred` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `title` varchar(64) NOT NULL,
  `username` varchar(250) DEFAULT NULL,
  `password` varchar(250),
  `description` longtext,
  `group_id` int(11) NOT NULL,
  `is_deleted` tinyint(1) NOT NULL,
  `url` varchar(200),
  `latest_id` int(11),
  `created` datetime NOT NULL,
  `iconname` varchar(64) NOT NULL,
  `descriptionmarkdown` tinyint(1) NOT NULL,
  `modified` datetime NOT NULL,
  `attachment` varchar(100),
  `attachment_name` varchar(64),
  PRIMARY KEY (`id`),
  KEY `cred_cred_5f412f9a` (`group_id`),
  KEY `cred_cred_1ef0d1ab` (`latest_id`),
  KEY `cred_cred_c379dc61` (`url`),
  KEY `cred_cred_9246ed76` (`title`),
  KEY `cred_cred_5951b9ef` (`is_deleted`),
  KEY `cred_cred_ec9ad377` (`modified`),
  KEY `cred_cred_ee0cafa2` (`username`),
  CONSTRAINT `group_id_refs_id_b5bfbd26` FOREIGN KEY (`group_id`) REFERENCES `auth_group` (`id`),
  CONSTRAINT `latest_id_refs_id_2656bdfd` FOREIGN KEY (`latest_id`) REFERENCES `cred_cred` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=5 DEFAULT CHARSET=utf8 |
+-----------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

Hopefully this helps :)

[enuno]

I am having the same issue. Is there a fix for this?

[ghost]

@smarthall I've just run into this problem and found out that Rattic uses session to pass file data as a python object(?). The problem is Django >1.6 uses JSONSerializer as the default and it doesn't support that kind of complex types. So, by forcing ratticweb app's SESSION_SERIALIZER to PickleSerializer, I've fixed my problem.

tl;dr Add this line to ratticweb/settings.py and everything will be fine: SESSION_SERIALIZER = 'django.contrib.sessions.serializers.PickleSerializer'

[iddqd2]

Using v1.3.1. It's already have that line in settings.py. Import with cyrrilic still fails with the following error: UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-12: ordinal not in range(128)

[jasonmarshall1]

I encountered the 500 server error and was able to import the keepass 2.34 file into Rattic by following the following process:
1) create keepass files one for each folder under the root directory in keepass. When I tried to export/import a keepass file with subdirectories/folders setup in keepass, the import puked. 2) export each file using V1 option for each directory/folder in keepass 3) this tripped me up, but use the finder feature to navigate to the export directory. When I typed in the folder and file name, it failed. I'm not sure what is different, but using the finder to locate the directory and then typing in the export file name seemed to work. My file extension was .kdb 4) I used the key file (yourKeePassFile.key) key value when prompted for the password during the import in rattic, not the user supplied pwd.

I hope this helps someone else.