search

libkeepass / pykeepass

Python library to interact with keepass databases (supports KDBX3 and KDBX4)
https://pypi.org/project/pykeepass/
GNU General Public License v3.0
403 stars 96 forks source link

All fields of items created in GNOME Secrets are hidden #376

Closed julianfairfax closed 4 months ago

julianfairfax commented 4 months ago

See https://github.com/aivanovski/keepassvault/issues/208. All fields of items created in GNOME Secrets are shown as hidden in apps such as KeePassVault. Here is a test file: Test.zip. cc @aivanovski @A6GibKm

A6GibKm commented 4 months ago

Could you please create an entry both in Secrets and another app (e.g. keepassxc) with the same fields and same values and dump the contents of said entry (in xml format) of both?

julianfairfax commented 4 months ago

Here: Tests.zip

I can already tell you what the problem is though: Secrets sets ProtectInMemory to True for all fields.

How Secrets looks:

<String>
                        <Key>Notes</Key>
                        <Value ProtectInMemory="True">test</Value>
                    </String>
                    <String>
                        <Key>Password</Key>
                        <Value ProtectInMemory="True">test</Value>
                    </String>
                    <String>
                        <Key>Title</Key>
                        <Value ProtectInMemory="True">Test</Value>
                    </String>
                    <String>
                        <Key>URL</Key>
                        <Value ProtectInMemory="True">https://example.com</Value>
                    </String>
                    <String>
                        <Key>UserName</Key>
                        <Value ProtectInMemory="True">test@example.com</Value>
                    </String>

How it's supposed to look:

<String>
                        <Key>Notes</Key>
                        <Value>test</Value>
                    </String>
                    <String>
                        <Key>Password</Key>
                        <Value ProtectInMemory="True">test</Value>
                    </String>
                    <String>
                        <Key>Title</Key>
                        <Value>Test</Value>
                    </String>
                    <String>
                        <Key>URL</Key>
                        <Value>https://example.com</Value>
                    </String>
                    <String>
                        <Key>UserName</Key>
                        <Value>test@example.com</Value>
                    </String>
A6GibKm commented 4 months ago

What suggests it should look like that? I couldn't find the docs for this protectmemory attribute.

Evidlo commented 4 months ago

Should this be moved to the Secrets issues? ProtectInMemory in not something that is set in PyKeePass.

A6GibKm commented 4 months ago

Its not something I set either afaik, how would I even begin doing it?

I mean, if that's the case then sure.

julianfairfax commented 4 months ago

What suggests it should look like that? I couldn't find the docs for this protectmemory attribute.

The fact that this attribute determines if a field is hidden or not, and only the password field should be hidden.

A6GibKm commented 4 months ago

What suggests it should look like that? I couldn't find the docs for this protectmemory attribute.

The fact that this attribute determines if a field is hidden or not, and only the password field should be hidden.

Where does it say that it should be hidden? Without extra information , this might be a stylistic choice made on the other apps.

julianfairfax commented 4 months ago

What suggests it should look like that? I couldn't find the docs for this protectmemory attribute.

The fact that this attribute determines if a field is hidden or not, and only the password field should be hidden.

Where does it say that it should be hidden? Without extra information , this might be a stylistic choice made on the other apps.

OFC it's a stylistic choice, but these other apps (at least KeePassVault) uses this attribute to determine which fields to hide. GNOME Secrets seems to make the technical choice of adding it to all fields for some reason. cc @aivanovski

A6GibKm commented 4 months ago

We dont use any internals of pykeepass, @Evidlo is there any public api to set this?

Evidlo commented 4 months ago

There's no API for this attribute. I haven't seen ProtectInMemory before, only Protected.

Here's what a newly created entry looks like.

>>> from pykeepass import PyKeePass; kp = PyKeePass('test4.kdbx', 'password', 'test4.key')

>>> g = kp.groups[3]

>>> kp.add_entry(g, 'banana_entry', 'user', 'pass')
Entry: "foobar_group/subgroup/subgroup2/banana_entry (user)"

>>> kp.dump_xml('/tmp/dump.xml')
            <Entry>
              <UUID>KjVPGNbWEe6C170xALbYkw==</UUID>
              <Times>
                <CreationTime>hSxy3Q4AAAA=</CreationTime>
                <LastModificationTime>hSxy3Q4AAAA=</LastModificationTime>
                <LastAccessTime>hSxy3Q4AAAA=</LastAccessTime>
                <ExpiryTime>hSxy3Q4AAAA=</ExpiryTime>
                <Expires>False</Expires>
                <UsageCount>0</UsageCount>
                <LocationChanged>hSxy3Q4AAAA=</LocationChanged>
              </Times>
              <String>
                <Key>Title</Key>
                <Value>banana_entry</Value>
              </String>
              <String>
                <Key>UserName</Key>
                <Value>user</Value>
              </String>
              <String>
                <Key>Password</Key>
                <Value Protected="True">pass</Value>
              </String>
              <AutoType>
                <Enabled>True</Enabled>
                <DataTransferObfuscation>0</DataTransferObfuscation>
                <DefaultSequence></DefaultSequence>
                <Association>
                  <Window></Window>
                  <KeystrokeSequence></KeystrokeSequence>
                </Association>
              </AutoType>
            </Entry>
A6GibKm commented 4 months ago

Thanks for the info.

I will take a look later and test it locally. I mean, we don't use any internal API (they would error out in CI) so I seriously cannot even begin to image how this is being set on our side. At least greping for it does not reveal any results.

A6GibKm commented 4 months ago

I dumped the xml of my testing db. Then I created a new db with a single entry, saved it and dumped it. ProtectInMemory is not in any of the two dumps. Bellow is the dump of the newly created DB. Both were dumped using the xml() function from pykeepass. 

<?xml version='1.0' encoding='' standalone='yes'?>
<KeePassFile>
  <Meta>
    <Generator>KeePassXC</Generator>
    <DatabaseName/>
    <DatabaseNameChanged>jV0X0w4AAAA=</DatabaseNameChanged>
    <DatabaseDescription/>
    <DatabaseDescriptionChanged>jV0X0w4AAAA=</DatabaseDescriptionChanged>
    <DefaultUserName/>
    <DefaultUserNameChanged>jV0X0w4AAAA=</DefaultUserNameChanged>
    <MaintenanceHistoryDays>365</MaintenanceHistoryDays>
    <Color/>
    <MasterKeyChanged>PeBy3Q4AAAA=</MasterKeyChanged>
    <MasterKeyChangeRec>-1</MasterKeyChangeRec>
    <MasterKeyChangeForce>-1</MasterKeyChangeForce>
    <MemoryProtection>
      <ProtectTitle>False</ProtectTitle>
      <ProtectUserName>False</ProtectUserName>
      <ProtectPassword>True</ProtectPassword>
      <ProtectURL>False</ProtectURL>
      <ProtectNotes>False</ProtectNotes>
    </MemoryProtection>
    <CustomIcons/>
    <RecycleBinEnabled>True</RecycleBinEnabled>
    <RecycleBinUUID>AAAAAAAAAAAAAAAAAAAAAA==</RecycleBinUUID>
    <RecycleBinChanged>jV0X0w4AAAA=</RecycleBinChanged>
    <EntryTemplatesGroup>AAAAAAAAAAAAAAAAAAAAAA==</EntryTemplatesGroup>
    <EntryTemplatesGroupChanged>jV0X0w4AAAA=</EntryTemplatesGroupChanged>
    <LastSelectedGroup>AAAAAAAAAAAAAAAAAAAAAA==</LastSelectedGroup>
    <LastTopVisibleGroup>AAAAAAAAAAAAAAAAAAAAAA==</LastTopVisibleGroup>
    <HistoryMaxItems>10</HistoryMaxItems>
    <HistoryMaxSize>6291456</HistoryMaxSize>
    <SettingsChanged>LV4X0w4AAAA=</SettingsChanged>
  </Meta>
  <Root>
    <Group>
      <UUID>14J2CaBO8I7R8zfVcxq3cw==</UUID>
      <Name>Root</Name>
      <Notes/>
      <IconID>48</IconID>
      <Times>
        <LastModificationTime>S+By3Q4AAAA=</LastModificationTime>
        <CreationTime>jV0X0w4AAAA=</CreationTime>
        <LastAccessTime>S+By3Q4AAAA=</LastAccessTime>
        <ExpiryTime>jV0X0w4AAAA=</ExpiryTime>
        <Expires>False</Expires>
        <UsageCount>0</UsageCount>
        <LocationChanged>jV0X0w4AAAA=</LocationChanged>
      </Times>
      <IsExpanded>True</IsExpanded>
      <DefaultAutoTypeSequence/>
      <EnableAutoType>null</EnableAutoType>
      <EnableSearching>null</EnableSearching>
      <LastTopVisibleEntry>AAAAAAAAAAAAAAAAAAAAAA==</LastTopVisibleEntry>
      <Entry>
        <UUID>UVPBZ9dBEe6UDeOkO1CbvQ==</UUID>
        <IconID>0</IconID>
        <Times>
          <CreationTime>S+By3Q4AAAA=</CreationTime>
          <LastModificationTime>T+By3Q4AAAA=</LastModificationTime>
          <LastAccessTime>T+By3Q4AAAA=</LastAccessTime>
          <ExpiryTime>S+By3Q4AAAA=</ExpiryTime>
          <Expires>False</Expires>
          <UsageCount>0</UsageCount>
          <LocationChanged>S+By3Q4AAAA=</LocationChanged>
        </Times>
        <String>
          <Key>UserName</Key>
          <Value></Value>
        </String>
        <String>
          <Key>Password</Key>
          <Value Protected="True"></Value>
        </String>
        <AutoType>
          <Enabled>True</Enabled>
          <DataTransferObfuscation>0</DataTransferObfuscation>
          <DefaultSequence></DefaultSequence>
          <Association>
            <Window></Window>
            <KeystrokeSequence></KeystrokeSequence>
          </Association>
        </AutoType>
        <String>
          <Key>Title</Key>
          <Value Protected="True">asdf</Value>
        </String>
        <String>
          <Key>URL</Key>
          <Value Protected="True">test</Value>
        </String>
      </Entry>
    </Group>
    <DeletedObjects/>
  </Root>
</KeePassFile>

EDIT: This was done with the nightly build of Secrets. But I don't think the result will change much if using an older stable release.

A6GibKm commented 4 months ago

By this point I am quite certain the issue does not lie in pykeepass. If you think the issue lies in Secrets open the issue there with steps to reproduce.

julianfairfax commented 4 months ago

(Maybe the real question is:) @aivanovski why do you hide all fields with that attribute set to true?

aivanovski commented 4 months ago

I've created new database in keepassxc, it also uses ProtectInMemory attribute and sets it by default only to password and any custom attribute that user marks as protected.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<KeePassFile>
    <Meta>
        <Generator>KeePassXC</Generator>
        <DatabaseName>test-protected</DatabaseName>
        <DatabaseNameChanged>WJlz3Q4AAAA=</DatabaseNameChanged>
        <DatabaseDescription/>
        <DatabaseDescriptionChanged>P5lz3Q4AAAA=</DatabaseDescriptionChanged>
        <DefaultUserName/>
        <DefaultUserNameChanged>P5lz3Q4AAAA=</DefaultUserNameChanged>
        <MaintenanceHistoryDays>365</MaintenanceHistoryDays>
        <Color/>
        <MasterKeyChanged>Z5lz3Q4AAAA=</MasterKeyChanged>
        <MasterKeyChangeRec>-1</MasterKeyChangeRec>
        <MasterKeyChangeForce>-1</MasterKeyChangeForce>
        <MemoryProtection>
            <ProtectTitle>False</ProtectTitle>
            <ProtectUserName>False</ProtectUserName>
            <ProtectPassword>True</ProtectPassword>
            <ProtectURL>False</ProtectURL>
            <ProtectNotes>False</ProtectNotes>
        </MemoryProtection>
        <CustomIcons/>
        <RecycleBinEnabled>True</RecycleBinEnabled>
        <RecycleBinUUID>AAAAAAAAAAAAAAAAAAAAAA==</RecycleBinUUID>
        <RecycleBinChanged>P5lz3Q4AAAA=</RecycleBinChanged>
        <EntryTemplatesGroup>AAAAAAAAAAAAAAAAAAAAAA==</EntryTemplatesGroup>
        <EntryTemplatesGroupChanged>P5lz3Q4AAAA=</EntryTemplatesGroupChanged>
        <LastSelectedGroup>AAAAAAAAAAAAAAAAAAAAAA==</LastSelectedGroup>
        <LastTopVisibleGroup>AAAAAAAAAAAAAAAAAAAAAA==</LastTopVisibleGroup>
        <HistoryMaxItems>10</HistoryMaxItems>
        <HistoryMaxSize>6291456</HistoryMaxSize>
        <SettingsChanged>P5lz3Q4AAAA=</SettingsChanged>
        <CustomData>
            <Item>
                <Key>KPXC_DECRYPTION_TIME_PREFERENCE</Key>
                <Value>100</Value>
            </Item>
            <Item>
                <Key>_LAST_MODIFIED</Key>
                <Value>Fri Mar 1 09:39:41 2024 GMT</Value>
            </Item>
        </CustomData>
    </Meta>
    <Root>
        <Group>
            <UUID>pACcbU9uRCunpFzwQbJiBA==</UUID>
            <Name>Root</Name>
            <Notes/>
            <IconID>48</IconID>
            <Times>
                <LastModificationTime>uplz3Q4AAAA=</LastModificationTime>
                <CreationTime>P5lz3Q4AAAA=</CreationTime>
                <LastAccessTime>uplz3Q4AAAA=</LastAccessTime>
                <ExpiryTime>P5lz3Q4AAAA=</ExpiryTime>
                <Expires>False</Expires>
                <UsageCount>0</UsageCount>
                <LocationChanged>P5lz3Q4AAAA=</LocationChanged>
            </Times>
            <IsExpanded>True</IsExpanded>
            <DefaultAutoTypeSequence/>
            <EnableAutoType>null</EnableAutoType>
            <EnableSearching>null</EnableSearching>
            <LastTopVisibleEntry>AAAAAAAAAAAAAAAAAAAAAA==</LastTopVisibleEntry>
            <Entry>
                <UUID>r4puvraFQ/G1R9MD+t8V0Q==</UUID>
                <IconID>0</IconID>
                <ForegroundColor/>
                <BackgroundColor/>
                <OverrideURL/>
                <Tags/>
                <Times>
                    <LastModificationTime>uplz3Q4AAAA=</LastModificationTime>
                    <CreationTime>cplz3Q4AAAA=</CreationTime>
                    <LastAccessTime>uplz3Q4AAAA=</LastAccessTime>
                    <ExpiryTime>cplz3Q4AAAA=</ExpiryTime>
                    <Expires>False</Expires>
                    <UsageCount>0</UsageCount>
                    <LocationChanged>uplz3Q4AAAA=</LocationChanged>
                </Times>
                <String>
                    <Key>Notes</Key>
                    <Value>Apple is evil</Value>
                </String>
                <String>
                    <Key>Password</Key>
                    <Value ProtectInMemory="True">abc123</Value>
                </String>
                <String>
                    <Key>Title</Key>
                    <Value>Apple credentials</Value>
                </String>
                <String>
                    <Key>URL</Key>
                    <Value>https://apple.com</Value>
                </String>
                <String>
                    <Key>UserName</Key>
                    <Value>john.doe@example.com</Value>
                </String>
                <String>
                    <Key>custom</Key>
                    <Value ProtectInMemory="True">protected value</Value>
                </String>
                <AutoType>
                    <Enabled>True</Enabled>
                    <DataTransferObfuscation>0</DataTransferObfuscation>
                    <DefaultSequence/>
                </AutoType>
                <History/>
            </Entry>
        </Group>
        <DeletedObjects/>
    </Root>
</KeePassFile>
Evidlo commented 4 months ago

I don't understand the different between Protected and ProtectInMemory (which was introduced here).

Anyway, it seems clear that pykeepass isn't marking anything as protected by default except the password field. Open another issue if ProtectInMemory is a problem in other ways.

julianfairfax commented 4 months ago

It appears the actual flag that is added to all entries is Protected.