CSV import job not importing passwords

[revit8]

Expected behavior

Import CSV file containing URLs, usernames, and passwords to automatically create credentials.

Actual behavior

CSV import job appears to complete successfully, but ends with error SMMC#2. Credential entries are created with URL and username correctly populated, but password field is empty.

Workflow log for each credential:

• JSON entries shown with password field contents as "<masked>"
• Next, parent/child nodes load
• Next, CSV import: updated password for login "my-username" for existing service "google.com"
• Next, MMM Save: tackling "my-username" for service "google.com" at address "0604" followed by Detected password change for "my-username" on "google.com"
• Finally, log ends with

  DEBUG: (2024-03-30T15:28:02.787) :0 - Number of parent orphans: 0
  DEBUG: (2024-03-30T15:28:02.787) :0 - Number of children orphans: 2
  DEBUG: (2024-03-30T15:28:02.787) :0 - Number of data parent orphans: 0
  DEBUG: (2024-03-30T15:28:02.787) :0 - Number of data children orphans: 0
  INFO: (2024-03-30T15:28:02.787) :0 - Errors were found in the database
  CRITICAL: (2024-03-30T15:28:02.787) :0 - Error in our local DB (algo PB?)
  INFO: (2024-03-30T15:28:02.787) :0 - "Exiting MMM"
  INFO: (2024-03-30T15:28:02.931) :0 - MMM exit ok
  DEBUG: (2024-03-30T15:28:02.933) :0 - Received status:  "09001100050005daefff000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
  DEBUG: (2024-03-30T15:28:02.933) :0 - Battery percent:  90
  DEBUG: (2024-03-30T15:28:02.933) :0 - received MPCmd::MOOLTIPASS_STATUS:  5
  DEBUG: (2024-03-30T15:28:02.933) :0 - Update client status changed:  WSServerCon(0xdb9e60)
  DEBUG: (2024-03-30T15:28:02.933) :0 - Update client status changed:  WSServerCon(0xde4310)
  INFO: (2024-03-30T15:28:02.933) :0 - "Fetch data files"
  DEBUG: (2024-03-30T15:28:02.936) :0 - Sending files cache
  DEBUG: (2024-03-30T15:28:02.936) :0 - Sending files cache
  DEBUG: (2024-03-30T15:28:02.941) :0 - JSON API recv: {
  "msg": "load_params"
  }

Step by step guide to reproduce the problem

• Export CSV from current password manager
• Cleanup CSV to contain only three columns: URL | USERNAME | PASSWORD
• Using Moolticute app, import CSV

Moolticute Version

• MooltiApp: v1.03.0

Operating System

• Debian Based Distribution

Mooltipass Device

• The Mooltipass Mini BLE

[limpkin]

that's a very odd one... can you confirm that you're using a comma separated CSV? Assuming so, would you be able to give us a sample csv file so we could reproduce the issue?

[revit8]

The CSV is indeed comma separated (though I had not verified previously with a text editor). I uploaded the original CSV but edited with fake credentials here. The issue was reproducible when importing the test CSV into a new user account on an unused smartcard. A screenshot of how the credentials appear on my side is here.

[limpkin]

a quick word to let you know we'll soon be working on it :)

[limpkin]

so that's interesting, I can't reproduce it on 1.03.12-testing: https://betas.themooltipass.com/v1.03.12-testing/

can you give it a go and check that you can't reproduce it on this release either?

[rastagraffix]

I have tried the https link in the list-unsubscribe header a half-dozen time, and every time I get an HTTP 404 error.

I have also emailed the mail address in the list-unsubscribe header four times. That just seems to go into a black home.

Somebody, please, get me off this list.

And fix the damn mailing list manager.

--lyndon

[limpkin]

hello @rastagraffix which mailing list are you referring to? github's? if so, we have no control over it...

[revit8]

The issue occurs for me on 1.03.12-testing as well. I used the same test csv file as previously uploaded.

Couple notes:

1. A new error popped up when I used the CSV containing the real credentials: Couldn't import CSV file: YLyKeEkZnpTvMFtLs885KdawJ6SMAKTsKt6hBeSigKoYgVNPrbiVkRemy7AXULnG6e5ucSYFFaBS5Qso has longer than supported length

   The file size is 28.3KB with a little over 350 entries.

2. Another error still comes up that I failed to mention previously. At the start of the import, the mini-ble prompts to verify adding the first credential. Upon selecting the checkmark for the first one, an error pops up in the app Credential store failed, however all of the credentials from the CSV are actually imported (minus the passwords).

   Selecting allow or deny for each subsequent credential does not affect whether it imports or not; they all import. This is useful in my case though :) , as it would be a bit cumbersome clicking the wheel each time for allowing 300-ish credentials. I understand this could be a different issue altogether, but just wanted to mention it as it occurs during the process.

[limpkin]

1. This is baffling... I simply can't reproduce it. Can you reproduce it with an empty DB?
2. yup... csv import is supposed to be performed with simple mode enabled, as so much clicking would indeed be quite annoying :)

[revit8]

Fascinating, apparently the issue only occurs when advanced mode is enabled. When in simple mode, the CSV import works fine.

On a related note -- thanks for the simple mode suggestion, it works wonders. I just noticed you even advise doing so in your documentation, if I'd have only read it.

I suppose that explains why you couldn't reproduce the issue; you were importing the correct way :P

[revit8]

Perhaps I spoke too soon. The uploaded test CSV worked in simple mode, but when I tried importing the production CSV the same issue occurs -- error Moolticute Internal Error (SMMC#2), and the password fields show Non-initialized password in light gray.

If helpful, I would welcome a remote support session at a day and time that works for us both. I could send a link that would allow you to remote in .

[limpkin]

I wonder if this has to do with the empty service / empty login... but I still can't reproduce your issue in advanced mode :D can you try with our latest beta? https://betas.themooltipass.com/v1.03.16-testing/

[revit8]

Ok, good news -- the CSV import works with both the betas and the current release.

I just realized that it appears the CSV import process does not overwrite existing credentials. The import finally succeeded after erasing the profile, but I don't know at what point during CSV pruning the issue was actually fixed, as there were a lot of manual changes made. The only three formatting issues I recall having to change in the production CSV were the number of fields, removing multi-URLs, and field length issues (generally long passwords or URLs containing session IDs). On a side note, I have no idea how the test CSV import suddenly worked the other day, as I had not yet erased the profile.

I'm still not sure what the root cause is though. This is the closest I got to identifying it:

• The CSV was generated by Bitwarden, which takes a lot of cleaning up depending on how many fields were utilized
• Bitwarden allows multiple URLs for matching varying domain names to a single credential
• When Bitwarden exports a multi-URL credential to CSV, it does so in the following format: "google.com,youtube.com",google-username,google-password
• Moolticute seems to honor this a three-field credential, but does not actually import that credential entry

I was hoping the multi-URL fields were the root cause, but upon going back and manually entering a multi-URL credential in the CSV, the original issue did not occur. That credential was simply not imported.

[limpkin]

oh right, we do have some checks on the URLs and aren't handling this particular case indeed. @deXol could you add support for this? should we close this issue then?

[revit8]

Yeah, I imagine if it hasn't happened to anyone else by this point then it's probably one of those astronomically rare occurrences. Thanks for all your help.

[limpkin]

ping me if that happens again then :)