buttercup / buttercup-desktop

:key: Cross-Platform Passwords & Secrets Vault
https://buttercup.pw
GNU General Public License v3.0
4.3k stars 332 forks source link

Can't import password from bitwarden #1017

Closed ghost closed 1 year ago

ghost commented 3 years ago

Hi, I'm trying to import password from Bitwarden to Buttercup Desktop Apps on Macos M1 but i'm encountering this error : "Failed importing: The first argument must be of type string or an instance of Buffer, ArrayBuffer, or Array or an Array-like Object. Received undefined". I have tried to import JSON, CSV, JSON Encrypted Files but it's not working. Any tricks to make it work ? Have a nice day !

0x3333 commented 3 years ago

Same error for 1Password

Failed importing: The first argument must be of type string or an instance of Buffer, ArrayBuffer, or Array or an Array-like Object. Received undefined

boerge30 commented 3 years ago

Same error ... Bitwarden (.json) / iMac (2011) / MacOS10.13.6

svankan commented 3 years ago

I have the same problem. Export from Bitwarden-portable 1.26.5 (.json) Import to Buttercup Desktop v.2.9.1 Windows 10

canadaduane commented 3 years ago

I've narrowed it down a bit to the following "type" property.

setProperty 0d6999ac-fb42-4425-8ac3-7e05cfc2e068 password H...
setProperty 8332737a-3cb1-4bc7-899b-ec4e4a6dc437 title VMWare Fusion 4.0
setProperty 8332737a-3cb1-4bc7-899b-ec4e4a6dc437 password M...
setProperty 4ff62c48-add1-4361-a1b9-74b4c8369c10 title USAA (AmEx)
setProperty 4ff62c48-add1-4361-a1b9-74b4c8369c10 type undefined
[ERR] 07:53:59: Failed importing TypeError [ERR_INVALID_ARG_TYPE]: The first argument must be of type string or an instance of Buffer, ArrayBuffer, or Array or an Array-like Object. Received undefined
at from               buffer.js:333                                                                                                                                
at encodeBase64       ../buttercup-core/dist/env/native/encoding.js:26             return Buffer.from(text, "utf8").toString("base64");                            
at encodeStringValue  ../buttercup-core/dist/tools/encoding.js:71                  const retval = `${exports.ENCODED_STRING_PREFIX}${encode(value)}`;              

It seems to be failing here (mapTreeLevelToVault in @buttercup/importer):

Object.keys(rawEntry.meta).forEach(function(metaKey) {
    entry.setProperty(metaKey, rawEntry.meta[metaKey]);
});

It appears the rawEntry.meta has a keypair "type": undefined. That's as far as I've been able to trace it.

UPDATE: Here's the offending line: https://github.com/buttercup/buttercup-importer/blob/master/source/tools/1passwordEntry.js#L41

The data in my 1pif file has the following fields:

    "secureContents": {
        "cardholder": "Duane ...",
        "ccnum": "3...",
        "expiry_yy": "2021",
        "expiry_mm": "1",
        "cvv": "5...1",
        "sections":[...]
    }

So in my case, it seems the importer assumes "type" is a required field for credit card "secureContents", while 1password does not necessarily export that field.

IroHaturo commented 3 years ago

same here. .json exported from Bitwarden App 1.27.1 macOS. When importing into Buttercup (Desktop @ v2.10.0 Core @ v6.0.0 Flags: installed), gives Error: Import failed: The first argument must be of type string or an instance of Buffer, ArrayBuffer, or Array or an Array-like Object. Received null

With .csv Import, Buttercup gives success message, generates the entries (URLs) but all users and password are empty.

forrany commented 3 years ago

same here. .json exported from Bitwarden App 1.27.1 macOS. When importing into Buttercup (Desktop @ v2.10.0 Core @ v6.0.0 Flags: installed), gives Error: Import failed: The first argument must be of type string or an instance of Buffer, ArrayBuffer, or Array or an Array-like Object. Received null

With .csv Import, Buttercup gives success message, generates the entries (URLs) but all users and password are empty.

Exactly the same on windows

KBone12 commented 3 years ago

I also have the same trouble. The JSON file is exported from Bitwarden (version 1.53.0, firefox extension) and imported to Buttercup (version 2.10.0, windows x64 portable). According to buttercup-desktop.log, it may caused when setProperty's second argument is null. In my case, the json file has some username property whose values are nulls. After I replaced all nulls to "" (empty string), there are no errors and Buttercup imports the edited JSON file correctly.

buttercup-desktop.log:

[ERR] 11:25:36: Failed importing TypeError [ERR_INVALID_ARG_TYPE]: The first argument must be of type string or an instance of Buffer, ArrayBuffer, or Array or an Array-like Object. Received null
at from               buffer.js:333                                                                                                                                 
at encodeBase64       ….asar/node_modules/buttercup/dist/env/native/encoding.js:26  return Buffer.from(text, "utf8").toString("base64");                            
at encodeStringValue  …s/app.asar/node_modules/buttercup/dist/tools/encoding.js:70  return `${exports.ENCODED_STRING_PREFIX}${encode(value)}`;                      
at wrap               …app.asar/node_modules/buttercup/dist/io/formatA/tools.js:24  wrap: (txt) => encoding_1.encodeStringValue(txt),                               
at addArgument        …app.asar/node_modules/buttercup/dist/io/formatA/tools.js:76  this._commandArgs.push(newArgRule.wrap(arg));                                   
at setEntryProperty   …app.asar/node_modules/buttercup/dist/io/VaultFormatA.js:345  .addArgument(value)                                                             
at setProperty        …rces/app.asar/node_modules/buttercup/dist/core/Entry.js:230  this.vault.format.setEntryProperty(this.id, property, value);                   
at                    …/@buttercup/importer/dist/importers/BitwardenImporter.js:12  groups[null]=vault.createGroup(DEFAULT_GROUP);bwJson.folders.forEach(bitwardenF…
at forEach                                                                                                                                                          
at                    …/@buttercup/importer/dist/importers/BitwardenImporter.js:12  groups[null]=vault.createGroup(DEFAULT_GROUP);bwJson.folders.forEach(bitwardenF…
zicklag commented 3 years ago

A thanks @KBone12, I didn't try your pull request, but I just used find and replace in the JSON file to replace all null usernames and notes with empty strings instead of null and the import worked!

id027102 commented 2 years ago

Today , I install buttercuo appimage on linux debian 11 But I must remove it : UNUSABLE Same problems still occurs when triing to import from bitwarden (json : failed , csv : all empty)

nick-delirium commented 2 years ago

Still getting same error on latest version (both buttercup and BW), even if I change all null fields to empty strings.

id027102 commented 2 years ago

NO COMMENT from developpers I get back to BW and forget buttercup Also unsubscribe here , so no answer more needed

perry-mitchell commented 2 years ago

NO COMMENT from developpers

I get back to BW and forget buttercup

Also unsubscribe here , so no answer more needed

I'm one developer. You want something fixed and I can't get to it? Consider fixing it yourself or at least be constructive when replying.

Sounds like the community isn't missing anything by your departure.

perry-mitchell commented 2 years ago

I apologise for the delay addressing this - I don't currently have the time to look into it. I'd absolutely appreciate any help on the matter in the way of debugging and development. It seems like it'd help a number of people, and having import/export capabilities are crucial for a password manager.

entrptaher commented 2 years ago

Faced the issue myself just now on Ubuntu 20.04. Hopefully will have some time to debug this.

sash85 commented 2 years ago

Hello, i had the same problem when i try to import the complete 1PW Data. Then i sort my wallet to category view and export only the logins (no notes, wifi, bankig, ...). All logins are imported without problems.

MilkSjeik commented 2 years ago

I would like to give a hand, having the same issue import with my 1Password file. Got some Javascript skills, but new to these modules. What is the best way to setup a test environment? Just make an own test script that uses the buttercup-importer module?

perry-mitchell commented 2 years ago

Hi @MilkSjeik - We'd absolutely love some help debugging the 1PW importer! If you have a chance, I'd recommend cloning the importer and setting that up (npm install etc.). Run the tests there to make sure it works and then I'd simply suggest dropping a test.js file in the root, importing the importer methods there and try testing with your own export - see if you can fix the problems mentioned here.. or at least reproduce them sufficiently so that you might contribute an updated 1PW test file. I could then try to fix it myself.

Hardest part for me is that I'm not a regular 1PW user, so having someone who uses it regularly build me an example vault export would really help.

MilkSjeik commented 2 years ago

Short update, I've tested with following script:

const { OnePasswordImporter } = require("./source/index.js");

OnePasswordImporter.loadFromFile("<path to my 1password.1pif directory>")
    .then(importer => importer.export())
    .then(vault => {
        this.vault = vault;
    });;

debugger;

This worked perfectly; the Promise returned a vault object with groups and entries. Hence, I assume the problem occurs when loading that vault object into buttercup.

MilkSjeik commented 2 years ago

Back to another "simple" question: how do I debug the desktop project? I would like to verify what

        const importedVault = await importerInst.export();
        // Import into source's vault
        await mergeVaults(sourceID, importedVault);

does in source/main/services/import.ts

Already started:

    "start:renderer": "concurrently \"tsc --build tsconfig.json -w\" \"webpack --mode development --watch\"",

But no idea how to connect with the Chrome dev tools/get the GUI up and running.

MilkSjeik commented 2 years ago

Found a way to start debugging, as mentioned in the README file:

Once cloned, make sure to install all dependencies: npm install. After that, open 2 terminals and run npm run start:renderer in one and npm run start:main in the other.

But I deviated a bit from the instructions:

MilkSjeik commented 2 years ago

Update, this line in OnePasswordImporter.js:

            const pifTree = convert1pifToJSON(this._data);

Behaves differently in the buttercup-desktop and the buttercup-importer project.

When loading credit card info from the 1password file, the import function in buttercup-desktop returns for example following entry (rawEntry.meta):

{type: undefined, cardholder: 'dummy', expiry-year: undefined, expiry-month: undefined, 

This causes issues with the undefined values later at:

            entry.setProperty(metaKey, rawEntry.meta[metaKey])

While the buttercup-importer project returns for the same file/entry (rawEntry.meta):

{cardholder: 'dummy'}

Will search for the difference between the 2 later on.

MilkSjeik commented 2 years ago

@perry-mitchell the difference is located in the 1passwordEntry.js file from buttercup-importer.

The version included in buttercup-desktop: No checks

Doesn`t contain the changes from following commit: https://github.com/buttercup/buttercup-importer/commit/353e66b2444ae550760dbec5ca8203e88741537b

They have to be included to avoid loading undefined values in the Vault object.

MilkSjeik commented 2 years ago

Tested with a manual copy of dist folder build from the main buttercup-importer branch. It worked, when comparing I' ve noticed that the current version of buttercup-importer doesn't contain the latest commits in that main branch. Hence, created a pull request in buttercup-importer to add a new version: 3.0.2. When completed, the package.json file should also be adjusted here to use the new buttercup-importer version.

amit510 commented 1 year ago

Same error here as well, trying to import bitwarden json file into Buttercup. I did try to remove null from the json file, still the import fails.

TheMetMan commented 1 year ago

Same for me. I was looking forward to keeping my passwords 'in house' but this is hopeless. However from Linux Format Magazine LXF297 this workaround worked for me: First, after attempting any platform-specific options, export your vault in CSV format. Then open it and reformat it so only four columns of data are kept: title, username, password and url, making sure those are the names in the first row (case-sensitive). Save the file and you should now be able to import it into Buttercup via the CSV (CSV) method provided.

MilkSjeik commented 1 year ago

Tested with a manual copy of dist folder build from the main buttercup-importer branch.

It worked, when comparing I' ve noticed that the current version of buttercup-importer doesn't contain the latest commits in that main branch.

Hence, created a pull request in buttercup-importer to add a new version: 3.0.2.

When completed, the package.json file should also be adjusted here to use the new buttercup-importer version.

@perry-mitchell how do we get things moving? I've noticed you also have a pull request pending (but in buttercup-desktop) with dependency updates.

perry-mitchell commented 1 year ago

@MilkSjeik I'm looking into this now.

EDIT: Released updated importer

EDIT 2: Have updated the desktop app with the new importer version, but due to the other dependency upgrades there it's failing to release. I'll try to get it out later today.

stijnme commented 1 year ago

Thanks @perry-mitchell , feel free to let me know if I can provide any assistance to reduce your workload.

perry-mitchell commented 1 year ago

Quick update: I got the build process fixed and have tested the Mac and Linux binaries.. they work well. I need to test the Windows binary today before I'll release v2.18. I have not tested the Bitwarden and 1Password importers, so please check those once the release comes out.

perry-mitchell commented 1 year ago

Released!

Sorry about the delay on this one. Will close for now.. let me know if the issues still occur.

MilkSjeik commented 1 year ago