Can't import dashbaords with CLI

xavier-GitHub76 commented 4 months ago

Bug description

Hello,

I try to export/import dashboard with cli but I have an error "superset.commands.exceptions.CommandInvalidError: Error importing dashboard"

How to reproduce the bug

I export my dashboards with CLI :

I delete all elements on my superset

I try import with cli superset import-dashboards -p myDashboards.zip -u xavier

I try also after a creation of database connexion :

Best regards

Screenshots/recordings

No response

Superset version

master / latest-dev

Python version

3.9

Node version

16

Browser

Chrome

Additional context

No response

Checklist

[ ] I have searched Superset docs and Slack and didn't find a solution to my problem.
[ ] I have searched the GitHub issue tracker and didn't find a similar bug report.
[ ] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.

dosubot[bot] commented 4 months ago

To resolve the superset.commands.exceptions.CommandInvalidError: Error importing dashboard issue when trying to import dashboards using the CLI in Apache Superset, follow these steps:

Check User Permissions: Ensure the user has the necessary permissions to create or overwrite dashboards. This is checked by security_manager.can_access("can_write", "Dashboard").
Validate Configuration: Ensure the dashboard configuration (config dictionary) is correctly formatted and includes all necessary fields. Pay special attention to the uuid, position, and metadata fields.
Handle UUIDs and IDs: Ensure that the UUIDs for charts and datasets are correctly mapped and updated in the configuration. The update_id_refs function helps in updating these references.
Remove Deprecated Fields: Remove any deprecated fields from the configuration, such as show_native_filters in the metadata.

Additionally, ensure that the CLI command for importing dashboards is correctly used. Here is an example of how to use the import_dashboards command from superset/cli/importexport.py:

@click.command()
@with_appcontext
@click.option(
    "--path",
    "-p",
    required=True,
    help="Path to a single ZIP file",
)
@click.option(
    "--username",
    "-u",
    required=True,
    help="Specify the user name to assign dashboards to",
)
def import_dashboards(path: str, username: Optional[str]) -> None:
    """Import dashboards from ZIP file"""
    from superset.commands.dashboard.importers.dispatcher import ImportDashboardsCommand
    from superset.commands.importers.v1.utils import get_contents_from_bundle

    if username is not None:
        g.user = security_manager.find_user(username=username)
    if is_zipfile(path):
        with ZipFile(path) as bundle:
            contents = get_contents_from_bundle(bundle)
    else:
        with open(path) as file:
            contents = {path: file.read()}
    try:
        ImportDashboardsCommand(contents, overwrite=True).run()
    except Exception:  # pylint: disable=broad-except
        logger.exception(
            "There was an error when importing the dashboards(s), please check "
            "the exception traceback in the log"
        )
        sys.exit(1)

By following these steps and ensuring the configuration and permissions are correctly handled, you should be able to resolve the CommandInvalidError when importing dashboards using the CLI in Apache Superset ^[1]^[2]^[3].

#### Details

**About Dosu** This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

anhvh1408 commented 3 months ago

I have same issue as you too, in my cases:

I have dashboards in json file first (exported from superset 2.0.0)
import json file to superset 3.1.3-dev -> work, dashboard shown perfectly
then I export these dashboards to zip and import again (both in superset 3.1.3-dev) -> some work, some throw errors same as yours

anhvh1408 commented 3 months ago

Okay, please follow this https://github.com/apache/superset/issues/26130 to debug your zip file and yaml file to see any arguments have errors

xavier-GitHub76 commented 3 months ago

Hello, my zip file is correct : the zip file (created from CLI) is imported with success from gui I have just the error when I import the zip file from CLI

I have try again on superset 4.0.0, I have the same behavior

Best regards

maximStafeevAtAllscripts commented 2 months ago

Yes, I have the same problem. Neither export nor import works on 4.0.1

maximStafeevAtAllscripts commented 2 months ago

If I import dashboard via GUI than import_dashboard from CLI successfully updates the dashboard. I'm also wondering where should I place the password for the database when importing via CLI? Superset prompts you for it when importing through the GUI.

muralikrishnankh commented 2 weeks ago

@maximStafeevAtAllscripts What worked for me:

Unzip the file
Go to dashboards directory
Password is represented as XXXXXXXXX there
I replaced it with actual password and compressed again.
Upload it using CLI or GUI. Both works and doesn't ask for password prompt.

dosubot[bot] commented 2 weeks ago

I found some similar issues and discussions that might help you resolve the "superset.commands.exceptions.CommandInvalidError: Error importing dashboard" error when using the CLI to import dashboards in Apache Superset:

Similar Discussions:
- superset import-dashboards KeyError: 'ZIPPED_FILE_MAX_SIZE' ^[1]
- Dashboard Import Error: An error occurred while importing dashboard: Error importing dashboard Please re-export your file and try importing again superset ^[2]

Closed Issues:

Importing dashboard error: The suggested solution is to ensure the folder structure of the zip file is correct. The structure should look like this:
```
Root
|
->  dataset_export_20240308T102914
      |
      -> Databases
      -> Datasets
```
^[3]
export, import dashboard error ..: The suggested solution is to manually create the database connection first, then the import wizard will ask for the database password ^[4].

Dashboard import asking for password database failed: The suggested solution is to use the API for importing dashboards instead of the CLI. Here is a sample script:

import requests

new_zip_file = 'path/to/dashboard_export.zip'
username = 'admin'
password = 'admin'

base_url = 'http://localhost:8088'

while True:
 try:
     response = requests.get(base_url, timeout=10)
     response.raise_for_status()
     break
 except Exception:
     pass

login_url = f"{base_url}/login/"
session = requests.Session()

payload = {}
headers = {
 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
 'Accept-Language': 'en-US,en;q=0.9'
}

response = session.request("GET", login_url, headers=headers, data=payload)
csrf_token = response.text.split('<input id="csrf_token" name="csrf_token" type="hidden" value="')[1].split('">')[0]

# perform login

payload = f'csrf_token={csrf_token}&username={username}&password={password}'
headers = {
 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
 'Content-Type': 'application/x-www-form-urlencoded'}

session.request("POST", login_url, headers=headers, data=payload, allow_redirects=False)
cookie = session.cookies.get_dict().get('session')
print(session.cookies.get_dict())

# Import dashboards

import_dashboard_url = f"{base_url}/api/v1/dashboard/import/"

with open(new_zip_file, 'rb') as f:
 payload = {
     'passwords': '{"databases/replaceWithYourDatabase.yaml":"databasePassword"}',
     'overwrite': 'true'
 }
 files = [
     ('formData', ('dashboards.zip', f, 'application/zip'))
 ]
 headers = {
     'Accept': 'application/json',
     'Cookie': f'session={cookie}',
     'X-CSRFToken': csrf_token
 }

 response = requests.request("POST", import_dashboard_url, headers=headers, data=payload, files=files)

print(response.text)

^[5]

Additionally, ensure that the user running the import command has the necessary permissions to either overwrite an existing dashboard or create a new one. The ImportDashboardsCommand class handles the import process and checks for these permissions ^[6]^[7]^[8].

If the user does not have the necessary permissions, you can either grant the required permissions or run the import command with a user who has the appropriate permissions. Also, make sure that the JSON file being imported is valid and contains the necessary data for the import process.

To continue talking to Dosu, mention @dosu.

rusackas commented 2 weeks ago

Pinging @betodealmeida in case they have any insight here.

apache / superset