conn.update_dataset does not work, self._dataset_id is None

[verbeemen]

Best

conn.update_dataset(data_frame=my_data, 
                                              dataset_id='AF9C692F11EA37B2D49A123456789',
                                              table_name='my_table', 
                                              update_policy='replace')

Does not work, it says

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-8-7f713753db3c> in <module>
      2                                               dataset_id='AF9C692F11EA37B2D49A123456789',
      3                                               table_name='my_table',
----> 4                                               update_policy='replace')

~/conda_environments/experiments/lib/python3.7/site-packages/mstrio/microstrategy.py in update_dataset(self, data_frame, dataset_id, table_name, update_policy)
    311 
    312         # create dataset instance, add table, then publish the updates to the dataset
--> 313         ds = Dataset(connection=self, dataset_id=dataset_id)
    314         ds.add_table(name=table_name, data_frame=_df, update_policy=update_policy)
    315         ds.update()

~/conda_environments/experiments/lib/python3.7/site-packages/mstrio/dataset.py in __init__(self, connection, name, description, dataset_id)
     61         if dataset_id is not None:
     62             self.__check_param_str(dataset_id, "Dataset ID should be a string.")
---> 63             self.__load_definition()
     64 
     65     def add_table(self, name, data_frame, update_policy, to_metric=None, to_attribute=None):

~/conda_environments/experiments/lib/python3.7/site-packages/mstrio/dataset.py in __load_definition(self)
    295         """Load definition of an existing dataset."""
    296 
--> 297         response = datasets.dataset_definition(connection=self._connection, dataset_id=self._dataset_id)
    298 
    299         if not response.ok:

~/conda_environments/experiments/lib/python3.7/site-packages/mstrio/api/datasets.py in dataset_definition(connection, dataset_id, fields, verbose)
     16 
     17     """
---> 18     response = requests.get(url=connection.base_url + '/datasets/' + dataset_id,
     19                             headers={'X-MSTR-AuthToken': connection.auth_token,
     20                                      'X-MSTR-ProjectID': connection.project_id},

TypeError: can only concatenate str (not "NoneType") to str

My guess is that : self._dataset_id is not assigned or wrongly used.

Line 59-63

        self._dataset_id = None

        if dataset_id is not None:
            self.__check_param_str(dataset_id, "Dataset ID should be a string.")
            self.__load_definition()

self._dataset_id is instantiated as None if dataset_id is not None, it checks if dataset_id is indeed a string or not. if it is not a string, it will raise an error, otherwise it will continue (returns True)

Next we go into _self.__loaddefinition() (without passing the variable dataset_id nor assigning it to self._dataset_id)

Line 294-297

    def __load_definition(self):
        """Load definition of an existing dataset."""

        response = datasets.dataset_definition(connection=self._connection, dataset_id=self._dataset_id)

datasets.dataset_definition will be executed, Notice that self._dataset_id is still None

Next we go to api/datasets.py

Line 4-23

def dataset_definition(connection, dataset_id, fields=None, verbose=False):
    """Get the definition of a dataset.

    Args:
        connection (object): MicroStrategy connection object returned by `microstrategy.Connection()`.
        dataset_id (str): Identifier of a pre-existing dataset. Used when updating a pre-existing dataset.
        fields(list, optional): Specifies object types to be returned. Possible values include tables, columns,
            attributes, and metrics. If no value is set, attributes and metrics are returned.
        verbose (bool, optional): Verbosity of server responses; defaults to False.

    Returns:
        HTTP response object returned by the MicroStrategy REST server

    """
    response = requests.get(url=connection.base_url + '/datasets/' + dataset_id,
                            headers={'X-MSTR-AuthToken': connection.auth_token,
                                     'X-MSTR-ProjectID': connection.project_id},
                            params={'fields': fields},
                            cookies=connection.cookies,
                            verify=connection.ssl_verify)

dataset_definition requires a dataset_id, and will be used in url of line 18. But dataset_id is the result of self._dataset_id, which is still None

Result, request.get will probably FAIL

Last but not least, i tried to change self._dataset_id = None manually into self._dataset_id = 'AF9C692F11EA37B2D49A123456789' and it works ....

My suggestion, pass dataset_id trough to __load_definition like

    def __load_definition(self, dataset_id):
        """Load definition of an existing dataset."""

        response = datasets.dataset_definition(connection=self._connection, dataset_id=dataset_id)

        if not response.ok:
            self.__response_handler(response=response,
                                    msg="Error loading dataset '{}'. Check dataset ID.".format(dataset_id))
        else:
            self._definition = response.json()
            print(self._definition)
            self._name = self._definition['name']
            self._dataset_id = self._definition['id']

Kind regards

Dieter Verbeemen

[scottrigney]

@verbeemen thanks for logging this! Our team is looking into it.

[scottrigney]

@verbeemen we have a fix coming in our next release, 11.2.1, releasing around March 2020.