aws / aws-step-functions-data-science-sdk-python

Step Functions Data Science SDK for building machine learning (ML) workflows and pipelines on AWS
Apache License 2.0
289 stars 88 forks source link

Execution list_events html output fails with boolean properties for inputDetails. #93

Open brightsparc opened 4 years ago

brightsparc commented 4 years ago

When an event includes boolean properties the execution.list_events(html=True) method fails with error:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-65-138c22b04713> in <module>
----> 1 execution.list_events(html=True) # Bug

~/anaconda3/envs/python3/lib/python3.6/site-packages/stepfunctions/workflow/stepfunctions.py in list_events(self, max_items, reverse_order, html)
    487 
    488         if html:
--> 489             return HTML(events_list.to_html())
    490         else:
    491             return events_list

~/anaconda3/envs/python3/lib/python3.6/site-packages/stepfunctions/workflow/stepfunctions.py in to_html(self)
     45 
     46     def to_html(self):
---> 47         return EventsTableWidget(self).show()
     48 
     49 

~/anaconda3/envs/python3/lib/python3.6/site-packages/stepfunctions/workflow/widgets/events_table.py in __init__(self, events)
    141             timestamp=format_time(event.get("timestamp")),
    142             event_detail=self._format_event_detail(event)
--> 143         ) for event in events]
    144 
    145         self.template = Template(TABLE_TEMPLATE.format(table_rows='\n'.join(table_rows)))

~/anaconda3/envs/python3/lib/python3.6/site-packages/stepfunctions/workflow/widgets/events_table.py in <listcomp>(.0)
    141             timestamp=format_time(event.get("timestamp")),
    142             event_detail=self._format_event_detail(event)
--> 143         ) for event in events]
    144 
    145         self.template = Template(TABLE_TEMPLATE.format(table_rows='\n'.join(table_rows)))

~/anaconda3/envs/python3/lib/python3.6/site-packages/stepfunctions/workflow/widgets/events_table.py in _format_event_detail(self, event)
    307     def _format_event_detail(self, event):
    308         event_details = self._get_step_detail(event)
--> 309         self._unpack_to_proper_dict(event_details)
    310         return json.dumps(event_details, indent=4)
    311 

~/anaconda3/envs/python3/lib/python3.6/site-packages/stepfunctions/workflow/widgets/events_table.py in _unpack_to_proper_dict(self, dictionary)
    313         for k, v in dictionary.items():
    314             if isinstance(v, dict):
--> 315                 self._unpack_to_proper_dict(v)
    316             else:
    317                 dictionary[k] = self._load_json(v)

~/anaconda3/envs/python3/lib/python3.6/site-packages/stepfunctions/workflow/widgets/events_table.py in _unpack_to_proper_dict(self, dictionary)
    315                 self._unpack_to_proper_dict(v)
    316             else:
--> 317                 dictionary[k] = self._load_json(v)
    318 
    319     def _load_json(self, value):

~/anaconda3/envs/python3/lib/python3.6/site-packages/stepfunctions/workflow/widgets/events_table.py in _load_json(self, value)
    319     def _load_json(self, value):
    320         try:
--> 321             return json.loads(value)
    322         except ValueError as e:
    323             return value

~/anaconda3/envs/python3/lib/python3.6/json/__init__.py in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
    346         if not isinstance(s, (bytes, bytearray)):
    347             raise TypeError('the JSON object must be str, bytes or bytearray, '
--> 348                             'not {!r}'.format(s.__class__.__name__))
    349         s = s.decode(detect_encoding(s), 'surrogatepass')
    350 

TypeError: the JSON object must be str, bytes or bytearray, not 'bool'

This is due to the {'truncated': False} value in the following event:

{'timestamp': datetime.datetime(2020, 10, 6, 2, 41, 57, 614000, tzinfo=tzlocal()),
 'type': 'ExecutionStarted',
 'id': 1,
 'previousEventId': 0,
 'executionStartedEventDetails': {'input': '{\n    "ExperimentName": "mlops-nyctaxi",\n    "TrialName": "mlops-nyctaxi-7f99979c077d11ebbed6ad48a7dcc771",\n    "GitBranch": "master",\n    "GitCommitHash": "xxx",\n    "DataVersionId": "yyy",\n    "BaselineJobName": "mlops-nyctaxi-7f99979c077d11ebbed6ad48a7dcc771",\n    "BaselineOutputUri": "s3://sagemaker-us-east-1-691313291965/nyctaxi/monitoring/baseline/mlops-nyctaxi-pbl-7f99979c077d11ebbed6ad48a7dcc771",\n    "TrainingJobName": "mlops-nyctaxi-7f99979c077d11ebbed6ad48a7dcc771"\n}',
  'inputDetails': {'truncated': False},
  'roleArn': 'arn:aws:iam::691313291965:role/mlops-nyctaxi-sfn-execution-role'}}
poojashinde610 commented 3 years ago

Hi ,

Thanks for your valuable comment. Any resolution you can suggest ?

hlmiao commented 3 years ago

Hi, Any update on the issue fix?

ca-nguyen commented 3 years ago

Hello @brightsparc , Sorry for the late response!

Thanks for bringing this to our attention - I was able to reproduce this error with the hello_world_workflow example. Tagging it as a bug and keeping it on our radar.

alesageProovStation commented 1 year ago

Hi. I made a quick fix in my code: in the file stepfunctions.workflow.widgets.events_table, just add the two lines at the end of the file

def _load_json(self, value): try: return json.loads(value) except ValueError: return value except TypeError: # These are the added lines to fix the AWS bug. return value