search

Avaiga / taipy-core

A Python library to build powerful and customized data-driven back-end applications.
Apache License 2.0
39 stars 15 forks source link

BUG-Unexpected JSON error when executing a lot of scenarios #206

Closed FlorianJacta closed 2 years ago

FlorianJacta commented 2 years ago

Description A problem arises when scenarios are submitted.

Traceback (most recent call last):
  File "c:\Users\jacta\Desktop\Stage\Avaiga\Code\taipy-CFM\convert_old_scenarios.py", line 121, in <module>
    submit_old_scenarios(scenario)
  File "c:\Users\jacta\Desktop\Stage\Avaiga\Code\taipy-CFM\convert_old_scenarios.py", line 32, in submit_old_scenarios
    while not scenario.metrics.is_ready_for_reading and wait < 8:
  File "C:\Users\jacta\AppData\Local\R-MINI~1\envs\cfm\lib\site-packages\taipy\core\scenario\scenario.py", line 173, in __getattr__
    if protected_attribute_name in pipeline.tasks:
  File "C:\Users\jacta\AppData\Local\R-MINI~1\envs\cfm\lib\site-packages\taipy\core\common\_reload.py", line 59, in _do_reload
    self = _reload(manager, self)
  File "C:\Users\jacta\AppData\Local\R-MINI~1\envs\cfm\lib\site-packages\taipy\core\common\_reload.py", line 35, in _reload
    return _get_manager(manager)._get(obj, obj)
  File "C:\Users\jacta\AppData\Local\R-MINI~1\envs\cfm\lib\site-packages\taipy\core\common\_manager.py", line 69, in _get
    return cls._repository.load(entity_id)
  File "C:\Users\jacta\AppData\Local\R-MINI~1\envs\cfm\lib\site-packages\taipy\core\_repository\_fs_base.py", line 98, in load
    return self.__to_entity(self.__get_model_filepath(model_id))
  File "C:\Users\jacta\AppData\Local\R-MINI~1\envs\cfm\lib\site-packages\taipy\core\_repository\_fs_base.py", line 149, in __to_entity
    return self._from_model(model)
  File "C:\Users\jacta\AppData\Local\R-MINI~1\envs\cfm\lib\site-packages\taipy\core\pipeline\_pipeline_repository.py", line 48, in _from_model
    tasks = self.__to_tasks(model.tasks)
  File "C:\Users\jacta\AppData\Local\R-MINI~1\envs\cfm\lib\site-packages\taipy\core\pipeline\_pipeline_repository.py", line 72, in __to_tasks
    if task := _TaskManager._get(_id):
  File "C:\Users\jacta\AppData\Local\R-MINI~1\envs\cfm\lib\site-packages\taipy\core\common\_manager.py", line 69, in _get
    return cls._repository.load(entity_id)
  File "C:\Users\jacta\AppData\Local\R-MINI~1\envs\cfm\lib\site-packages\taipy\core\_repository\_fs_base.py", line 98, in load
    return self.__to_entity(self.__get_model_filepath(model_id))
  File "C:\Users\jacta\AppData\Local\R-MINI~1\envs\cfm\lib\site-packages\taipy\core\_repository\_fs_base.py", line 149, in __to_entity
    return self._from_model(model)
  File "C:\Users\jacta\AppData\Local\R-MINI~1\envs\cfm\lib\site-packages\taipy\core\task\_task_repository.py", line 46, in _from_model
    output=self.__to_data_nodes(model.output_ids),
  File "C:\Users\jacta\AppData\Local\R-MINI~1\envs\cfm\lib\site-packages\taipy\core\task\_task_repository.py", line 61, in __to_data_nodes
    if data_node := _DataManager._get(_id):
  File "C:\Users\jacta\AppData\Local\R-MINI~1\envs\cfm\lib\site-packages\taipy\core\common\_manager.py", line 69, in _get
    return cls._repository.load(entity_id)
  File "C:\Users\jacta\AppData\Local\R-MINI~1\envs\cfm\lib\site-packages\taipy\core\_repository\_fs_base.py", line 98, in load
    return self.__to_entity(self.__get_model_filepath(model_id))
  File "C:\Users\jacta\AppData\Local\R-MINI~1\envs\cfm\lib\site-packages\taipy\core\_repository\_fs_base.py", line 147, in __to_entity
    data = json.load(f, cls=_CustomDecoder)
  File "C:\Users\jacta\AppData\Local\R-MINI~1\envs\cfm\lib\json\__init__.py", line 293, in load
    return loads(fp.read(),
  File "C:\Users\jacta\AppData\Local\R-MINI~1\envs\cfm\lib\json\__init__.py", line 370, in loads
    return cls(**kw).decode(s)
  File "C:\Users\jacta\AppData\Local\R-MINI~1\envs\cfm\lib\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Users\jacta\AppData\Local\R-MINI~1\envs\cfm\lib\json\decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

How to reproduce Clone the ITM-CFM repo in Avaiga and run the convert_old_scenarios.py.

Expected behavior The execution of pipelines shouldn't create any problems.

jrobinAV commented 2 years ago

Assumption: This is due to concurrency in accessing the entities (a solution would be to use a database with transactions and lock)