search

cubewise-code / rushti

Smooth parallelization of TI Processes with TM1py
https://code.cubewise.com/tm1py-help-content/run-processes-in-parallel-using-only-connection
MIT License
9 stars 13 forks source link

Default }bedrock processes elements start delimeter "¦" within tasks file terminates the script #40

Closed 4rtem closed 2 years ago

4rtem commented 3 years ago

Sample from the tasks file id="97" predecessors="96" instance="One Plan GP" process="}bedrock.cube.data.clear" pCube="User Properties" pFilter="User Properties m¦CUP Role overrule" id="98" predecessors="97" instance="One Plan FP" process="}bedrock.cube.data.clear" pCube="User Properties" pFilter="User Properties m¦CUP Role overrule"

The problem is sitting within input_file.readlines()

2021-05-08 07:50:36,687 - RushTI - ERROR - Fatal Error Traceback (most recent call last): File "C:/Users/adekht/PycharmProjects/Documentor/rushti.py", line 497, in results = event_loop.run_until_complete( File "C:\Python\lib\asyncio\base_events.py", line 616, in run_until_complete return future.result() File "C:/Users/adekht/PycharmProjects/Documentor/rushti.py", line 377, in work_through_tasks lines = get_lines(file_path, max_workers, mode) File "C:/Users/adekht/PycharmProjects/Documentor/rushti.py", line 292, in get_lines return extract_lines_from_file_type_opt(max_workers, file_path) File "C:/Users/adekht/PycharmProjects/Documentor/rushti.py", line 157, in extract_lines_from_file_type_opt tasks = extract_tasks_from_file_type_opt(file_path) File "C:/Users/adekht/PycharmProjects/Documentor/rushti.py", line 181, in extract_tasks_from_file_type_opt lines = input_file.readlines() File "C:\Python\lib\codecs.py", line 322, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa6 in position 1765: invalid start byte 2021-05-08 07:50:41,425 - RushTI - ERROR - RushTI aborted with error RushTI aborted with error

MariusWirtz commented 3 years ago

I think you need to make sure, that the encoding of the tasks.txt file is utf-8. This should do the trick: SetOutputCharacterSet (sOutfile, 'TM1CS_UTF8');

https://www.tm1forum.com/viewtopic.php?t=8720

4rtem commented 3 years ago

I still think it is the problem with "¦" symbol. The problem is resolved when I changed the delimiter to ":" in the same TI generating the tasks file. I will test SetOutputCharacterSet (sOutfile, 'TM1CS_UTF8');

MariusWirtz commented 2 years ago

In f52fe0b1476e476152a63228ebc757e8e51fcc98 we added a preprocessing step to RushTI that should resolve this problem. Can you verify that the problem is resolved when using the current master branch?