The code does not work

[soumya-ghosh-wdc]

First of all I fixed the error as shown below :

if "choices" not in res: continue parse_logs=res["choice"][0]["message"]["content"] it should be if "choices" not in res: continue parse_logs=res["choices"][0]["message"]["content"]

Even after for a small raw log file (arranaged in excel with 1 column -log) the code keeps running forever...

{'id': 'chatcmpl-9yJ7BmJDro4hNQBFhzRoQTlLXBw34', 'object': 'chat.completion', 'created': 1724160185, 'model': 'gpt-3.5-turbo-0125', 'choices': [{'index': 0, 'message': {'role': 'assistant', 'content': '(1) [] [Sun Dec 04 04:47:44 2005] [notice] workerEnv.init() ok /etc/httpd/conf/workers2.properties\n(2) [] [Sun Dec 04 04:47:44 2005] [error] mod_jk child workerEnv in error state 6\n(3) [] [Sun Dec 04 04:51:08 2005] [notice] jk2_init() Found child in scoreboard slot 10\n(4) [] [Sun Dec 04 04:51:09 2005] [notice] jk2_init() Found child in scoreboard slot 8\n(5) [] [Sun Dec 04 04:51:09 2005] [notice] jk2_init() Found child in scoreboard slot 6\n(6) [] [Sun Dec 04 04:51:14 2005] [notice] workerEnv.init() ok /etc/httpd/conf/workers2.properties\n(7) [] [Sun Dec 04 04:51:14 2005] [notice] workerEnv.init() ok /etc/httpd/conf/workers2.properties\n(8) [] [Sun Dec 04 04:51:14 2005] [notice] workerEnv.init() ok /etc/httpd/conf/workers2.properties\n(9) [] [Sun Dec 04 04:51:18 2005] [error] mod_jk child workerEnv in error state 6\n(10) [] [Sun Dec 04 04:51:18 2005] [error] mod_jk child workerEnv in error state 6\n(11) [] [Sun Dec 04 04:51:18 2005] [error] mod_jk child workerEnv in error state 6\n(12) [] [Sun Dec 04 04:51:37 2005] [notice] jk2_init() Found child in scoreboard slot 10\n(13) [] [Sun Dec 04 04:51:38 2005] [notice] jk2_init() Found child in scoreboard slot 7', 'refusal': None}, 'logprobs': None, 'finish_reason': 'stop'}], 'usage': {'prompt_tokens': 554, 'completion_tokens': 476, 'total_tokens': 1030}, 'system_fingerprint': None} Parse logs : (1) [] [Sun Dec 04 04:47:44 2005] [notice] workerEnv.init() ok /etc/httpd/conf/workers2.properties (2) [] [Sun Dec 04 04:47:44 2005] [error] mod_jk child workerEnv in error state 6 (3) [] [Sun Dec 04 04:51:08 2005] [notice] jk2_init() Found child in scoreboard slot 10 (4) [] [Sun Dec 04 04:51:09 2005] [notice] jk2_init() Found child in scoreboard slot 8 (5) [] [Sun Dec 04 04:51:09 2005] [notice] jk2_init() Found child in scoreboard slot 6 (6) [] [Sun Dec 04 04:51:14 2005] [notice] workerEnv.init() ok /etc/httpd/conf/workers2.properties (7) [] [Sun Dec 04 04:51:14 2005] [notice] workerEnv.init() ok /etc/httpd/conf/workers2.properties (8) [] [Sun Dec 04 04:51:14 2005] [notice] workerEnv.init() ok /etc/httpd/conf/workers2.properties (9) [] [Sun Dec 04 04:51:18 2005] [error] mod_jk child workerEnv in error state 6 (10) [] [Sun Dec 04 04:51:18 2005] [error] mod_jk child workerEnv in error state 6 (11) [] [Sun Dec 04 04:51:18 2005] [error] mod_jk child workerEnv in error state 6 (12) [] [Sun Dec 04 04:51:37 2005] [notice] jk2_init() Found child in scoreboard slot 10 (13) [] [Sun Dec 04 04:51:38 2005] [notice] jk2_init() Found child in scoreboard slot 7

I don't understand the pupose of endless while loop - Can you please help ?

[lunyiliu]

Sorry for the typo in the code. In addition to the ''choice" error, this is probaboly due to "parsed_logs", which should be "parsed_log" to correctly assign the result to avoid triggering "while parsed_log=='' "

[lunyiliu]

If this correct you errors and get it working, you can make a new merge request with everything you modified

[soumya-ghosh-wdc]

The while loop problem seems to be not appearring any more after making the changes but I get the following excpetion 0it [00:05, ?it/s]/it] Traceback (most recent call last): File "C:\LogPrompt\LogPrompt_code.py", line 263, in write_toexcel('Candidate%d_'%(i+1)+OUTPUT_FILE,df_raw_answer,logs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\LogPrompt\LogPrompt_code.py", line 164, in write_to_excel prompts=df_raw_answer['prompt'].tolist()

  File "C:\python-3.12\Lib\site-packages\pandas\core\frame.py", line 4102, in __getitem__
    indexer = self.columns.get_loc(key)
              ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\python-3.12\Lib\site-packages\pandas\core\indexes\range.py", line 417, in get_loc
    raise KeyError(key)
KeyError: 'prompt'

[soumya-ghosh-wdc]

Can you please help here.

[soumya-ghosh-wdc]

I fixed the above error by changing:

write_to_excel(_OUTPUTFILE,df_raw_answer,logs) to the following

write_toexcel('Candidate%d_'%(i+1)+OUTPUT_FILE,df_raw_answer,logs)

[soumya-ghosh-wdc]

But I still get into another endless while loop in reprompt function:

0it [00:00, ?it/s]prompt 1 1it [00:04, 4.03s/it] Unnamed: 0 prompt answer 0 0 Parse the following logs into a template forma... (1) [] [] workerEnv.init() ok /etc/httpd/con... 'DataFrame' object has no attribute 'append' reprompting... 'DataFrame' object has no attribute 'append' reprompting... 'NoneType' object has no attribute 'group' reprompting... 'DataFrame' object has no attribute 'append' reprompting... 'NoneType' object has no attribute 'group' reprompting... 'DataFrame' object has no attribute 'append' reprompting... 'DataFrame' object has no attribute 'append' reprompting... 'NoneType' object has no attribute 'group' reprompting... 'DataFrame' object has no attribute 'append' reprompting... 'DataFrame' object has no attribute 'append' reprompting... 'DataFrame' object has no attribute 'append' reprompting... 'DataFrame' object has no attribute 'append' reprompting... 'DataFrame' object has no attribute 'append' reprompting... 'DataFrame' object has no attribute 'append' reprompting... 'DataFrame' object has no attribute 'append' reprompting... 'DataFrame' object has no attribute 'append' reprompting... 'DataFrame' object has no attribute 'append' reprompting... 'DataFrame' object has no attribute 'append' reprompting... 'DataFrame' object has no attribute 'append' reprompting... 'DataFrame' object has no attribute 'append' reprompting... 'DataFrame' object has no attribute 'append' reprompting... 'DataFrame' object has no attribute 'append' reprompting... 'DataFrame' object has no attribute 'append' reprompting... 'DataFrame' object has no attribute 'append' reprompting... 'DataFrame' object has no attribute 'append' reprompting... 'DataFrame' object has no attribute 'append' reprompting... 'DataFrame' object has no attribute 'append' reprompting... 'DataFrame' object has no attribute 'append' reprompting... 'DataFrame' object has no attribute 'append' reprompting... 'DataFrame' object has no attribute 'append' reprompting... 'DataFrame' object has no attribute 'append' reprompting... 'DataFrame' object has no attribute 'append' reprompting...

[lunyiliu]

Hi, to end the while loop and debug the code, simply comment the while-loop and try-excep code. Then the error will be located by python. For the 'DataFrame' object has no attribute 'append' error, it may be caused by the version of pandas, here is a article that may help: https://stackoverflow.com/questions/75956209/error-dataframe-object-has-no-attribute-append

[soumya-ghosh-wdc]

Even concat does not help- Unnamed: 0 prompt answer 0 0 Parse the following logs into a template forma... (1) [] [] workerEnv.init() ok /etc/httpd/con... 'DataFrame' object has no attribute 'concat' reprompting... 'DataFrame' object has no attribute 'concat' reprompting... 'DataFrame' object has no attribute 'concat' reprompting...

[soumya-ghosh-wdc]

The problem is at this line of write_to_excel function

parsed_logs_df=parsed_logs_df.append({'log':logs[index],'pred':pred_label},ignore_index=True)

[soumya-ghosh-wdc]

I fixed the above issue.

Solution

Instead of parsed_logs_df=parsed_logs_df.concat({'log':logs[index],'pred':pred_label},ignore_index=True) in the write_to_excel function It should be the following:

df1 = pd.DataFrame(list(logs[index]), columns=['log']) df2 = pd.DataFrame(list(pred_label), columns=['pred']) parsed_logs_df=pd.concat([df1,df2],ignore_index=True)

[soumya-ghosh-wdc]

Hi : I am almost at the last mile to make the code working.Can you please help why during the processing of the second prompt with 'Self' strategy I am getting the following error :

1it [00:08, 8.99s/it]prompt 2 1it [00:08, 8.99s/it] Traceback (most recent call last): File "C:\LogPrompt\LogPrompt_code.py", line 273, in parse_logs = parse_logs(OUTPUT_FILE,prompt_parts,prompt_parts_count) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: 'list' object is not callable

[soumya-ghosh-wdc]

Fixed the above error as well:

Changed the following :

parse_logs_result=[] parse_logs_result = parse_logs(OUTPUT_FILE,prompt_parts,prompt_parts_count)

[soumya-ghosh-wdc]

I will create a PR with all the changes and will submit for a merge.