Closed biswaroop1547 closed 2 years ago
To convert a previously used csv into the new charon format use this snippet by putting the whole csv file path in CSV_PATH variable:
import pandas as pd
import json
import tqdm
CSV_PATH = ""
def new_charon_format_conversion(CSV_PATH):
df = pd.read_csv(CSV_PATH)
df.drop(columns=['tag'], inplace = True)
df['tag'] = df['intent'].values
for i, row in tqdm.tqdm(df.iterrows(), total = len(df)):
try:
intent_pred = json.loads(row['prediction'])['name']
except:
intent_pred = None
df.iloc[i, df.columns.get_loc('intent')] = intent_pred
df.to_csv(CSV_PATH[:-4] + 'new.csv', index = False)
print(f"{CSV_PATH} saved!")
new_charon_format_conversion(CSV_PATH)
currently the template uses
intent
column as labels for training and nottag
column whereasintent
column actually now model predictions for new charon's dataframe output format.more info