Closed michaelthwan closed 1 year ago
Currently using p extraction which have many useless text (sentences) in the text_df. Need to trim only useful one. BTW if it is still too long, the get_prompt will trim it using config's prompt_length_limit
p
text
text_df
get_prompt
prompt_length_limit
Checklist
@peterhcyuen used trafilatura to extract. Merged.
Related tuning #35
Possibly close. Not bad now @peterhcyuen
Currently using
p
extraction which have many uselesstext
(sentences) in thetext_df
. Need to trim only useful one. BTW if it is still too long, theget_prompt
will trim it using config'sprompt_length_limit
Checklist