Closed Softdev1 closed 1 month ago
The raw ICP content is filled with unnecessary tags and keywords which increase the number of tokens for the model input which is not cost efficient and also the titles are absurd which needs to be fixed before indexing
Description
The raw ICP content is filled with unnecessary tags and keywords which increase the number of tokens for the model input which is not cost efficient and also the titles are absurd which needs to be fixed before indexing