Closed twtdata closed 11 months ago
I was able to reproduce your error using this dataset: https://www.kaggle.com/datasets/eliasdabbas/twitter_dataframe?select=python_tweets.csv
The problem is not one of the API but of how the prompting is done. The prompt is long, abstract and reads a bit more like a statement than an order to the model. Using the regex code provided above, the counts return as:
`
`
With the new prompt First, using regex, make a column called 'hashtags' of a list of hashtags extracted from the 'tweet_full_text' column. Then, return the top 5 hashtags in the dataframe by frequency and show the frequency.
we see:
`
Entering new chain... Thought: To extract the hashtags from the 'tweet_full_text' column, I can use the
str.findall()
method with a regex pattern. Then, I can use thevalue_counts()
method to get the frequency of each hashtag and return the top 5.
Action: python_repl_ast
Action Input: df['hashtags'] = df['tweet_full_text'].str.findall(r'#\w+')
Observation:
Thought:I have created a new column called 'hashtags' in the dataframe df
which contains a list of hashtags extracted from the 'tweet_full_text' column.
Action: python_repl_ast Action Input: df['hashtags'].value_counts().head(5) Observation: [#CSharp, #HTML, #CSS, #python, #xcode, #SQL, #React, #PHP, #JavaScript, #swift, #ObjectiveC, #UI, #UX] 108 [#Python] 89 [] 63 [#python] 46 [#MachineLearning, #Python] 35 Name: hashtags, dtype: int64 Thought:Final Answer: The top 5 hashtags in the dataframe by frequency are:
Finished chain. 'The top 5 hashtags in the dataframe by frequency are:\n1. [#CSharp, #HTML, #CSS, #python, #xcode, #SQL, #React, #PHP, #JavaScript, #swift, #ObjectiveC, #UI, #UX] - 108 occurrences\n2. [#Python] - 89 occurrences\n3. [] - 63 occurrences\n4. [#python] - 46 occurrences\n5. [#MachineLearning, #Python] - 35 occurrences' `
The counts don't match but changing the prompt can serve as a good guide.
Hi, @twtdata! I'm Dosu, and I'm helping the LangChain team manage their backlog. I wanted to let you know that we are marking this issue as stale.
From what I understand, the issue you reported is about the re
module not being defined when trying to use regular expressions in a Python tool. User HashemAlsaket provided a dataset to reproduce the error and suggested that the problem might be with how the prompting is done. They also proposed changing the prompt as a possible solution. User batmanscode confirmed this solution by giving a thumbs up reaction to the comment.
Before we close this issue, we wanted to check with you if it is still relevant to the latest version of the LangChain repository. If it is, please let us know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days.
Thank you for your contribution to the LangChain repository!
System Info
I am using a create_pandas_dataframe_agent like so:
When I try to prompt it to do anything that requires a regex the output i get:
Observation: NameError: name 're' is not defined