snowby666 / poe-api-wrapper

👾 A Python API wrapper for Poe.com. With this, you will have free access to GPT-4, Claude, Llama, Gemini, Mistral and more! 🚀
https://pypi.org/project/poe-api-wrapper/
GNU General Public License v3.0
687 stars 82 forks source link

Is there a way to grab output message citation content ? #96

Closed XenocodeRCE closed 6 months ago

XenocodeRCE commented 6 months ago

Hello,

API Wrapper works like a charm, easy to setup.

I trained variations of Claude using my own data in PDFs files.

Output message contains quotes from my PDFs.

Is it possible to grab the content of these :

[[1]](https://poe.com/citation?message_id=93885468122&citation=1) 

I'm scared because even when I try in normal webbrowser it doesn't fetch any data and I can access it only as a popup in the current convo on their website. Looks like a JavaScript pop up.

snowby666 commented 6 months ago

Hi, Thanks for your suggestion. I've just figured out how to implement this. I'll add it in the next update

XenocodeRCE commented 6 months ago

Nice, would you mind sharing some info ?

To be more precise I don't need to fetch the URLs using a regex for instance, I need to grab the content of the quotes produced by the bot when using custom database

snowby666 commented 6 months ago

Nice, would you mind sharing some info ?

To be more precise I don't need to fetch the URLs using a regex for instance, I need to grab the content of the quotes produced by the bot when using custom database

You can use the message id to fetch all the citations. The return will look like this:

 "citations":[
          {
             "__typename":"MessageKnowledgeBaseCitation",
             "__isMessageCitation":"MessageKnowledgeBaseCitation",
             "index":0,
             "citationType":"knowledge_base",
             "title":"python data science cheat sheet.pdf",
             "text":"...[ 4.,  5.,  6.]]])]\n\nDataCamp\nLearn Python for Data Science Interactively\n\n 2\n\fData Wrangling\nwith pandas\nCheat Sheet\nhttp://pandas.pydata.org\n\nTidy Data \u2013 A foundation for wrangling in pandas\n\nF\n\nM\n\nA\n\nTidy data complements pandas\u2019s vectorized\noperations. pandas will automatically preserve\nobservations as you manipulate variables. No\nother format works as intuitively with pandas.\n\nM\n\n*\n\nA\n\nF\n\nF\n\nM\n\nA\n\nIn a tidy\ndata set:\n\n&\n\nEach variable is saved\nin its own column\n\nEach observation is\nsaved in its own row\n\nM\n\nA*\n\nSyntax \u2013 Creating DataFrames\n\nReshaping Data \u2013 Change the layout of a data set\n\na\n\n4\n\n5\n\n6\n\nb\n\n7\n\n8\n\n9\n\nc\n\n10\n\n11\n\n12\n\n1\n\n2\n\n3\n\ndf = pd.DataFrame...",
             "__isNode":"MessageKnowledgeBaseCitation",
             "id":"TWVzc2FnZUtub3dsZWRnZUJhc2VDaXRhdGlvbjo5MzcyMDk0Njg1Njow"
          },
          {
             "__typename":"MessageKnowledgeBaseCitation",
             "__isMessageCitation":"MessageKnowledgeBaseCitation",
             "index":1,
             "citationType":"knowledge_base",
             "title":"python data science cheat sheet.pdf",
             "text":"...Python For Data Science Cheat Sheet\nScikit-Learn\nLearn Python for data science Interactively at  www.DataCamp.com\n\nScikit-learn\nScikit-learn is an open source Python library that\nimplements a range of machine learning,\npreprocessing, cross-validation and visualization\nalgorithms using a unified interface.\n\nA Basic Example\n\n>>> from sklearn import neighbors, datasets, preprocessing\n>>> from sklearn.model_selection import train_test_split\n>>> from sklearn.metrics import accuracy_score\n>>> iris = datasets.load_iris()\n>>> X, y = iris.data[:, :2], iris.target\n>>> X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=33)\n>>> scaler = preprocessing.StandardScaler().fit(X_train)\n>>> X_train = scaler.transform(X_train)\n>>> X_test = scaler.transform(X_test)\n>>> knn = neighbors.KNeighborsClassifier(n_neighbors=...",
             "__isNode":"MessageKnowledgeBaseCitation",
             "id":"TWVzc2FnZUtub3dsZWRnZUJhc2VDaXRhdGlvbjo5MzcyMDk0Njg1Njox"
          },
          {
             "__typename":"MessageKnowledgeBaseCitation",
             "__isMessageCitation":"MessageKnowledgeBaseCitation",
             "index":2,
             "citationType":"knowledge_base",
             "title":"python data science cheat sheet.pdf",
             "text":"\n>>> print(rsearch.best_score_)\n\nDataCamp\nLearn Python for Data Science Interactively",
             "__isNode":"MessageKnowledgeBaseCitation",
             "id":"TWVzc2FnZUtub3dsZWRnZUJhc2VDaXRhdGlvbjo5MzcyMDk0Njg1Njoy"
          },
          {
             "__typename":"MessageKnowledgeBaseCitation",
             "__isMessageCitation":"MessageKnowledgeBaseCitation",
             "index":3,
             "citationType":"knowledge_base",
             "title":"python data science cheat sheet.pdf",
             "text":"...=\"arc3\"),)\n\nStep 1\n\nStep 2\n\n>>> import matplotlib.pyplot as plt\n>>> x = [1,2,3,4]\n>>> y = [10,20,25,30]\n>>> fig = plt.figure()\n>>> ax = fig.add_subplot(111)\n>>> ax.plot(x, y, color='lightblue', linewidth=3)\n>>> ax.scatter([2,4,6],\n               [5,15,25],\n               color='darkgreen',\n               marker='^')\n>>> ax.set_xlim(1, 6.5)\n>>> plt.savefig('foo.png')\n>>> plt.show()\n\nStep 6\n\nStep 3\n\nStep 3, 4\n\nMathtext\n\n>>> plt.title(r'$sigma_i=15$', fontsize=20)\nLimits, Legends & Layouts...",
             "__isNode":"MessageKnowledgeBaseCitation",
             "id":"TWVzc2FnZUtub3dsZWRnZUJhc2VDaXRhdGlvbjo5MzcyMDk0Njg1Njoz"
          },
          {
             "__typename":"MessageKnowledgeBaseCitation",
             "__isMessageCitation":"MessageKnowledgeBaseCitation",
             "index":4,
             "citationType":"knowledge_base",
             "title":"python data science cheat sheet.pdf",
             "text":"...a'] > 10, ['a','c']]\n\nSelect rows meeting logical condition, and only the specific columns .\n\nhttp://pandas.pydata.org/ This cheat sheet inspired by Rstudio Data Wrangling Cheatsheet (https://www.rstudio.com/wp-content/uploads/2015/02/data-wrangling-cheatsheet.pdf)  Written by Irv Lustig, Princeton Consultants\n\n\fSummarize Data\n\nHandling Missing Data\n\nCombine Data Sets\n\ndf['w'].value_counts()\n\nCount number of rows with each unique value of variable\n\nlen(df)\n\n# of rows in DataFrame.\n\ndf['w'].nunique()\n\n# of distinct values in a column.\n\ndf.describe()\n\nBasic descriptive statistics for each column (or GroupBy)\n\ndf.dropna()\n\nDrop rows with any column having NA/null data.\n\ndf.fillna(value)\n\nReplace all NA/null data with value.\n\nMake New Columns\n\npandas provides a large set of summary functions that operate on...",
             "__isNode":"MessageKnowledgeBaseCitation",
             "id":"TWVzc2FnZUtub3dsZWRnZUJhc2VDaXRhdGlvbjo5MzcyMDk0Njg1Njo0"
          },
          {
             "__typename":"MessageKnowledgeBaseCitation",
             "__isMessageCitation":"MessageKnowledgeBaseCitation",
             "index":5,
             "citationType":"knowledge_base",
             "title":"python data science cheat sheet.pdf",
             "text":"...clustering algos\n\n>>> from sklearn.cross_validation import cross_val_score\n>>> print(cross_val_score(knn, X_train, y_train, cv=4))\n>>> print(cross_val_score(lr, X, y, cv=2))\n  Tune Your Model\nGrid Search\n\n  Preprocessing The Data\nStandardization\n\n>>> from sklearn.preprocessing import StandardScaler\n>>> scaler = StandardScaler().fit(X_train)\n>>> standardized_X = scaler.transform(X_train)\n>>> standardized_X_test = scaler.transform(X_test)\nNormalization\n\n>>> from sklearn.preprocessing import Normalizer\n>>> scaler = Normalizer().fit(X_train)\n>>> normalized_X = scaler.transform(X_train)\n>>> normalized_X_test = scaler.transform(X_test)\nBinarization\n\nEncoding Categorical Features\n\n>>> from sklearn.preprocessing import LabelEncoder\n>>> enc = LabelEncoder()\n>>> y = enc.fit_transform(y)\n\nImputing...",
             "__isNode":"MessageKnowledgeBaseCitation",
             "id":"TWVzc2FnZUtub3dsZWRnZUJhc2VDaXRhdGlvbjo5MzcyMDk0Njg1Njo1"
          }
       ],
XenocodeRCE commented 6 months ago

Really impressive ! I'll wait for the next update to be pushed. Have a nice day :-)