search

mayooear / gpt4-pdf-chatbot-langchain

GPT4 & LangChain Chatbot for large PDF docs
https://www.youtube.com/watch?v=ih9PBGVVOO4
14.74k stars 3k forks source link

Failed to ingest data error:Network error #164

Closed yizhanchengyan closed 9 months ago

yizhanchengyan commented 1 year ago

Hello，this is my error log： Document { pageContent: 'whether a variable is more or less important when constructing the decision tree. \n' + 'Yeh et al. (2014) find that random forest techniques are very robust and allow for the presence of \n' + 'outliers and noise in the training set. JPMorgan researchers consider that random forest shows \n' + 'promise for trading 10-year US Treasury market instruments. Medeiros et al. (2017) recognise the \n' + 'nonlinearity property with inflation dynamics and compare 16 ML methods against benchmark \n' + 'statistical models. In general, ML methods with a large set of covariates provide superior results \n' + 'compared to univariate benchmarks and factor models. Medeiros et al. (2017) find that random forests \n' + 'is the best model indicating a degree of nonlinearity in the dynamics of inflation. Using ML applications \n' + 'in bankruptcy prediction, Barboza et al. (2017) find that random forest techniques outperform other \n' + 'methods. \n' + '5.2.3. Neural networks', metadata: { source: 'D:\gpt4-pdf-chatbot-langchain-main\docs\finance\turingfinance.pdf', pdf_numpages: 50 } }, ... 252 more items ] creating vector store... error [Error: Network Error] { config: { transitional: { silentJSONParsing: true, forcedJSONParsing: true, clarifyTimeoutError: false }, adapter: [AsyncFunction: fetchAdapter], transformRequest: [ [Function: transformRequest] ], transformResponse: [ [Function: transformResponse] ], timeout: 0, xsrfCookieName: 'XSRF-TOKEN', xsrfHeaderName: 'X-XSRF-TOKEN', maxContentLength: -1, maxBodyLength: -1, validateStatus: [Function: validateStatus], headers: { Accept: 'application/json, text/plain, /', 'Content-Type': 'application/json', 'User-Agent': 'OpenAI/NodeJS/3.2.1', Authorization: 'Bearer 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' }, method: 'post', data: {"model":"text-embedding-ada-002","input":["Artificial intelligence in finance Bonnie G. Buchanan, PhD, FRSA https://doi.org/10.5281/zenodo.2612537 This work was supported by The Alan Turing Institute under the EPSRC grant EP/N510129/1","Artificial intelligence in finance April 201 9 Bonnie G. Buchan an, PhD, FRSA Howard Bosanko Professor of Economics and Finance Department of Finance, Albers School of Business and Economics Seattle University Seattle, Washington 98122-1090 Email: buchanab@seattleu.edu Ph: (206) 296-5977 Hanken School of Economics Department of Finance, Statistics and Economics P.O. Box 479, FI-00101 Helsinki, Finland A bstract Artificial intelligence (AI) is rapidly transforming the global financial services industry. As a group of related technologies that include machine learning (ML) and deep learning (DL), AI has the potential to disrupt and refine the existing financial services industry. I review the extant academic, practitioner and policy related AI literature. I also detail the AI, ML and DL taxonomy as well as their various applications in the financial services industry.","and policy related AI literature. I also detail the AI, ML and DL taxonomy as well as their various applications in the financial services industry. A literature survey of AI and financial services cannot ignore the econometric aspects and their implications. ML methods are all about algorithms, rather than asymptotic statistical processes. Unlike maximum likelihood estimation, ML’s framework is less unified. To that end, I will discuss the ML approaches of unsupervised and supervised learning.","Contents 1. Introduction ..................................................................................................................................................................... 1 2. Taxonomy and historical overview of AI, ML and DL ..................................................................................... 4 3. Global growth of the AI industry ............................................................................................................................. 7 4. How AI is changing the financial services industry ..................................................................................... 11 4.1. Fraud detection and compliance .................................................................................................................... 11 4.2. Banking chatbots and robo-advisory services ............................................................................................ 13","4.2. Banking chatbots and robo-advisory services ............................................................................................ 13 4.3. Algorithmic trading .............................................................................................................................................. 15 4.4. Other applications of AI ...................................................................................................................................... 18 5. Econometrics versus ML ........................................................................................................................................ 19 5.1. Unsupervised machine learning ..................................................................................................................... 20 5.1.1. Clustering algorithms .................................................................................................................................. 20","5.1.1. Clustering algorithms .................................................................................................................................. 20 5.1.2. Topic models ................................................................................................................................................. 20 5.1.2. Topic models (cont.) .................................................................................................................................... 22 5.2. Supervised machine learning models ........................................................................................................... 22 5.2.1. Predictive analytics ...................................................................................................................................... 23 5.2.2. Random forests ............................................................................................................................................. 23","5.2.2. Random forests ............................................................................................................................................. 23 5.2.3. Neural networks ........................................................................................................................................... 23 5.2.4. Support vector machine (SVM) ................................................................................................................ 24 6. Machine learning versus quantum computing .............................................................................................. 25 7. Regulation and policy-making .............................................................................................................................. 26 8. Conclusion and directions for future research .............................................................................................. 29","8. Conclusion and directions for future research .............................................................................................. 29 Appendix ........................................................................................................................................................................... 30 Timeline of artificial intelligence milestones ....................................................................................................... 30 References ........................................................................................................................................................................ 32","1 “AI is the ‘new electricity’ ... just as electricity transformed many industries roughly one hundred years ago; AI will also now change every major industry.” Andrew Ng, 2007 “What we're seeing is something unprecedented, which is the arrival of artificial intelligence, which has a big impact ... it creates tremendous uncertainty and impacts different people differently ... a nd some people could be left out.\\" Robert Shiller, 2018 Davos Forum 1. Introduction In 1950, Alan Turing posed the question “Can machines think?” and since then artificial intelligence (hereafter known as AI) applications have met with varying degrees of success. However, in recent years there has been a resurgence of interest and AI has found innovative applications in the global financial services industry. The availability of big data, improved technology, cloud computing and","financial services industry. The availability of big data, improved technology, cloud computing and faster special purpose hardware have been key drivers of the latest AI innovation wave. AI capabilities and machine learning (ML) are boosting growth in the emerging Fintech market. Broadly speaking, the term “Fintech” describes the new technologies, services and companies that have changed financial services. It includes (but is not limited to): cryptocurrencies, blockchain1, robo-advising, smart contracts, crowdfunding, mobile payments and AI platforms. In 2017 AI topped the list as a key trend in financial services and Fintech (Future Today Institute, 2017). In this literature review, I will detail the AI, ML and deep learning (DL) taxonomy as well as their various applications in the financial services industry. I will summarise the current academic, practitioner and","applications in the financial services industry. I will summarise the current academic, practitioner and policy related AI literature. This includes drawing upon economic, finance and computer science literature as well as regulatory publications. I specifically discuss four ways in which AI is changing the financial services industry: (1) fraud detection (how AI is used to keep criminal funds out of the financial system); (2) banking chatbots; (3) algorithmic trading and (4) regulatory and policy aspects.","2 Professor John McCarthy coined the term “artificial intelligence” in 1955 and the term “machine learning” was coined in 1959 by Arthur Samuel of IBM. AI can mimic actions it has seen or previously have been taught about without any new intervention. ML is defined as a particular approach to AI able to take the data and algorithms and apply it to new scenarios and patterns without being programmed directly. Deep learning (DL) is viewed as a branch of ML. DL provides machines with algorithms necessary to understand the underlying principles of an action and significant portions of data. They can then be combined to learn on their own and deepen the knowledge and skills with which they are provided. High frequency trading (HFT) and algorithmic trading use high speed communications and computer programs in the financial services industry. For at least a decade banks have been using ML to detect credit card fraud.","computer programs in the financial services industry. For at least a decade banks have been using ML to detect credit card fraud. The UK Financial Conduct Authority (FCA) is utilising ML to help individuals manage their current accounts. Approximately 9% of all hedge funds use ML to build large statistical models. In 2016, Aidyia launched an AI hedge fund to make all... 296414 more characters, url: 'https://api.openai.com/v1/embeddings' }, code: 'ERR_NETWORK', request: Request { [Symbol(realm)]: { settingsObject: [Object] },

  method: 'POST',
  localURLsOnly: false,
  unsafeRequest: false,
  body: [Object],
  client: [Object],
  reservedClient: null,
  replacesClientId: '',
  window: 'client',
  keepalive: false,
  serviceWorkers: 'all',
  initiator: '',
  destination: '',
  priority: null,
  origin: 'client',
  policyContainer: 'client',
  referrer: 'client',
  referrerPolicy: '',
  mode: 'cors',
  useCORSPreflightFlag: false,
  credentials: 'same-origin',
  useCredentials: false,
  cache: 'default',
  redirect: 'follow',
  integrity: '',
  cryptoGraphicsNonceMetadata: '',
  parserMetadata: '',
  reloadNavigation: false,
  historyNavigation: false,
  userActivation: false,
  taintedOrigin: false,
  redirectCount: 0,
  responseTainting: 'basic',
  preventNoCacheCacheControlHeaderModification: false,
  done: false,
  timingAllowFailed: false,
  headersList: [HeadersList],
  urlList: [Array],
  url: [URL]
},
[Symbol(signal)]: AbortSignal { aborted: false },
[Symbol(headers)]: HeadersList {
  cookies: null,
  [Symbol(headers map)]: [Map],
  [Symbol(headers map sorted)]: null
}

}, response: undefined, isAxiosError: true, toJSON: [Function: toJSON] } d:\gpt4-pdf-chatbot-langchain-main\scripts\ingest-data.ts:44 throw new Error('Failed to ingest your data'); ^

[Error: Failed to ingest your data]

Node.js v18.15.0  ELIFECYCLE  Command failed with exit code 1.

I use a VPN to switch to a US IP to run this program. Is it possible that if I use a VPN to inject data into pinecone, a network error will occur and fail. How to solve it? Thanks

codica commented 1 year ago

change other vpn server，and try serveral more

yizhanchengyan commented 1 year ago

change other vpn server，and try serveral more

thanks very useful

LilacZhang commented 1 year ago

I have encountered a similar problem with you. What vpn service did you use to make pinecone access it normally?

jliu513612 commented 1 year ago

change other vpn server，and try serveral more

thanks very useful

Hi! Have your problem solved yet? I am also using VPN service in China and I meet the same problem as you do. I have tried many times and it still doesn't work.

cason0126 commented 1 year ago

I have the same error. but my vpn can success use openai api with curl example .

LilacZhang commented 1 year ago

I use enhance mode in ClashX, set the Proxy forwarding in terminal, and make sure the VPN is available for openai.com, it seems fix my problem

2023年4月17日 14:39，jliu513612 @.***> 写道：

change other vpn server，and try serveral more

thanks very useful

Hi! Have your problem solved yet? I am also using VPN service in China and I meet the same problem as you do. I have tried many times and it still doesn't work.

— Reply to this email directly, view it on GitHub https://github.com/mayooear/gpt4-pdf-chatbot-langchain/issues/164#issuecomment-1510786808, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKLLISGXOBMPMICELVJ2UZLXBTQSBANCNFSM6AAAAAAW5GRRNU. You are receiving this because you commented.

spectramaster commented 1 year ago

same problem

cason0126 commented 1 year ago

https://docs.helicone.ai/ may good choiche

billyjingxuan commented 1 year ago

Did you solve the problem? I'm facing the same issue.

jliu513612 commented 1 year ago

Did you solve the problem? I'm facing the same issue.

If you are in China, Proxifier will be good. But they only offer thirty-day evaluation. After it you need to buy it.

dosubot[bot] commented 9 months ago

Hi, @yizhanchengyan! I'm Dosu, and I'm helping the gpt4-pdf-chatbot-langchain team manage their backlog. I wanted to let you know that we are marking this issue as stale.

Based on my understanding, you are experiencing a network error when trying to ingest data into Pinecone using a VPN with a US IP. There have been some suggestions from other users, such as changing VPN servers and trying different VPN services. One user mentioned that they were able to fix the issue by using the enhance mode in ClashX and setting the Proxy forwarding in the terminal. Another user recommended using Proxifier if in China. However, it seems that the issue remains unresolved for you and another user.

Before we close this issue, we wanted to check if it is still relevant to the latest version of the gpt4-pdf-chatbot-langchain repository. If it is, please let us know by commenting on this issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days.

Thank you for your understanding and cooperation!