search

masa-finance / user-support

Tracking and triaging of client issues across all Masa products
0 stars 0 forks source link

[Problem]: Running the docs - Can't see the endpoints as well as navigating to /swaggger/#/ shows a list of endpoints but prompts for Dev API key. #2

Open prakhar728 opened 1 week ago

prakhar728 commented 1 week ago

Environment

Local

Masa Component

Other (please specify in description)

Version

Application Version: v0.8.5

Peer ID

16Uiu2HAkzqrHRZQtHBpdiEiSYLVU8KL4LHArEALi2Mme3hQ1VkPN

What happened?

Trying to make the local binary installtion run, wanted to use the developer configuration to scrape twitter data without using my own twitter account.

I ran the binary installtion using make run and tried to search the docs for endpoints. Couldn't find any. Later on I came accross the /swagger/# that runs when running the masa node on my local computer. It prompts for a dev api key. Which I don't have.

Expected the list of endpoints that showed up to directly work without having to use an API key.

Steps to reproduce

  1. Follow the Protocol Binary Installation guide
  2. Navigate to http://localhost:8080/swagger/#/

Visibily can't make a request without dev key.

Relevant log output

No response

Additional context

No response

giovaroma commented 1 week ago

Hi @prakhar728, it seems that you did not enable API requests.

Please go to your .env and add the following: API_ENABLED=true

Also when you run the protocol node just to add another layer of higher success use: make run-api-enabled

prakhar728 commented 4 days ago

@giovaroma I've run the node using this.

My requests_list.json looks like:

[
  {
    "scraper": "XTwitterScraper",
    "endpoint": "data/twitter/tweets/recent",
    "priority": 5,
    "params": {
      "query": "#Bitcoin since:2023-01-01 until:2023-01-05",
      "count": 100
    }
  }
]

I ran running this command in my shell.

 masa-ai-cli process request_list.json
  1. Cannot see any data in the data folder where requests_list.json.
  2. In the cli I get the error 
    
2024-10-28 23:13:27 - QCManager - ERROR -  - _default_error_handler - /home/prakhar/Desktop/me/near/masa/.venv/lib/python3.13/site-packages/requests/models.py:978: Unexpected error in XTwitterConnection.handle_response: Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
File "/home/prakhar/Desktop/me/near/masa/.venv/lib/python3.13/site-packages/requests/models.py", line 974, in json
return complexjson.loads(self.text, **kwargs)
       ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.13/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
       ~~~~~~~~~~~~~~~~~~~~~~~^^^
File "/usr/local/lib/python3.13/json/decoder.py", line 344, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.13/json/decoder.py", line 362, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/prakhar/Desktop/me/near/masa/.venv/lib/python3.13/site-packages/masa_ai/tools/qc/error_handler.py", line 50, in wrapper return func(*args, **kwargs) File "/home/prakhar/Desktop/me/near/masa/.venv/lib/python3.13/site-packages/masa_ai/connections/xtwitter_connection.py", line 81, in handle_response return response.json()


  File "/home/prakhar/Desktop/me/near/masa/.venv/lib/python3.13/site-packages/requests/models.py", line 978, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
2024-10-28 23:13:27 - QCManager - ERROR - XTwitterScraper - scrape_tweets: API call failed: Expecting value: line 1 column 1 (char 0)
2024-10-28 23:13:27 - QCManager - ERROR -  - _default_error_handler - /home/prakhar/Desktop/me/near/masa/.venv/lib/python3.13/site-packages/masa_ai/tools/scrape/scrape_xtwitter.py:130: Unexpected error in XTwitterScraper.scrape_tweets: cannot access local variable 'response' where it is not associated with a value
Traceback (most recent call last):
  File "/home/prakhar/Desktop/me/near/masa/.venv/lib/python3.13/site-packages/masa_ai/tools/qc/error_handler.py", line 50, in wrapper
    return func(*args, **kwargs)
  File "/home/prakhar/Desktop/me/near/masa/.venv/lib/python3.13/site-packages/masa_ai/tools/scrape/scrape_xtwitter.py", line 130, in scrape_tweets
    new_records = self._handle_response(response, request_id, query, iteration_end_date, all_tweets, records_fetched)
                                        ^^^^^^^^
UnboundLocalError: cannot access local variable 'response' where it is not associated with a value
2024-10-28 23:13:27 - QCManager - ERROR - RequestRouter - route_request - /home/prakhar/Desktop/me/near/masa/.venv/lib/python3.13/site-packages/masa_ai/tools/scrape/scrape_xtwitter.py:130: Error in route_request for Query '#Bitcoin since:2023-01-01 until:2023-01-05' (ID: c0d16c61c0c891f89d05b556889d6302050a671c60f8934ff2474c94f5b99d3c): cannot access local variable 'response' where it is not associated with a value
Traceback (most recent call last):
  File "/home/prakhar/Desktop/me/near/masa/.venv/lib/python3.13/site-packages/masa_ai/orchestration/request_router.py", line 73, in route_request
    result = scraper.scrape_tweets(request_id, params['query'], params['count'])
  File "/home/prakhar/Desktop/me/near/masa/.venv/lib/python3.13/site-packages/masa_ai/tools/qc/error_handler.py", line 54, in wrapper
    return self._default_error_handler(e, func.__qualname__)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/home/prakhar/Desktop/me/near/masa/.venv/lib/python3.13/site-packages/masa_ai/tools/qc/error_handler.py", line 50, in wrapper
    return func(*args, **kwargs)
  File "/home/prakhar/Desktop/me/near/masa/.venv/lib/python3.13/site-packages/masa_ai/tools/scrape/scrape_xtwitter.py", line 130, in scrape_tweets
    new_records = self._handle_response(response, request_id, query, iteration_end_date, all_tweets, records_fetched)
                                        ^^^^^^^^
UnboundLocalError: cannot access local variable 'response' where it is not associated with a value
2024-10-28 23:13:27 - QCManager - ERROR - RequestRouter - route_request: Traceback: Traceback (most recent call last):
  File "/home/prakhar/Desktop/me/near/masa/.venv/lib/python3.13/site-packages/masa_ai/orchestration/request_router.py", line 73, in route_request
    result = scraper.scrape_tweets(request_id, params['query'], params['count'])
  File "/home/prakhar/Desktop/me/near/masa/.venv/lib/python3.13/site-packages/masa_ai/tools/qc/error_handler.py", line 54, in wrapper
    return self._default_error_handler(e, func.__qualname__)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/home/prakhar/Desktop/me/near/masa/.venv/lib/python3.13/site-packages/masa_ai/tools/qc/error_handler.py", line 50, in wrapper
    return func(*args, **kwargs)
  File "/home/prakhar/Desktop/me/near/masa/.venv/lib/python3.13/site-packages/masa_ai/tools/scrape/scrape_xtwitter.py", line 130, in scrape_tweets
    new_records = self._handle_response(response, request_id, query, iteration_end_date, all_tweets, records_fetched)
                                        ^^^^^^^^
UnboundLocalError: cannot access local variable 'response' where it is not associated with a value

2024-10-28 23:13:27 - QCManager - ERROR -  - _process_single_request: Error in request c0d16c61c0c891f89d05b556889d6302050a671c60f8934ff2474c94f5b99d3c: cannot access local variable 'response' where it is not associated with a value
2024-10-28 23:13:27 - QCManager - DEBUG - StateManager - update_request_state: Updating state for request ID: c0d16c61c0c891f89d05b556889d6302050a671c60f8934ff2474c94f5b99d3c, status: failed
2024-10-28 23:13:27 - QCManager - DEBUG - StateManager - _save_state: Saving state to file
2024-10-28 23:13:27 - QCManager - DEBUG - StateManager - _save_state: State saved successfully
2024-10-28 23:13:27 - QCManager - DEBUG - StateManager - update_request_state: State updated and saved for request c0d16c61c0c891f89d05b556889d6302050a671c60f8934ff2474c94f5b99d3c
2024-10-28 23:13:27 - QCManager - ERROR - RequestManager - _process_queue: Error processing request: cannot access local variable 'response' where it is not associated with a value
2024-10-28 23:13:27 - QCManager - INFO - : Completed processing all 1 requests
2024-10-28 23:13:27 - QCManager - INFO - Masa: Processed all requests

```
theMultitude commented 4 days ago

@prakhar728 Were you getting 504s before you ran into this error?

prakhar728 commented 3 days ago

No, just 500. I run into 500 everytime I make the request.

theMultitude commented 3 days ago

@prakhar728 500 means your not connecting to the network properly. Will you look at your configurations again and make sure they conform to these standards:

  1. Twitter Scraper Config Ensure your username and password is formatted properly.
  2. Protocol Environment Ensure your environment file has the categories needed.
  3. Testnet Bootnodes Ensure you have the bootnodes properly listed.
giovaroma commented 3 days ago

Can you please also share an example of your .env file so we can review it to make sure it looks correct @prakhar728