mendableai / firecrawl

🔥 Turn entire websites into LLM-ready markdown or structured data. Scrape, crawl and extract with a single API.
https://firecrawl.dev
GNU Affero General Public License v3.0
19.26k stars 1.5k forks source link

[Bug] JSONDecodeError for Firecrawl #912

Open zahra-teb opened 1 week ago

zahra-teb commented 1 week ago

Describe the Bug The following code encounters JSONDecodeError: Expecting value: line 2 column 1 (char 1)

from firecrawl import FirecrawlApp

app = FirecrawlApp(api_key='fc-*********************')

response = app.scrape_url(url='https://www.zoomit.ir/review/416149-samsung-galaxy-book-3-ultra-review/', params={
    'formats': [ 'markdown' ],
})

{ "name": "JSONDecodeError", "message": "Expecting value: line 2 column 1 (char 1)", "stack": "--------------------------------------------------------------------------- JSONDecodeError Traceback (most recent call last) File ~/newest_software/software/myenv/lib/python3.11/site-packages/requests/models.py:974, in Response.json(self, kwargs) 973 try: --> 974 return complexjson.loads(self.text, kwargs) 975 except JSONDecodeError as e: 976 # Catch JSON-related errors and raise as requests.JSONDecodeError 977 # This aliases json.JSONDecodeError and simplejson.JSONDecodeError

File /usr/lib/python3.11/json/init.py:346, in loads(s, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw) 343 if (cls is None and object_hook is None and 344 parse_int is None and parse_float is None and 345 parse_constant is None and object_pairs_hook is None and not kw): --> 346 return _default_decoder.decode(s) 347 if cls is None:

File /usr/lib/python3.11/json/decoder.py:337, in JSONDecoder.decode(self, s, _w) 333 \"\"\"Return the Python representation of s (a str instance 334 containing a JSON document). 335 336 \"\"\" --> 337 obj, end = self.raw_decode(s, idx=_w(s, 0).end()) 338 end = _w(s, end).end()

File /usr/lib/python3.11/json/decoder.py:355, in JSONDecoder.raw_decode(self, s, idx) 354 except StopIteration as err: --> 355 raise JSONDecodeError(\"Expecting value\", s, err.value) from None 356 return obj, end

JSONDecodeError: Expecting value: line 2 column 1 (char 1)

During handling of the above exception, another exception occurred:

JSONDecodeError Traceback (most recent call last) Cell In[1], line 6 2 from firecrawl import FirecrawlApp 4 app = FirecrawlApp(api_key='fc-***') ----> 6 response = app.scrape_url(url='https://www.zoomit.ir/review/416149-samsung-galaxy-book-3-ultra-review/', params={ 7 \t'formats': [ 'markdown' ], 8 })

File ~/newest_software/software/myenv/lib/python3.11/site-packages/firecrawl/firecrawl.py:89, in FirecrawlApp.scrape_url(self, url, params) 87 raise Exception(f'Failed to scrape URL. Error: {response}') 88 else: ---> 89 self._handle_error(response, 'scrape URL')

File ~/newest_software/software/myenv/lib/python3.11/site-packages/firecrawl/firecrawl.py:592, in FirecrawlApp._handle_error(self, response, action) 581 def _handle_error(self, response: requests.Response, action: str) -> None: 582 \"\"\" 583 Handle errors from API responses. 584 (...) 590 Exception: An exception with a message containing the status code and error details from the response. 591 \"\"\" --> 592 error_message = response.json().get('error', 'No error message provided.') 593 error_details = response.json().get('details', 'No additional error details provided.') 595 if response.status_code == 402:

File ~/newest_software/software/myenv/lib/python3.11/site-packages/requests/models.py:978, in Response.json(self, kwargs) 974 return complexjson.loads(self.text, kwargs) 975 except JSONDecodeError as e: 976 # Catch JSON-related errors and raise as requests.JSONDecodeError 977 # This aliases json.JSONDecodeError and simplejson.JSONDecodeError --> 978 raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)

JSONDecodeError: Expecting value: line 2 column 1 (char 1)" }

zahra-teb commented 1 week ago

Well, I found the reason :)) The response status is 403!