asg017 / sqlite-http

A SQLite extension for making HTTP requests purely in SQL
MIT License
229 stars 11 forks source link

Issue Filtering Non-Working URLs in SQLite Using http_get Methods #34

Open jobyjames1 opened 1 week ago

jobyjames1 commented 1 week ago

I am trying to filter out non-working URLs from my SQLite table using http_get methods. Here's what I attempted and the challenges I encountered:

1. Fetching Body Content:

I tried fetching the body content of some URLs using the http_get_bodymethod but encountered the following error:

 Error: Get "http://emailer.sbicard.com/TrackableURL/Confirmation.html": dial tcp: lookup emailer.sbicard.com on 192.168.128.1:53: no such host 

2. Filtering by Status Code:

To handle this issue, I decided to filter URLs based on their HTTP status code (e.g., 404). Here's the query I used:

select  
  iif(  
    response_status_code not between 200 and 299,  
    'An error happened!',  
    null  
  ) as status,  
  response_body  
from http_get("http://emailer.sbicard.com/TrackableURL/Confirmation.html");

However, this resulted in another error:

Error: SQL logic error

Steps to Reproduce:

  1. Use the following URL: http://emailer.sbicard.com/TrackableURL/Confirmation.html
  2. Run the query:
    select  
    iif(  
    response_status_code not between 200 and 299,  
    'An error happened!',  
    null  
    ) as status,  
    response_body  
    from http_get("http://emailer.sbicard.com/TrackableURL/Confirmation.html");  

    Expected Outcome:

Actual Outcome:

Additional Information: