Yahoo finance is returning empty data?

[cotyreh]

Description

I think yahoo finance is returning empty data when retrieving options. I have this function below in a script that will sort out most issues early on and print NAs if there is some connection/server issue that will result in missing data:

main <- function() {
  cat("Starting main function execution...\n")
  tasks <- list(
    fetchOptionChainData = function() {
      cat("Fetching option chain data...\n")
      tryCatch({
        manage()
      }, error = function(e) {
        cat("Error in fetching option chain data:", e$message, "\n")
        return(NULL)
      })
    },
    fetchMarketData = function() {
      cat("Fetching market data...\n")
      tryCatch({
        fetch_market_data()
      }, error = function(e) {
        cat("Error in fetching market data:", e$message, "\n")
        return(NULL)
      })
    }
  )

  results <- mclapply(tasks, function(task) task(), mc.cores = 2)
  cat("Data fetched. Processing...\n")

  option_data <- results$fetchOptionChainData
  market_data <- results$fetchMarketData

  if (!is.null(option_data) && !is.null(market_data)) {
    cat("Data is valid, processing options...\n")
    processed_calls <- process_options(option_data$calls, "call", market_data)
    processed_puts <- process_options(option_data$puts, "put", market_data)
    cat("Options processed. Applying adjustments...\n")

    adjusted_calls <- apply_adjustments(processed_calls, "call", market_data)
    adjusted_puts <- apply_adjustments(processed_puts, "put", market_data)
    cat("Adjustments applied. Saving data...\n")

    calculate_and_save_GEX(adjusted_calls, adjusted_puts, market_data)
    cat("Data saved. Process complete.\n")

    assign("adjusted_calls", adjusted_calls, envir = .GlobalEnv)
    assign("adjusted_puts", adjusted_puts, envir = .GlobalEnv)
  } else {
    cat("Data is null, checking issues...\n")
    calculate_and_save_GEX(NULL, NULL, NULL)
  }
}

Normally, this will catch most issues. As of a few days ago, however, yahoo is sometimes not returning data even when making a successful connection. I noticed I was missing data, but my script wasn't writing NAs into those spots to tell me one of the normal errors had occurred. This is what is printed in the log when that happens:

Attaching package: 'dplyr'

The following objects are masked from 'package:xts':

    first, last

The following objects are masked from 'package:stats':

    filter, lag

The following objects are masked from 'package:base':

    intersect, setdiff, setequal, union

Starting main function execution...
Fetching option chain data...
Fetching market data...
Warning: no data for '^SPX' expiry , omitting
    (server response: Timeout was reached: [query2.finance.yahoo.com] SSL connection timeout)
Data fetched. Processing...
Data is valid, processing options...
Error in UseMethod("mutate") : 
  no applicable method for 'mutate' applied to an object of class "NULL"
Calls: main -> process_options -> %>% -> mutate -> mutate
Execution halted

It looks like yahoo returns an empty data frame. The script will go on to calculate the greeks if there wasn't an error and it has the data to work with, but in these cases it is moving on thinking it has data but failing later on. It runs once every minute and rarely found issues like this for a couple months now. This is a new one that has been happening every 5 to 10 minutes for a few days. I imagine it's something happening on yahoo's end?

[joshuaulrich]

I can't run your code and spx <- quantmod::getOptionChain("^SPX") works for me. Can you give me a minimal example that only uses quantmod functions so I investigate?

Also, Yahoo seems to be having issues with historical price data too (#417), so this may not be something I can fix in quantmod.

[cotyreh]

It looks like you are correct. I tried this morning to recreate the issue and it seems to have resolved itself.

[joshuaulrich]

Closing this because it was an issue on Yahoo's end. Nothing to do in quantmod.