Error when adding facets argument

[leighmatth]

Hi, I'm having an issue when I try to use the eia_get function when I'm passing in facets. Using the examples in the readme, I'm able to run the examples for df1, df2, and df3, but when I try to run the code for df4 where we pass in the facets, I get the following error:

facets <- list(respondent = "US48", fueltype = "NG")

df4 <- eia_get(
  api_key = api_key,
  api_path = "electricity/rto/fuel-type-data/data/",
  data = "value",
  length = 5000,
  format = "data.table",
  facets = facets
)

curl: (3) bad range in URL position 127:
https://api.eia.gov/v2/electricity/rto/fuel-type-data/data?data[]=value&facets[respondent][]=US48&facets[fueltype][]=NG&length=5000
                                                                                                                              ^
Error in eia_get(api_key = api_key, api_path = "electricity/rto/fuel-type-data/data/",  : 
  Could not pull the data... ❌
Check the query parameters (e.g., api key, path, etc.) or the error log

(Note: I removed my api_key from the error message)

I've tried a few other calls with the facets argument and always get that same error. Any thoughts?

[RamiKrispin]

Hi @leighmatth, could you please send me the query you are trying to pull via the API dashboard?

https://www.eia.gov/opendata/browser/

[RamiKrispin]

@leighmatth btw, your query works on my end. Could you please check if your eia_key is valid?

> library(EIAapi)
> facets <- list(respondent = "US48", fueltype = "NG")
> df4 <- eia_get(
+   api_key = api_key,
+   api_path = "electricity/rto/fuel-type-data/data/",
+   data = "value",
+   length = 5000,
+   format = "data.table",
+   facets = facets
+ )
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  864k    0  864k    0     0   686k      0 --:--:--  0:00:01 --:--:--  686k
> df4
             period respondent        respondent-name fueltype   type-name  value
   1: 2019-09-03T02       US48 United States Lower 48       NG Natural gas 238622
   2: 2019-09-02T08       US48 United States Lower 48       NG Natural gas 149180
   3: 2019-09-02T06       US48 United States Lower 48       NG Natural gas 164671
   4: 2019-09-03T03       US48 United States Lower 48       NG Natural gas 217190
   5: 2019-09-02T16       US48 United States Lower 48       NG Natural gas 206225
  ---                                                                            
4996: 2018-11-09T20       US48 United States Lower 48       NG Natural gas 145412
4997: 2018-11-09T21       US48 United States Lower 48       NG Natural gas 147572
4998: 2018-11-07T11       US48 United States Lower 48       NG Natural gas 109269
4999: 2018-11-07T12       US48 United States Lower 48       NG Natural gas 123157
5000: 2018-11-07T13       US48 United States Lower 48       NG Natural gas 132755
        value-units
   1: megawatthours
   2: megawatthours
   3: megawatthours
   4: megawatthours
   5: megawatthours
  ---              
4996: megawatthours
4997: megawatthours
4998: megawatthours
4999: megawatthours
5000: megawatthours

[leighmatth]

Hi @RamiKrispin

This is the API query I'm trying to pull: https://api.eia.gov/v2/electricity/facility-fuel/data/?frequency=monthly&data[0]=average-heat-content&data[1]=consumption-for-eg&data[2]=generation&data[3]=gross-generation&facets[plantCode][]=6137&facets[fuel2002][]=BIT&facets[primeMover][]=ST&start=2013-01&end=2023-05&sort[0][column]=period&sort[0][direction]=asc&offset=0&length=5000

I tested the readme example again and the one with the facets is still producing the same error for me. I double checked my API key, and it is active and valid, and works for other cases when I don't pass in facets.

Based on the error I'm getting and some googling, it seems like this might be a globbing issue with curl related to using a Mac (I'm running MacOS v12.5.1).

[RamiKrispin]

@leighmatth could you please check if you have jq installed?

You can check on your terminal:

jq --version

[leighmatth]

@leighmatth could you please check if you have jq installed?

You can check on your terminal:

jq --version

@RamiKrispin I'm running jq-1.6

[leighmatth]

Hi @RamiKrispin I made the following change on line 114 of query.R, and I'm now able to run those commands without getting an error.

Changed:

      f<-paste(f,
               sprintf("&facets[%s][]=%s", i, l),
               sep="")

to:

      f <- paste(f,
                 sprintf("&facets%%5B%s%%5D[]=%s", i, l),
                 sep = "")

[RamiKrispin]

Thanks, I was able to reproduce the error - I developed the code and test it on Mac but inside a dockerized environment. It works fine on the dockerized environment (e.g., Linux) but fails on MacOS. I am going to look for a solution that is OS agnostic and push update the CRAN.

[leighmatth]

Great, thank you!

[RamiKrispin]

The issue was fixed by adding to the curl command the -g argument. This should be available in CRAN (version 0.1.2) in the coming days. Thanks for the feedback!

[RamiKrispin]

@leighmatth, the fix is now on CRAN (v0.1.2), could you please reinstall and check if this work on your end?

[leighmatth]

@RamiKrispin I reinstalled with the new version from CRAN, and everything seems to be working now. Thank you!

[RamiKrispin]

Great, thanks again for the feedback!