arthur-shaw / susoapi

R interface for Survey Solutions' APIs
https://arthur-shaw.github.io/susoapi/
Other
9 stars 5 forks source link

`get_interviews()` error #34

Open ashwinikalantri opened 1 year ago

ashwinikalantri commented 1 year ago

The get_interviews() function randomly spits out this error. The function works when rerun. On further testing, this seems to happen on the first run of a new R session.

Could the number of interviews have anything to do with this? This was probably working fine, until the number of interviews crossed 1000.

I tried changing the chunk size, lowering it to 50 didn't resolve the issue. Increasing it to 200 seemed to have stopped the error, but only fetched 600 out of the ~1100 interviews. The function seems to be pulling only 100 interviews per chunk (chunk_size = 500 fetched 300 interviews, chunk_size = 1000 fetched 200 interviews and so on...)

interviews <- get_interviews()
Error in get_interviews_by_chunk(workspace = workspace, take_n = chunk_size,  : 
  Value cannot be null. (Parameter 'provider')
Value cannot be null. (Parameter 'provider')
Value cannot be null. (Parameter 'provider')
Value cannot be null. (Parameter 'provider')
Value cannot be null. (Parameter 'provider')
Value cannot be null. (Parameter 'provider')
Value cannot be null. (Parameter 'provider')
Value cannot be null. (Parameter 'provider')
Value cannot be null. (Parameter 'provider')
Value cannot be null. (Parameter 'provider')
Value cannot be null. (Parameter 'provider')
Value cannot be null. (Parameter 'provider')
Value cannot be null. (Parameter 'provider')
Value cannot be null. (Parameter 'provider')
Value cannot be null. (Parameter 'provider')
Value cannot be null. (Parameter 'provider')
Value cannot be null. (Parameter 'provider')
Value cannot be null. (Parameter 'provider')
Value cannot be null. (Parameter 'provider')
Value cannot be null. (Parameter 'provider')
Value cannot be null. (Parameter 'provider')
Value cannot be null. (Parameter 'provider')
V
arthur-shaw commented 1 year ago

Sorry for the odd and uninformative error.

A few questions to help reproduce:

Initially, I thought you were going to report this known issues, which seems to be a Survey Solutions issue. But your issue is new to me.

ashwinikalantri commented 1 year ago

I tried and failed to reproduce the error today on first try (on starting a new r session) and a few tries after that, but was able to reproduce after a few minutes. Very random.

The chunk size issue remains. Only 100 interviews are returned per chunk.