search

matthewjrogers / rairtable

Efficient, Tidyverse-friendly wrapper for the Airtable API
https://matthewjrogers.github.io/rairtable/
Other
13 stars 4 forks source link

Error in reading User field types #4

Closed Tyr30 closed 1 year ago

Tyr30 commented 2 years ago

Getting this error when trying to read_airtable:

Error in data.table::rbindlist(dta, use.names = TRUE, fill = TRUE) : Column 9 of item 1 is length 3 inconsistent with column 1 which is length 100. Only length-1 columns are recycled.

Error seems to correspond with "User" field types. I know that the airtabler package splits such fields into three columns: User name, email address, and something else I'm not remembering right now. Maybe error is related to this oddity of airtable, where they have one field that's actually 3?

matthewjrogers commented 2 years ago

Thanks @Tyr30 I will take a look at this and see how airtabler handles user field types and try to re-create locally. Hoping to block some time for bug fixes in the next couple weeks

matthewjrogers commented 2 years ago

@Tyr30 I think this ended up being relatively straightforward. The user fields are returned from the API as a JSON containing a user ID, an email, and a name. I switched to using dplyr::bind_rows under the hood which natively flattened this structure out into 3 columns FIELDNAME.id, FIELDNAME.email, and FIELDNAME.name.

Worth noting, the presence of a user type field is going to cause insert_records to choke because the names do not match. update_records will also fail for this data type, though you should still be able to update fields that are not of the user type. It's probable that there is some amount of tracking, processing, or handling that could be done to support these features for the user type, but my sense is that it would be a significant can of worms. I've settled for now for a warning message.

You can install the dev version with this functionality using remotes::install_github('matthewjrogers/rairtable', ref = 'dev'). Please let me know if any new issues surface -- I have a vague memory that the choice to use rbindlist over bind_rows was deliberate, but I wasn't able to turn up any issues.

In the absence of breaking changes, this version will go to CRAN in the next couple of months.

matthewjrogers commented 2 years ago

Actually, going to leave this open for documentation re. user fields

Tyr30 commented 2 years ago

Finally managed to try this out and everything downloaded with no errors. Fortunately I don't need to insert or update records with this script yet. Thanks so much @matthewjrogers!

timothoms commented 1 year ago

Thank for you for this very useful package.

I come up against the error message that started this thread, not for a "user" field type, but for a "multiple select" field. This makes intuitive sense because this field type allows different lengths for this field for any given record. I'm not sure what a good a solution would be to deal with this, but would be great if there was a way around it. For instance, there could be an option to exclude such fields with a warning.

matthewjrogers commented 1 year ago

@timothoms I suggest trying to install the dev branch of this package from GitHub using remotes::install_github('matthewjrogers/rairtable', ref = 'dev'). I did a quick test reading in a multiple select field with the development version and it seemed to work just fine.

Hopefully that helps -- I intend to push changes to CRAN in the next month or so when I get a chance.

timothoms commented 1 year ago

That works, great. I tried this before, unsuccessfully, because I had not uninstallled the first package version, but it works now. Thanks again.