API hangs when trying to return many fields

crew102 commented 5 years ago
Hi -
I'm the maintainer of the R package called patentsview, which provides an R client to the PatentsView API. I noticed some unexpected behavior where the API isn't responding to requests when the user is asking for many fields, especially when it comes to non-patents endpoints. This behavior is not deterministic, meaning sometimes I can get all the fields associated with the patents endpoint in a reasonable time frame, while other times I can't (though note I haven't been able to get all fields associated with the assignees endpoint at all, at least recently):
library(patentsview)
library(httr)

# timeout http request after 2 minutes
set_config(timeout(120), override = TRUE)

query <- "{\"patent_number\":\"5116621\"}"

# patents endpoint sometimes returns the complete set of fields in a timely fashion:
all_patents_fields <- get_fields("patents")
# fields being requested:
all_patents_fields
#>   [1] "appcit_app_number"                     
#>   [2] "appcit_category"                       
#>   [3] "appcit_date"                           
#>   [4] "appcit_kind"                           
#>   [5] "appcit_sequence"                       
#>   [6] "app_country"                           
#>   [7] "app_date"                              
#>   [8] "app_number"                            
#>   [9] "app_type"                              
#>  [10] "assignee_city"                         
#>  [11] "assignee_country"                      
#>  [12] "assignee_county"                       
#>  [13] "assignee_county_fips"                  
#>  [14] "assignee_first_name"                   
#>  [15] "assignee_first_seen_date"              
#>  [16] "assignee_id"                           
#>  [17] "assignee_last_name"                    
#>  [18] "assignee_last_seen_date"               
#>  [19] "assignee_lastknown_city"               
#>  [20] "assignee_lastknown_country"            
#>  [21] "assignee_lastknown_latitude"           
#>  [22] "assignee_lastknown_location_id"        
#>  [23] "assignee_lastknown_longitude"          
#>  [24] "assignee_lastknown_state"              
#>  [25] "assignee_latitude"                     
#>  [26] "assignee_location_id"                  
#>  [27] "assignee_longitude"                    
#>  [28] "assignee_organization"                 
#>  [29] "assignee_sequence"                     
#>  [30] "assignee_state"                        
#>  [31] "assignee_state_fips"                   
#>  [32] "assignee_total_num_inventors"          
#>  [33] "assignee_total_num_patents"            
#>  [34] "assignee_type"                         
#>  [35] "cited_patent_category"                 
#>  [36] "cited_patent_date"                     
#>  [37] "cited_patent_kind"                     
#>  [38] "cited_patent_number"                   
#>  [39] "cited_patent_sequence"                 
#>  [40] "cited_patent_title"                    
#>  [41] "citedby_patent_category"               
#>  [42] "citedby_patent_date"                   
#>  [43] "citedby_patent_kind"                   
#>  [44] "citedby_patent_number"                 
#>  [45] "citedby_patent_title"                  
#>  [46] "cpc_category"                          
#>  [47] "cpc_first_seen_date"                   
#>  [48] "cpc_group_id"                          
#>  [49] "cpc_group_title"                       
#>  [50] "cpc_last_seen_date"                    
#>  [51] "cpc_section_id"                        
#>  [52] "cpc_sequence"                          
#>  [53] "cpc_subgroup_id"                       
#>  [54] "cpc_subgroup_title"                    
#>  [55] "cpc_subsection_id"                     
#>  [56] "cpc_subsection_title"                  
#>  [57] "cpc_total_num_assignees"               
#>  [58] "cpc_total_num_inventors"               
#>  [59] "cpc_total_num_patents"                 
#>  [60] "detail_desc_length"                    
#>  [61] "examiner_first_name"                   
#>  [62] "examiner_id"                           
#>  [63] "examiner_last_name"                    
#>  [64] "examiner_role"                         
#>  [65] "examiner_group"                        
#>  [66] "forprior_country"                      
#>  [67] "forprior_date"                         
#>  [68] "forprior_docnumber"                    
#>  [69] "forprior_kind"                         
#>  [70] "forprior_sequence"                     
#>  [71] "govint_contract_award_number"          
#>  [72] "govint_org_id"                         
#>  [73] "govint_org_level_one"                  
#>  [74] "govint_org_level_two"                  
#>  [75] "govint_org_level_three"                
#>  [76] "govint_org_name"                       
#>  [77] "govint_raw_statement"                  
#>  [78] "inventor_city"                         
#>  [79] "inventor_country"                      
#>  [80] "inventor_county"                       
#>  [81] "inventor_county_fips"                  
#>  [82] "inventor_first_name"                   
#>  [83] "inventor_first_seen_date"              
#>  [84] "inventor_id"                           
#>  [85] "inventor_last_name"                    
#>  [86] "inventor_last_seen_date"               
#>  [87] "inventor_lastknown_city"               
#>  [88] "inventor_lastknown_country"            
#>  [89] "inventor_lastknown_latitude"           
#>  [90] "inventor_lastknown_location_id"        
#>  [91] "inventor_lastknown_longitude"          
#>  [92] "inventor_lastknown_state"              
#>  [93] "inventor_latitude"                     
#>  [94] "inventor_location_id"                  
#>  [95] "inventor_longitude"                    
#>  [96] "inventor_sequence"                     
#>  [97] "inventor_state"                        
#>  [98] "inventor_state_fips"                   
#>  [99] "inventor_total_num_patents"            
#> [100] "ipc_action_date"                       
#> [101] "ipc_class"                             
#> [102] "ipc_classification_data_source"        
#> [103] "ipc_classification_value"              
#> [104] "ipc_first_seen_date"                   
#> [105] "ipc_last_seen_date"                    
#> [106] "ipc_main_group"                        
#> [107] "ipc_section"                           
#> [108] "ipc_sequence"                          
#> [109] "ipc_subclass"                          
#> [110] "ipc_subgroup"                          
#> [111] "ipc_symbol_position"                   
#> [112] "ipc_total_num_assignees"               
#> [113] "ipc_total_num_inventors"               
#> [114] "ipc_version_indicator"                 
#> [115] "lawyer_first_name"                     
#> [116] "lawyer_first_seen_date"                
#> [117] "lawyer_id"                             
#> [118] "lawyer_last_name"                      
#> [119] "lawyer_last_seen_date"                 
#> [120] "lawyer_organization"                   
#> [121] "lawyer_sequence"                       
#> [122] "lawyer_total_num_assignees"            
#> [123] "lawyer_total_num_inventors"            
#> [124] "lawyer_total_num_patents"              
#> [125] "nber_category_id"                      
#> [126] "nber_category_title"                   
#> [127] "nber_first_seen_date"                  
#> [128] "nber_last_seen_date"                   
#> [129] "nber_subcategory_id"                   
#> [130] "nber_subcategory_title"                
#> [131] "nber_total_num_assignees"              
#> [132] "nber_total_num_inventors"              
#> [133] "nber_total_num_patents"                
#> [134] "patent_abstract"                       
#> [135] "patent_average_processing_time"        
#> [136] "patent_date"                           
#> [137] "patent_firstnamed_assignee_city"       
#> [138] "patent_firstnamed_assignee_country"    
#> [139] "patent_firstnamed_assignee_id"         
#> [140] "patent_firstnamed_assignee_latitude"   
#> [141] "patent_firstnamed_assignee_location_id"
#> [142] "patent_firstnamed_assignee_longitude"  
#> [143] "patent_firstnamed_assignee_state"      
#> [144] "patent_firstnamed_inventor_city"       
#> [145] "patent_firstnamed_inventor_country"    
#> [146] "patent_firstnamed_inventor_id"         
#> [147] "patent_firstnamed_inventor_latitude"   
#> [148] "patent_firstnamed_inventor_location_id"
#> [149] "patent_firstnamed_inventor_longitude"  
#> [150] "patent_firstnamed_inventor_state"      
#> [151] "patent_kind"                           
#> [152] "patent_num_cited_by_us_patents"        
#> [153] "patent_num_claims"                     
#> [154] "patent_num_combined_citations"         
#> [155] "patent_num_foreign_citations"          
#> [156] "patent_num_us_application_citations"   
#> [157] "patent_num_us_patent_citations"        
#> [158] "patent_number"                         
#> [159] "patent_processing_time"                
#> [160] "patent_title"                          
#> [161] "patent_type"                           
#> [162] "patent_year"                           
#> [163] "pct_102_date"                          
#> [164] "pct_371_date"                          
#> [165] "pct_date"                              
#> [166] "pct_docnumber"                         
#> [167] "pct_doctype"                           
#> [168] "pct_kind"                              
#> [169] "rawinventor_first_name"                
#> [170] "rawinventor_last_name"                 
#> [171] "uspc_first_seen_date"                  
#> [172] "uspc_last_seen_date"                   
#> [173] "uspc_mainclass_id"                     
#> [174] "uspc_mainclass_title"                  
#> [175] "uspc_sequence"                         
#> [176] "uspc_subclass_id"                      
#> [177] "uspc_subclass_title"                   
#> [178] "uspc_total_num_assignees"              
#> [179] "uspc_total_num_inventors"              
#> [180] "uspc_total_num_patents"                
#> [181] "wipo_field_id"                         
#> [182] "wipo_field_title"                      
#> [183] "wipo_sector_title"                     
#> [184] "wipo_sequence"

search_pv(
    query,
    endpoint = "patents",
    fields = all_patents_fields
)
#> http://www.patentsview.org/api/patents/query?q=%7B%22patent_number%22:%225116621%22%7D&f=[%22appcit_app_number%22,%22appcit_category%22,%22appcit_date%22,%22appcit_kind%22,%22appcit_sequence%22,%22app_country%22,%22app_date%22,%22app_number%22,%22app_type%22,%22assignee_city%22,%22assignee_country%22,%22assignee_county%22,%22assignee_county_fips%22,%22assignee_first_name%22,%22assignee_first_seen_date%22,%22assignee_id%22,%22assignee_last_name%22,%22assignee_last_seen_date%22,%22assignee_lastknown_city%22,%22assignee_lastknown_country%22,%22assignee_lastknown_latitude%22,%22assignee_lastknown_location_id%22,%22assignee_lastknown_longitude%22,%22assignee_lastknown_state%22,%22assignee_latitude%22,%22assignee_location_id%22,%22assignee_longitude%22,%22assignee_organization%22,%22assignee_sequence%22,%22assignee_state%22,%22assignee_state_fips%22,%22assignee_total_num_inventors%22,%22assignee_total_num_patents%22,%22assignee_type%22,%22cited_patent_category%22,%22cited_patent_date%22,%22cited_patent_kind%22,%22cited_patent_number%22,%22cited_patent_sequence%22,%22cited_patent_title%22,%22citedby_patent_category%22,%22citedby_patent_date%22,%22citedby_patent_kind%22,%22citedby_patent_number%22,%22citedby_patent_title%22,%22cpc_category%22,%22cpc_first_seen_date%22,%22cpc_group_id%22,%22cpc_group_title%22,%22cpc_last_seen_date%22,%22cpc_section_id%22,%22cpc_sequence%22,%22cpc_subgroup_id%22,%22cpc_subgroup_title%22,%22cpc_subsection_id%22,%22cpc_subsection_title%22,%22cpc_total_num_assignees%22,%22cpc_total_num_inventors%22,%22cpc_total_num_patents%22,%22detail_desc_length%22,%22examiner_first_name%22,%22examiner_id%22,%22examiner_last_name%22,%22examiner_role%22,%22examiner_group%22,%22forprior_country%22,%22forprior_date%22,%22forprior_docnumber%22,%22forprior_kind%22,%22forprior_sequence%22,%22govint_contract_award_number%22,%22govint_org_id%22,%22govint_org_level_one%22,%22govint_org_level_two%22,%22govint_org_level_three%22,%22govint_org_name%22,%22govint_raw_statement%22,%22inventor_city%22,%22inventor_country%22,%22inventor_county%22,%22inventor_county_fips%22,%22inventor_first_name%22,%22inventor_first_seen_date%22,%22inventor_id%22,%22inventor_last_name%22,%22inventor_last_seen_date%22,%22inventor_lastknown_city%22,%22inventor_lastknown_country%22,%22inventor_lastknown_latitude%22,%22inventor_lastknown_location_id%22,%22inventor_lastknown_longitude%22,%22inventor_lastknown_state%22,%22inventor_latitude%22,%22inventor_location_id%22,%22inventor_longitude%22,%22inventor_sequence%22,%22inventor_state%22,%22inventor_state_fips%22,%22inventor_total_num_patents%22,%22ipc_action_date%22,%22ipc_class%22,%22ipc_classification_data_source%22,%22ipc_classification_value%22,%22ipc_first_seen_date%22,%22ipc_last_seen_date%22,%22ipc_main_group%22,%22ipc_section%22,%22ipc_sequence%22,%22ipc_subclass%22,%22ipc_subgroup%22,%22ipc_symbol_position%22,%22ipc_total_num_assignees%22,%22ipc_total_num_inventors%22,%22ipc_version_indicator%22,%22lawyer_first_name%22,%22lawyer_first_seen_date%22,%22lawyer_id%22,%22lawyer_last_name%22,%22lawyer_last_seen_date%22,%22lawyer_organization%22,%22lawyer_sequence%22,%22lawyer_total_num_assignees%22,%22lawyer_total_num_inventors%22,%22lawyer_total_num_patents%22,%22nber_category_id%22,%22nber_category_title%22,%22nber_first_seen_date%22,%22nber_last_seen_date%22,%22nber_subcategory_id%22,%22nber_subcategory_title%22,%22nber_total_num_assignees%22,%22nber_total_num_inventors%22,%22nber_total_num_patents%22,%22patent_abstract%22,%22patent_average_processing_time%22,%22patent_date%22,%22patent_firstnamed_assignee_city%22,%22patent_firstnamed_assignee_country%22,%22patent_firstnamed_assignee_id%22,%22patent_firstnamed_assignee_latitude%22,%22patent_firstnamed_assignee_location_id%22,%22patent_firstnamed_assignee_longitude%22,%22patent_firstnamed_assignee_state%22,%22patent_firstnamed_inventor_city%22,%22patent_firstnamed_inventor_country%22,%22patent_firstnamed_inventor_id%22,%22patent_firstnamed_inventor_latitude%22,%22patent_firstnamed_inventor_location_id%22,%22patent_firstnamed_inventor_longitude%22,%22patent_firstnamed_inventor_state%22,%22patent_kind%22,%22patent_num_cited_by_us_patents%22,%22patent_num_claims%22,%22patent_num_combined_citations%22,%22patent_num_foreign_citations%22,%22patent_num_us_application_citations%22,%22patent_num_us_patent_citations%22,%22patent_number%22,%22patent_processing_time%22,%22patent_title%22,%22patent_type%22,%22patent_year%22,%22pct_102_date%22,%22pct_371_date%22,%22pct_date%22,%22pct_docnumber%22,%22pct_doctype%22,%22pct_kind%22,%22rawinventor_first_name%22,%22rawinventor_last_name%22,%22uspc_first_seen_date%22,%22uspc_last_seen_date%22,%22uspc_mainclass_id%22,%22uspc_mainclass_title%22,%22uspc_sequence%22,%22uspc_subclass_id%22,%22uspc_subclass_title%22,%22uspc_total_num_assignees%22,%22uspc_total_num_inventors%22,%22uspc_total_num_patents%22,%22wipo_field_id%22,%22wipo_field_title%22,%22wipo_sector_title%22,%22wipo_sequence%22]&o=%7B%22include_subentity_total_counts%22:false,%22matched_subentities_only%22:true,%22page%22:1,%22per_page%22:25%7D&s=
#> $data
#> #### A list with a single data frame (with list column(s) inside) on a patent level:
#> 
#> List of 1
#>  $ patents:'data.frame': 1 obs. of  47 variables:
#>   ..$ detail_desc_length                    : chr "11598"
#>   ..$ patent_abstract                       : chr "An anti-inflammatory "..
#>   ..$ patent_average_processing_time        : chr "820"
#>   ..$ patent_date                           : chr "1992-05-26"
#>   ..$ patent_firstnamed_assignee_city       : chr "Tokyo"
#>   ..$ patent_firstnamed_assignee_country    : chr "JP"
#>   ..$ patent_firstnamed_assignee_id         : chr "org_tPNZrwMT6w8Llh6MS"..
#>   ..$ patent_firstnamed_assignee_latitude   : chr "35.685"
#>   ..$ patent_firstnamed_assignee_location_id: chr "35.685|139.7514"
#>   ..$ patent_firstnamed_assignee_longitude  : chr "139.751"
#>   ..$ patent_firstnamed_assignee_state      : logi NA
#>   ..$ patent_firstnamed_inventor_city       : chr "Ohtsu"
#>   ..$ patent_firstnamed_inventor_country    : chr "JA"
#>   ..$ patent_firstnamed_inventor_id         : chr "5082663-2"
#>   ..$ patent_firstnamed_inventor_latitude   : chr "33.3103"
#>   ..$ patent_firstnamed_inventor_location_id: chr "33.3103|130.5125"
#>   ..$ patent_firstnamed_inventor_longitude  : chr "130.512"
#>   ..$ patent_firstnamed_inventor_state      : logi NA
#>   ..$ patent_kind                           : chr "A"
#>   ..$ patent_num_cited_by_us_patents        : chr "15"
#>   ..$ patent_num_claims                     : chr "9"
#>   ..$ patent_num_combined_citations         : chr "9"
#>   ..$ patent_num_foreign_citations          : chr "3"
#>   ..$ patent_num_us_application_citations   : chr "0"
#>   ..$ patent_num_us_patent_citations        : chr "6"
#>   ..$ patent_number                         : chr "5116621"
#>   ..$ patent_processing_time                : chr "519"
#>   ..$ patent_title                          : chr "Anti-inflammatory ana"..
#>   ..$ patent_type                           : chr "utility"
#>   ..$ patent_year                           : chr "1992"
#>   ..$ inventors                             :List of 1
#>   ..$ rawinventors                          :List of 1
#>   ..$ assignees                             :List of 1
#>   ..$ applications                          :List of 1
#>   ..$ IPCs                                  :List of 1
#>   ..$ application_citations                 :List of 1
#>   ..$ cited_patents                         :List of 1
#>   ..$ citedby_patents                       :List of 1
#>   ..$ uspcs                                 :List of 1
#>   ..$ cpcs                                  :List of 1
#>   ..$ nbers                                 :List of 1
#>   ..$ wipos                                 :List of 1
#>   ..$ gov_interests                         :List of 1
#>   ..$ lawyers                               :List of 1
#>   ..$ examiners                             :List of 1
#>   ..$ foreign_priority                      :List of 1
#>   ..$ pct_data                              :List of 1
#> 
#> $query_results
#> #### Distinct entity counts across all downloadable pages of output:
#> 
#> total_patent_count = 1

# ...but the assignees endpoint hasn't been able to return results for large
# sets of fields in a timely fashion:
all_assignees_endpoint_fields <- get_fields("assignees")
# fields requested:
all_assignees_endpoint_fields
#>   [1] "app_country"                           
#>   [2] "app_date"                              
#>   [3] "app_number"                            
#>   [4] "app_type"                              
#>   [5] "assignee_first_name"                   
#>   [6] "assignee_first_seen_date"              
#>   [7] "assignee_id"                           
#>   [8] "assignee_key_id"                       
#>   [9] "assignee_last_name"                    
#>  [10] "assignee_last_seen_date"               
#>  [11] "assignee_lastknown_city"               
#>  [12] "assignee_lastknown_country"            
#>  [13] "assignee_lastknown_latitude"           
#>  [14] "assignee_lastknown_location_id"        
#>  [15] "assignee_lastknown_longitude"          
#>  [16] "assignee_lastknown_state"              
#>  [17] "assignee_organization"                 
#>  [18] "assignee_total_num_inventors"          
#>  [19] "assignee_total_num_patents"            
#>  [20] "assignee_type"                         
#>  [21] "cpc_category"                          
#>  [22] "cpc_first_seen_date"                   
#>  [23] "cpc_group_id"                          
#>  [24] "cpc_group_title"                       
#>  [25] "cpc_last_seen_date"                    
#>  [26] "cpc_num_patents_for_assignee"          
#>  [27] "cpc_section_id"                        
#>  [28] "cpc_subgroup_id"                       
#>  [29] "cpc_subgroup_title"                    
#>  [30] "cpc_subsection_id"                     
#>  [31] "cpc_subsection_title"                  
#>  [32] "cpc_total_num_assignees"               
#>  [33] "cpc_total_num_inventors"               
#>  [34] "cpc_total_num_patents"                 
#>  [35] "detail_desc_length"                    
#>  [36] "examiner_first_name"                   
#>  [37] "examiner_id"                           
#>  [38] "examiner_last_name"                    
#>  [39] "examiner_role"                         
#>  [40] "examiner_group"                        
#>  [41] "forprior_country"                      
#>  [42] "forprior_date"                         
#>  [43] "forprior_docnumber"                    
#>  [44] "forprior_kind"                         
#>  [45] "forprior_sequence"                     
#>  [46] "govint_contract_award_number"          
#>  [47] "govint_org_id"                         
#>  [48] "govint_org_level_one"                  
#>  [49] "govint_org_level_two"                  
#>  [50] "govint_org_level_three"                
#>  [51] "govint_org_name"                       
#>  [52] "govint_raw_statement"                  
#>  [53] "inventor_city"                         
#>  [54] "inventor_country"                      
#>  [55] "inventor_county"                       
#>  [56] "inventor_county_fips"                  
#>  [57] "inventor_first_name"                   
#>  [58] "inventor_first_seen_date"              
#>  [59] "inventor_id"                           
#>  [60] "inventor_last_name"                    
#>  [61] "inventor_last_seen_date"               
#>  [62] "inventor_lastknown_city"               
#>  [63] "inventor_lastknown_country"            
#>  [64] "inventor_lastknown_latitude"           
#>  [65] "inventor_lastknown_location_id"        
#>  [66] "inventor_lastknown_longitude"          
#>  [67] "inventor_lastknown_state"              
#>  [68] "inventor_latitude"                     
#>  [69] "inventor_longitude"                    
#>  [70] "inventor_num_patents_for_assignee"     
#>  [71] "inventor_state"                        
#>  [72] "inventor_state_fips"                   
#>  [73] "inventor_total_num_patents"            
#>  [74] "ipc_action_date"                       
#>  [75] "ipc_class"                             
#>  [76] "ipc_classification_data_source"        
#>  [77] "ipc_classification_value"              
#>  [78] "ipc_first_seen_date"                   
#>  [79] "ipc_last_seen_date"                    
#>  [80] "ipc_main_group"                        
#>  [81] "ipc_section"                           
#>  [82] "ipc_subclass"                          
#>  [83] "ipc_subgroup"                          
#>  [84] "ipc_symbol_position"                   
#>  [85] "ipc_total_num_assignees"               
#>  [86] "ipc_total_num_inventors"               
#>  [87] "ipc_version_indicator"                 
#>  [88] "lawyer_first_name"                     
#>  [89] "lawyer_first_seen_date"                
#>  [90] "lawyer_id"                             
#>  [91] "lawyer_last_name"                      
#>  [92] "lawyer_last_seen_date"                 
#>  [93] "lawyer_organization"                   
#>  [94] "lawyer_sequence"                       
#>  [95] "lawyer_total_num_assignees"            
#>  [96] "lawyer_total_num_inventors"            
#>  [97] "lawyer_total_num_patents"              
#>  [98] "location_city"                         
#>  [99] "location_country"                      
#> [100] "location_county"                       
#> [101] "location_county_fips"                  
#> [102] "location_id"                           
#> [103] "location_latitude"                     
#> [104] "location_longitude"                    
#> [105] "location_state"                        
#> [106] "location_state_fips"                   
#> [107] "nber_category_id"                      
#> [108] "nber_category_title"                   
#> [109] "nber_first_seen_date"                  
#> [110] "nber_last_seen_date"                   
#> [111] "nber_num_patents_for_assignee"         
#> [112] "nber_subcategory_id"                   
#> [113] "nber_subcategory_title"                
#> [114] "nber_total_num_assignees"              
#> [115] "nber_total_num_inventors"              
#> [116] "nber_total_num_patents"                
#> [117] "patent_abstract"                       
#> [118] "patent_date"                           
#> [119] "patent_firstnamed_assignee_city"       
#> [120] "patent_firstnamed_assignee_country"    
#> [121] "patent_firstnamed_assignee_id"         
#> [122] "patent_firstnamed_assignee_latitude"   
#> [123] "patent_firstnamed_assignee_location_id"
#> [124] "patent_firstnamed_assignee_longitude"  
#> [125] "patent_firstnamed_assignee_state"      
#> [126] "patent_firstnamed_inventor_city"       
#> [127] "patent_firstnamed_inventor_country"    
#> [128] "patent_firstnamed_inventor_id"         
#> [129] "patent_firstnamed_inventor_latitude"   
#> [130] "patent_firstnamed_inventor_location_id"
#> [131] "patent_firstnamed_inventor_longitude"  
#> [132] "patent_firstnamed_inventor_state"      
#> [133] "patent_kind"                           
#> [134] "patent_num_cited_by_us_patents"        
#> [135] "patent_num_claims"                     
#> [136] "patent_num_combined_citations"         
#> [137] "patent_num_foreign_citations"          
#> [138] "patent_num_us_application_citations"   
#> [139] "patent_num_us_patent_citations"        
#> [140] "patent_number"                         
#> [141] "patent_title"                          
#> [142] "patent_type"                           
#> [143] "pct_102_date"                          
#> [144] "pct_371_date"                          
#> [145] "pct_date"                              
#> [146] "pct_docnumber"                         
#> [147] "pct_doctype"                           
#> [148] "pct_kind"                              
#> [149] "rawinventor_first_name"                
#> [150] "rawinventor_last_name"                 
#> [151] "uspc_first_seen_date"                  
#> [152] "uspc_last_seen_date"                   
#> [153] "uspc_mainclass_id"                     
#> [154] "uspc_mainclass_title"                  
#> [155] "uspc_num_patents_for_assignee"         
#> [156] "uspc_sequence"                         
#> [157] "uspc_subclass_id"                      
#> [158] "uspc_subclass_title"                   
#> [159] "uspc_total_num_assignees"              
#> [160] "uspc_total_num_inventors"              
#> [161] "wipo_field_id"                         
#> [162] "wipo_field_title"                      
#> [163] "wipo_sector_title"                     
#> [164] "wipo_sequence"                         
#> [165] "year_id"                               
#> [166] "year_num_patents_for_assignee"

search_pv(
    query,
    endpoint = "assignees",
    fields = all_assignees_endpoint_fields
)
#> http://www.patentsview.org/api/assignees/query?q=%7B%22patent_number%22:%225116621%22%7D&f=[%22app_country%22,%22app_date%22,%22app_number%22,%22app_type%22,%22assignee_first_name%22,%22assignee_first_seen_date%22,%22assignee_id%22,%22assignee_key_id%22,%22assignee_last_name%22,%22assignee_last_seen_date%22,%22assignee_lastknown_city%22,%22assignee_lastknown_country%22,%22assignee_lastknown_latitude%22,%22assignee_lastknown_location_id%22,%22assignee_lastknown_longitude%22,%22assignee_lastknown_state%22,%22assignee_organization%22,%22assignee_total_num_inventors%22,%22assignee_total_num_patents%22,%22assignee_type%22,%22cpc_category%22,%22cpc_first_seen_date%22,%22cpc_group_id%22,%22cpc_group_title%22,%22cpc_last_seen_date%22,%22cpc_num_patents_for_assignee%22,%22cpc_section_id%22,%22cpc_subgroup_id%22,%22cpc_subgroup_title%22,%22cpc_subsection_id%22,%22cpc_subsection_title%22,%22cpc_total_num_assignees%22,%22cpc_total_num_inventors%22,%22cpc_total_num_patents%22,%22detail_desc_length%22,%22examiner_first_name%22,%22examiner_id%22,%22examiner_last_name%22,%22examiner_role%22,%22examiner_group%22,%22forprior_country%22,%22forprior_date%22,%22forprior_docnumber%22,%22forprior_kind%22,%22forprior_sequence%22,%22govint_contract_award_number%22,%22govint_org_id%22,%22govint_org_level_one%22,%22govint_org_level_two%22,%22govint_org_level_three%22,%22govint_org_name%22,%22govint_raw_statement%22,%22inventor_city%22,%22inventor_country%22,%22inventor_county%22,%22inventor_county_fips%22,%22inventor_first_name%22,%22inventor_first_seen_date%22,%22inventor_id%22,%22inventor_last_name%22,%22inventor_last_seen_date%22,%22inventor_lastknown_city%22,%22inventor_lastknown_country%22,%22inventor_lastknown_latitude%22,%22inventor_lastknown_location_id%22,%22inventor_lastknown_longitude%22,%22inventor_lastknown_state%22,%22inventor_latitude%22,%22inventor_longitude%22,%22inventor_num_patents_for_assignee%22,%22inventor_state%22,%22inventor_state_fips%22,%22inventor_total_num_patents%22,%22ipc_action_date%22,%22ipc_class%22,%22ipc_classification_data_source%22,%22ipc_classification_value%22,%22ipc_first_seen_date%22,%22ipc_last_seen_date%22,%22ipc_main_group%22,%22ipc_section%22,%22ipc_subclass%22,%22ipc_subgroup%22,%22ipc_symbol_position%22,%22ipc_total_num_assignees%22,%22ipc_total_num_inventors%22,%22ipc_version_indicator%22,%22lawyer_first_name%22,%22lawyer_first_seen_date%22,%22lawyer_id%22,%22lawyer_last_name%22,%22lawyer_last_seen_date%22,%22lawyer_organization%22,%22lawyer_sequence%22,%22lawyer_total_num_assignees%22,%22lawyer_total_num_inventors%22,%22lawyer_total_num_patents%22,%22location_city%22,%22location_country%22,%22location_county%22,%22location_county_fips%22,%22location_id%22,%22location_latitude%22,%22location_longitude%22,%22location_state%22,%22location_state_fips%22,%22nber_category_id%22,%22nber_category_title%22,%22nber_first_seen_date%22,%22nber_last_seen_date%22,%22nber_num_patents_for_assignee%22,%22nber_subcategory_id%22,%22nber_subcategory_title%22,%22nber_total_num_assignees%22,%22nber_total_num_inventors%22,%22nber_total_num_patents%22,%22patent_abstract%22,%22patent_date%22,%22patent_firstnamed_assignee_city%22,%22patent_firstnamed_assignee_country%22,%22patent_firstnamed_assignee_id%22,%22patent_firstnamed_assignee_latitude%22,%22patent_firstnamed_assignee_location_id%22,%22patent_firstnamed_assignee_longitude%22,%22patent_firstnamed_assignee_state%22,%22patent_firstnamed_inventor_city%22,%22patent_firstnamed_inventor_country%22,%22patent_firstnamed_inventor_id%22,%22patent_firstnamed_inventor_latitude%22,%22patent_firstnamed_inventor_location_id%22,%22patent_firstnamed_inventor_longitude%22,%22patent_firstnamed_inventor_state%22,%22patent_kind%22,%22patent_num_cited_by_us_patents%22,%22patent_num_claims%22,%22patent_num_combined_citations%22,%22patent_num_foreign_citations%22,%22patent_num_us_application_citations%22,%22patent_num_us_patent_citations%22,%22patent_number%22,%22patent_title%22,%22patent_type%22,%22pct_102_date%22,%22pct_371_date%22,%22pct_date%22,%22pct_docnumber%22,%22pct_doctype%22,%22pct_kind%22,%22rawinventor_first_name%22,%22rawinventor_last_name%22,%22uspc_first_seen_date%22,%22uspc_last_seen_date%22,%22uspc_mainclass_id%22,%22uspc_mainclass_title%22,%22uspc_num_patents_for_assignee%22,%22uspc_sequence%22,%22uspc_subclass_id%22,%22uspc_subclass_title%22,%22uspc_total_num_assignees%22,%22uspc_total_num_inventors%22,%22wipo_field_id%22,%22wipo_field_title%22,%22wipo_sector_title%22,%22wipo_sequence%22,%22year_id%22,%22year_num_patents_for_assignee%22]&o=%7B%22include_subentity_total_counts%22:false,%22matched_subentities_only%22:true,%22page%22:1,%22per_page%22:25%7D&s=
#> Error in curl::curl_fetch_memory(url, handle = handle): Timeout was reached: Operation timed out after 120004 milliseconds with 0 bytes received
Created on 2019-09-19 by the reprex package (v0.2.0.9000).
few notes:
I've had search_pv() print out the URL that it's requesting in the examples above, in case that's easier for you to debug. I know there are a lot of fields in the URLs and hence the example isn't exactly minimal/targeted, but I'm not sure what more I can do to hone in on what the issue actually is.
Previously the examples shown above were able to run in a reasonable time frame. It's been like 6-12 months since I last worked on the patentsview package, but the API was returning results for all fields (across all endpoints) pretty quickly at that time, say within 10 seconds for each request.
search_pv() gets the set of retrievable fields for a given endpoint by scrapping the fields from your webpage (e.g., the patents endpoint field list here: http://www.patentsview.org/api/patent.html). I don't think it's a matter of the wrong fields being requested, as sometimes a request will work for a given set of fields and other times it will hang indefinitely (at least for the patents endpoint...Recently I haven't been able to get all fields for the assignees endpoint).
sarvo-madhavan commented 5 years ago
Hi Chris! Thanks for reporting this issue. From your description this looks like an issue with load on the API server. We'll further look into this and let you know what we find
crew102 commented 5 years ago
Now also having trouble getting all the fields for the patents endpoint. I just sent the request shown below at 9:29 with "https://github.com/ropensci/patentsview" as the user agent. Hope that helps.
crew102 commented 4 years ago
This is happening again, where it takes a long time to get results if many fields are asked for. I haven't used the API until recently, so I'm not sure if this is a problem with the migration to the new server.
killahjones commented 4 years ago
Hi Chris - We are taking a look at this and will get back to you as soon as we can. ~Christina
PatentsView / PatentsView-API

API hangs when trying to return many fields #42
