search

cpsievert / pitchRx

Tools for scraping MLB Gameday data and Visualizing PITCHf/x
http://cpsievert.github.io/pitchRx/
Other
124 stars 33 forks source link

Error when using scrape() with a db connection #33

Closed efbbrown closed 9 years ago

efbbrown commented 9 years ago

I'm currently receiving this error when using scrape().

require(data.table)
require(pitchRx)
require(RSQLite)
require(dplyr)

startDate <- "2015-09-13"
endDate <- "2015-09-15"

test <- scrape(startDate, endDate) # this works fine

db <- src_sqlite("pitchfx.sqlite3", create=TRUE)

scrape(startDate, endDate, connect = db$con)
If file names don't print right away, please be patient.
grabbing new game IDs
http://gd2.mlb.com/components/game/mlb/year_2015/month_09/day_12/miniscoreboard.xml 
http://gd2.mlb.com/components/game/mlb/year_2015/month_09/day_13/miniscoreboard.xml 
http://gd2.mlb.com/components/game/mlb/year_2015/month_09/day_14/miniscoreboard.xml 
http://gd2.mlb.com/components/game/mlb/year_2015/month_09/day_13/gid_2015_09_13_tormlb_nyamlb_1/inning/inning_all.xml 
http://gd2.mlb.com/components/game/mlb/year_2015/month_09/day_13/gid_2015_09_13_bosmlb_tbamlb_1/inning/inning_all.xml 

... [truncated]

http://gd2.mlb.com/components/game/mlb/year_2015/month_09/day_14/gid_2015_09_14_anamlb_seamlb_1/inning/inning_all.xml 
http://gd2.mlb.com/components/game/mlb/year_2015/month_09/day_14/gid_2015_09_14_cinmlb_sfnmlb_1/inning/inning_all.xml 
Error in names(value) <- sub("\\.", "_", names(value)) : 
  attempt to set an attribute on NULL

This same error occurs if I use a MySQL() database instead of the sqllite one.

db = dbConnect(MySQL(), user=username, password=password, dbname=dbname, host=host)
scrape(startDate, endDate, connect = db$con)

Here is some session info

devtools::session_info()
Session info -----------------------------------------------------------------------------------------------------------
 setting  value                       
 version  R version 3.2.0 (2015-04-16)
 system   x86_64, mingw32             
 ui       RStudio (0.99.441)          
 language (EN)                        
 collate  English_Australia.1252      
 tz       Australia/Sydney            

Packages ---------------------------------------------------------------------------------------------------------------
 package    * version  date       source        
 assertthat   0.1      2013-12-06 CRAN (R 3.2.0)
 bitops       1.0-6    2013-08-17 CRAN (R 3.2.0)
 chron        2.3-45   2014-02-11 CRAN (R 3.2.0)
 colorspace   1.2-6    2015-03-11 CRAN (R 3.2.0)
 curl         0.8      2015-06-06 CRAN (R 3.2.0)
 data.table * 1.9.4    2014-10-02 CRAN (R 3.2.0)
 DBI        * 0.3.1    2014-09-24 CRAN (R 3.2.0)
 devtools     1.8.0    2015-05-09 CRAN (R 3.2.0)
 digest       0.6.8    2014-12-31 CRAN (R 3.2.0)
 dplyr      * 0.4.1    2015-01-14 CRAN (R 3.2.0)
 ggplot2    * 1.0.1    2015-03-17 CRAN (R 3.2.0)
 git2r        0.10.1   2015-05-07 CRAN (R 3.2.0)
 gtable       0.1.2    2012-12-05 CRAN (R 3.2.0)
 hexbin       1.27.1   2015-08-19 CRAN (R 3.2.2)
 lattice      0.20-31  2015-03-30 CRAN (R 3.2.0)
 magrittr     1.5      2014-11-22 CRAN (R 3.2.0)
 MASS         7.3-40   2015-03-21 CRAN (R 3.2.0)
 Matrix       1.2-0    2015-04-04 CRAN (R 3.2.0)
 memoise      0.2.1    2014-04-22 CRAN (R 3.2.0)
 mgcv         1.8-6    2015-03-31 CRAN (R 3.2.0)
 munsell      0.4.2    2013-07-11 CRAN (R 3.2.0)
 nlme         3.1-120  2015-02-20 CRAN (R 3.2.0)
 pitchRx    * 1.7      2015-04-04 CRAN (R 3.2.2)
 plyr         1.8.3    2015-06-12 CRAN (R 3.2.1)
 proto        0.3-10   2012-12-22 CRAN (R 3.2.0)
 Rcpp         0.12.1   2015-09-10 CRAN (R 3.2.2)
 RCurl        1.95-4.7 2015-06-30 CRAN (R 3.2.2)
 reshape2     1.4.1    2014-12-06 CRAN (R 3.2.0)
 RMySQL     * 0.10.6   2015-09-05 CRAN (R 3.2.2)
 RSQLite    * 1.0.0    2014-10-25 CRAN (R 3.2.2)
 rstudioapi   0.3.1    2015-04-07 CRAN (R 3.2.0)
 rversions    1.0.1    2015-06-06 CRAN (R 3.2.0)
 scales       0.2.5    2015-06-12 CRAN (R 3.2.1)
 stringi      0.4-1    2014-12-14 CRAN (R 3.2.0)
 stringr      1.0.0    2015-04-30 CRAN (R 3.2.0)
 XML          3.98-1.2 2015-05-31 CRAN (R 3.2.0)
 xml2         0.1.1    2015-06-02 CRAN (R 3.2.0)
 XML2R        0.0.6    2014-03-10 CRAN (R 3.2.2)

Thanks again Carson, love the package.

cpsievert commented 9 years ago

Your example works for me when using the most recent GitHub version of pitchRx. Could you retry after doing:

devtools::install_github("cpsievert/pitchRx")
octaviantu commented 8 years ago

Hi! I have the following code:

devtools::install_github("cpsievert/pitchRx")

Load required packages

library('DBI') library('dplyr') library('ggplot2') library('pitchRx')

Create new database

db <- src_sqlite("pitchRx.sqlite3", create = TRUE)

Make sure your database was created and is empty.

db

Set up a scrape that will write to the newdb

scrape(start = "2016-04-01", end = as.character(Sys.Date() - 1), connect = db$con)

Download additional data and join

files <- c("inning/inning_hit.xml", "miniscoreboard.xml", "players.xml") scrape(start = "2016-04-01", end = as.character(Sys.Date() - 1), suffix = files, connect = db$con)]

And I'm getting the same issue:

Error in names(value) <- sub(".", "_", names(value)) : attempt to set an attribute on NULL

The output of

devtools::session_info()

is

Session info ------------------------------------------------------------------- Packages ----------------------------------------------------------------------- setting value
version R version 3.2.3 (2015-12-10) system x86_64, linux-gnu
ui X11
language (EN)
collate en_US.UTF-8
tz
date 2016-06-28

package * version date source
assertthat 0.1 2013-12-06 CRAN (R 3.2.3)
bitops 1.0-6 2013-08-17 CRAN (R 3.2.3)
colorspace 1.2-6 2015-03-11 CRAN (R 3.2.3)
curl 0.9.7 2016-04-10 CRAN (R 3.2.3)
DBI * 0.4-1 2016-05-08 CRAN (R 3.2.3)
devtools 1.11.1 2016-04-21 CRAN (R 3.2.3)
digest 0.6.9 2016-01-08 CRAN (R 3.2.3)
dplyr * 0.4.3 2015-09-01 CRAN (R 3.2.3)
ggplot2 * 2.1.0 2016-03-01 CRAN (R 3.2.3)
git2r 0.15.0 2016-05-11 CRAN (R 3.2.3)
gtable 0.2.0 2016-02-26 CRAN (R 3.2.3)
hexbin 1.27.1 2015-08-19 CRAN (R 3.2.3)
httr 1.1.0 2016-01-28 CRAN (R 3.2.3)
hwriter * 1.3.2 2014-09-10 CRAN (R 3.2.3)
hwriterPlus * 1.0-3 2015-01-05 CRAN (R 3.2.3)
lattice 0.20-33 2015-07-14 CRAN (R 3.2.3)
magrittr 1.5 2014-11-22 CRAN (R 3.2.3)
MASS 7.3-45 2016-04-21 CRAN (R 3.2.3)
Matrix 1.2-6 2016-05-02 CRAN (R 3.2.3)
memoise 1.0.0 2016-01-29 CRAN (R 3.2.3)
mgcv 1.8-7 2015-07-23 CRAN (R 3.2.1)
munsell 0.4.3 2016-02-13 CRAN (R 3.2.3)
nlme 3.1-124 2016-01-20 CRAN (R 3.2.3)
pitchRx * 1.8.3 2016-06-28 Github (cpsievert/pitchRx@7349b23) plyr 1.8.4 2016-06-08 CRAN (R 3.2.3)
R6 2.1.2 2016-01-26 CRAN (R 3.2.3)
Rcpp 0.12.5 2016-05-14 CRAN (R 3.2.3)
RCurl 1.95-4.8 2016-03-01 CRAN (R 3.2.3)
RODBC * 1.3-12 2015-06-29 CRAN (R 3.2.1)
Rserve * 1.7-3 2013-08-21 CRAN (R 3.0.2)
RSQLite 1.0.0 2014-10-25 CRAN (R 3.2.3)
scales 0.4.0 2016-02-26 CRAN (R 3.2.3)
SparkR * 1.6.1 2016-06-22 local
TeachingDemos 2.10 2016-02-12 CRAN (R 3.2.3)
withr 1.0.1 2016-02-04 CRAN (R 3.2.3)
XML 3.98-1.4 2016-03-01 CRAN (R 3.2.3)
XML2R 0.0.6 2014-03-10 CRAN (R 3.2.3)