Arturus / kaggle-web-traffic

1st place solution
MIT License
1.82k stars 667 forks source link

in make_features.py: TypeError: Cannot cast Index to dtype M8[D] #34

Closed wlemmon closed 5 years ago

wlemmon commented 5 years ago

I'm sure the issue is just that I don't have the correct versions of dependencies. Can you update the requirements.txt to state dependency versions for pandas, numpy, and scipy. I'm afraid I will have to play the guessing game.

wlemmon commented 5 years ago

Ok, for others, had to use older version of pandas: 0.22.0

wlemmon commented 5 years ago

if using aws deep learning ami + gpu instance, i got working starting with conda env tensorflow_p36, here is an environment.yml file you can use to create a replacement env that works. the only real changes are the pandas version and the addition of tqdm iirc:

name: tensorflow_p36
channels:
  - conda-forge
  - defaults
dependencies:
  - bazel=0.15.0=0
  - ffmpeg=4.0.2=ha0c5888_2
  - freeglut=3.0.0=hf484d3e_1005
  - gnutls=3.5.19=h2a4e5f8_1
  - libgfortran=3.0.0=1
  - libglu=9.0.0=hf484d3e_1000
  - libiconv=1.15=h516909a_1005
  - libprotobuf=3.7.1=h8b12597_0
  - mpi=1.0=openmpi
  - nb_conda=2.2.1=py36_2
  - nb_conda_kernels=2.2.2=py36_0
  - nettle=3.3=0
  - openh264=1.8.0=hdbcaa40_1000
  - openjdk=11.0.1=h516909a_1016
  - openmpi=4.0.1=hc99cbb1_1
  - protobuf=3.7.1=py36he1b5a44_0
  - requests=2.20.0=py36_1000
  - x264=1!152.20180806=h14c3975_0
  - xorg-fixesproto=5.0=h14c3975_1002
  - xorg-inputproto=2.3.2=h14c3975_1002
  - xorg-kbproto=1.0.7=h14c3975_1002
  - xorg-libx11=1.6.8=h516909a_0
  - xorg-libxau=1.0.9=h14c3975_0
  - xorg-libxext=1.3.4=h516909a_0
  - xorg-libxfixes=5.0.3=h516909a_1004
  - xorg-libxi=1.7.10=h516909a_0
  - xorg-xextproto=7.3.0=h14c3975_1002
  - xorg-xproto=7.0.31=h14c3975_1007
  - _libgcc_mutex=0.1=main
  - alabaster=0.7.10=py36h306e16b_0
  - anaconda-client=1.6.14=py36_0
  - anaconda-project=0.8.2=py36h44fb852_0
  - asn1crypto=0.24.0=py36_0
  - astroid=1.6.3=py36_0
  - astropy=3.0.2=py36h3010b51_1
  - attrs=18.1.0=py36_0
  - babel=2.5.3=py36_0
  - backcall=0.1.0=py36_0
  - backports=1.0=py36hfa02d7e_1
  - backports.shutil_get_terminal_size=1.0.0=py36hfea85ff_2
  - beautifulsoup4=4.6.0=py36h49b8c8c_1
  - bitarray=0.8.1=py36h14c3975_1
  - bkcharts=0.2=py36h735825a_0
  - blas=1.0=mkl
  - blaze=0.11.3=py36h4e06776_0
  - bleach=2.1.3=py36_0
  - blosc=1.14.3=hdbcaa40_0
  - bokeh=1.0.4=py36_0
  - boto=2.48.0=py36h6e4cd66_1
  - boto3=1.9.162=py_0
  - botocore=1.12.163=py_0
  - bottleneck=1.2.1=py36haac1ea0_0
  - bzip2=1.0.6=h14c3975_5
  - ca-certificates=2019.5.15=1
  - cairo=1.14.12=h8948797_3
  - certifi=2019.6.16=py36_1
  - cffi=1.11.5=py36h9745a5d_0
  - chardet=3.0.4=py36h0f667ec_1
  - click=6.7=py36h5253387_0
  - cloudpickle=0.5.3=py36_0
  - clyent=1.2.2=py36h7e57e65_1
  - colorama=0.3.9=py36h489cec4_0
  - contextlib2=0.5.5=py36h6c84a62_0
  - cryptography=2.2.2=py36h14c3975_0
  - curl=7.60.0=h84994c4_0
  - cycler=0.10.0=py36h93f1223_0
  - cython=0.28.2=py36h14c3975_0
  - cytoolz=0.9.0.1=py36h14c3975_0
  - dask=1.2.2=py_0
  - dask-core=1.2.2=py_0
  - datashape=0.5.4=py36h3ad6b5c_0
  - dbus=1.13.2=h714fa37_1
  - decorator=4.3.0=py36_0
  - defusedxml=0.6.0=py_0
  - distributed=1.28.1=py36_0
  - docutils=0.14=py36hb0f60f5_0
  - entrypoints=0.2.3=py36h1aec115_2
  - et_xmlfile=1.0.1=py36hd6bccc3_0
  - expat=2.2.5=he0dffb1_0
  - fastcache=1.0.2=py36h14c3975_2
  - filelock=3.0.4=py36_0
  - flask=1.0.2=py36_1
  - flask-cors=3.0.4=py36_0
  - fontconfig=2.13.0=h9420a91_0
  - freetype=2.9.1=h8a8886c_1
  - fribidi=1.0.5=h7b6447c_0
  - get_terminal_size=1.0.0=haa9412d_0
  - gevent=1.3.0=py36h14c3975_0
  - glib=2.56.1=h000015b_0
  - glob2=0.6=py36he249c77_0
  - gmp=6.1.2=h6c8ec71_1
  - gmpy2=2.0.8=py36hc8893dd_2
  - graphite2=1.3.11=h16798f4_2
  - graphviz=2.40.1=h21bd128_2
  - greenlet=0.4.13=py36h14c3975_0
  - gst-plugins-base=1.14.0=hbbd80ab_1
  - gstreamer=1.14.0=hb453b48_1
  - h5py=2.8.0=py36h989c5e5_3
  - harfbuzz=1.8.8=hffaf4a1_0
  - hdf5=1.10.2=hba1933b_1
  - heapdict=1.0.0=py36_2
  - html5lib=1.0.1=py36h2f9c1c0_0
  - icu=58.2=h9c2bf20_1
  - idna=2.6=py36h82fb2a8_1
  - imageio=2.3.0=py36_0
  - imagesize=1.0.0=py36_0
  - intel-openmp=2018.0.0=8
  - ipykernel=4.8.2=py36_0
  - ipython=6.4.0=py36_0
  - ipython_genutils=0.2.0=py36hb52b0d5_0
  - ipywidgets=7.2.1=py36_0
  - isort=4.3.4=py36_0
  - itsdangerous=0.24=py36h93cc618_1
  - jasper=2.0.14=h07fcdf6_1
  - jbig=2.1=hdba287a_0
  - jdcal=1.4=py36_0
  - jedi=0.12.0=py36_1
  - jinja2=2.10=py36ha16c418_0
  - jmespath=0.9.4=py_0
  - joblib=0.13.2=py36_0
  - jpeg=9b=h024ee3a_2
  - jsonschema=2.6.0=py36h006f8b5_0
  - jupyter=1.0.0=py36_4
  - jupyter_client=5.2.3=py36_0
  - jupyter_console=5.2.0=py36he59e554_1
  - jupyter_core=4.4.0=py36h7c827e3_0
  - jupyterlab=0.32.1=py36_0
  - jupyterlab_launcher=0.10.5=py36_0
  - kiwisolver=1.0.1=py36h764f252_0
  - krb5=1.14.2=hcdc1b81_6
  - lazy-object-proxy=1.3.1=py36h10fcdad_0
  - libcurl=7.60.0=h1ad7b7a_0
  - libedit=3.1.20170329=h6b74fdf_2
  - libffi=3.2.1=hd88cf55_4
  - libgcc-ng=9.1.0=hdf63c60_0
  - libgfortran-ng=7.2.0=hdf63c60_3
  - libopencv=3.4.2=hb342d67_1
  - libpng=1.6.34=hb9fc6fc_0
  - libsodium=1.0.16=h1bed415_0
  - libssh2=1.8.0=h9cfc8f7_4
  - libstdcxx-ng=9.1.0=hdf63c60_0
  - libtiff=4.0.9=he85c1e1_1
  - libtool=2.4.6=h544aabb_3
  - libuuid=1.0.3=h1bed415_2
  - libxcb=1.13=h1bed415_1
  - libxml2=2.9.8=h26e45fe_1
  - libxslt=1.1.32=h1312cb7_0
  - llvmlite=0.23.1=py36hdbcaa40_0
  - locket=0.2.0=py36h787c0ad_1
  - lxml=4.2.1=py36h23eabaa_0
  - lzo=2.10=h49e0be7_2
  - markupsafe=1.0=py36hd9260cd_1
  - matplotlib=2.2.3=py36hb69df0a_0
  - mccabe=0.6.1=py36h5ad9710_1
  - mistune=0.8.3=py36h14c3975_1
  - mkl=2019.4=243
  - mkl-service=2.0.2=py36h7b6447c_0
  - mkl_fft=1.0.12=py36ha843d7b_0
  - mkl_random=1.0.2=py36hd81dba3_0
  - more-itertools=4.1.0=py36_0
  - mpc=1.0.3=hec55b23_5
  - mpfr=3.1.5=h11a74b3_2
  - mpmath=1.0.0=py36hfeacd6b_2
  - msgpack-python=0.5.6=py36h6bb024c_0
  - multipledispatch=0.5.0=py36_0
  - nbconvert=5.4.1=py36_3
  - nbformat=4.4.0=py36h31c9010_0
  - ncurses=6.1=hf484d3e_0
  - networkx=2.1=py36_0
  - nltk=3.3.0=py36_0
  - nose=1.3.7=py36hcdf7029_2
  - notebook=5.5.0=py36_0
  - numba=0.38.0=py36h637b7d7_0
  - numexpr=2.6.9=py36h9e4a6bb_0
  - numpy=1.16.4=py36h7e9f1db_0
  - numpy-base=1.16.4=py36hde5b4d6_0
  - numpydoc=0.8.0=py36_0
  - odo=0.5.1=py36h90ed295_0
  - olefile=0.45.1=py36_0
  - opencv=3.4.2=py36h6fd60c2_1
  - openpyxl=2.5.3=py36_0
  - openssl=1.0.2s=h7b6447c_0
  - packaging=17.1=py36_0
  - pandas=0.22.0=py36hf484d3e_0
  - pandoc=1.19.2.1=hea2e7c5_1
  - pandocfilters=1.4.2=py36ha6701b7_1
  - pango=1.42.3=h8589676_0
  - parso=0.2.0=py36_0
  - partd=0.3.8=py36h36fd896_0
  - patchelf=0.9=hf79760b_2
  - path.py=11.0.1=py36_0
  - pathlib2=2.3.2=py36_0
  - patsy=0.5.0=py36_0
  - pcre=8.42=h439df22_0
  - pep8=1.7.1=py36_0
  - pexpect=4.5.0=py36_0
  - pickleshare=0.7.4=py36h63277f8_0
  - pillow=5.2.0=py36heded4f4_0
  - pip=10.0.1=py36_0
  - pixman=0.34.0=hceecf20_3
  - pkginfo=1.4.2=py36_1
  - pluggy=0.6.0=py36hb689045_0
  - ply=3.11=py36_0
  - prompt_toolkit=1.0.15=py36h17d85b1_0
  - psutil=5.4.5=py36h14c3975_0
  - ptyprocess=0.5.2=py36h69acd42_0
  - py=1.5.3=py36_0
  - py-opencv=3.4.2=py36hb342d67_1
  - pycodestyle=2.4.0=py36_0
  - pycosat=0.6.3=py36h0a5515d_0
  - pycparser=2.18=py36hf9f622e_1
  - pycrypto=2.6.1=py36h14c3975_8
  - pycurl=7.43.0.1=py36hb7f436b_0
  - pyflakes=1.6.0=py36h7bd6a15_0
  - pygments=2.2.0=py36h0d3125c_0
  - pykerberos=1.2.1=py36h14c3975_0
  - pylint=1.8.4=py36_0
  - pyodbc=4.0.23=py36hf484d3e_0
  - pyopenssl=18.0.0=py36_0
  - pyparsing=2.2.0=py36hee85983_1
  - pyqt=5.9.2=py36h751905a_0
  - pysocks=1.6.8=py36_0
  - pytables=3.4.3=py36h02b9ad4_2
  - pytest=3.5.1=py36_0
  - pytest-arraydiff=0.2=py36_0
  - pytest-astropy=0.3.0=py36_0
  - pytest-doctestplus=0.1.3=py36_0
  - pytest-openfiles=0.3.0=py36_0
  - pytest-remotedata=0.2.1=py36_0
  - python=3.6.5=hc3d631a_2
  - python-dateutil=2.7.3=py36_0
  - pytz=2018.4=py36_0
  - pywavelets=0.5.2=py36he602eb0_0
  - pyyaml=3.12=py36hafb9ca4_1
  - pyzmq=17.0.0=py36h14c3975_0
  - qt=5.9.6=h52aff34_0
  - qtawesome=0.4.4=py36h609ed8c_0
  - qtconsole=4.3.1=py36h8f73b5b_0
  - qtpy=1.4.1=py36_0
  - readline=7.0=ha6073c6_4
  - rope=0.10.7=py36h147e2ec_0
  - ruamel_yaml=0.15.35=py36h14c3975_1
  - s3fs=0.1.5=py36_0
  - s3transfer=0.2.0=py36_0
  - scikit-image=0.13.1=py36h14c3975_1
  - scikit-learn=0.21.2=py36hd81dba3_0
  - scipy=1.3.0=py36h7c811a0_0
  - seaborn=0.8.1=py36hfad7ec4_0
  - send2trash=1.5.0=py36_0
  - setuptools=39.1.0=py36_0
  - simplegeneric=0.8.1=py36_2
  - singledispatch=3.4.0.3=py36h7a266c3_0
  - sip=4.19.8=py36hf484d3e_0
  - six=1.11.0=py36h372c433_1
  - snappy=1.1.7=hbae5bb6_3
  - snowballstemmer=1.2.1=py36h6febd40_0
  - sortedcollections=0.6.1=py36_0
  - sortedcontainers=1.5.10=py36_0
  - sphinx=1.7.4=py36_0
  - sphinxcontrib=1.0=py36h6d0f590_1
  - sphinxcontrib-websupport=1.0.1=py36hb5cb234_1
  - spyder=3.2.8=py36_0
  - sqlalchemy=1.2.7=py36h6b74fdf_0
  - sqlite=3.23.1=he433501_0
  - statsmodels=0.9.0=py36h3010b51_0
  - sympy=1.1.1=py36hc6d1c1c_0
  - tblib=1.3.2=py36h34cf8b6_0
  - terminado=0.8.1=py36_1
  - testpath=0.3.1=py36h8cadb63_0
  - tk=8.6.7=hc745277_3
  - toolz=0.9.0=py36_0
  - tornado=5.0.2=py36_0
  - tqdm=4.32.1=py_0
  - traitlets=4.3.2=py36h674d592_0
  - typing=3.6.4=py36_0
  - unicodecsv=0.14.1=py36ha668878_0
  - unixodbc=2.3.6=h1bed415_0
  - urllib3=1.23=py36_0
  - wcwidth=0.1.7=py36hdf4376a_0
  - webencodings=0.5.1=py36h800622e_1
  - werkzeug=0.14.1=py36_0
  - wheel=0.31.1=py36_0
  - widgetsnbextension=3.2.1=py36_0
  - wrapt=1.10.11=py36h28b7045_0
  - xlrd=1.1.0=py36h1db9f0c_1
  - xlsxwriter=1.0.4=py36_0
  - xlwt=1.3.0=py36h7b00a1f_0
  - xz=5.2.4=h14c3975_4
  - yaml=0.1.7=had09818_2
  - zeromq=4.2.5=h439df22_0
  - zict=0.1.3=py36h3a3bf81_0
  - zlib=1.2.11=ha838bed_2
  - pip:
    - absl-py==0.7.1
    - astor==0.8.0
    - automat==0.3.0
    - autovizwidget==0.12.9
    - bcrypt==3.1.7
    - cached-property==1.5.1
    - characteristic==14.3.0
    - docker==3.7.3
    - docker-compose==1.24.1
    - docker-pycreds==0.4.0
    - dockerpty==0.4.1
    - docopt==0.6.2
    - environment-kernels==1.1.1
    - gast==0.2.2
    - google-pasta==0.1.7
    - grpcio==1.10.1
    - hdijupyterutils==0.12.9
    - horovod==0.16.4
    - ipyparallel==6.2.2
    - keras==2.2.4
    - keras-applications==1.0.8
    - keras-preprocessing==1.1.0
    - markdown==3.1.1
    - mock==3.0.5
    - msgpack==0.6.0
    - paramiko==2.6.0
    - plotly==4.0.0
    - protobuf3-to-dict==0.1.5
    - psycopg2==2.7.5
    - py4j==0.10.7
    - pygal==2.4.0
    - pynacl==1.3.0
    - pyspark==2.3.2
    - requests-kerberos==0.12.0
    - retrying==1.3.3
    - sagemaker==1.34.1
    - sagemaker-pyspark==1.2.4
    - sparkmagic==0.12.5
    - tables==3.4.3
    - tensorboard==1.14.0
    - tensorflow==1.14.0
    - tensorflow-estimator==1.14.0
    - tensorflow-serving-api==1.14.0
    - termcolor==1.1.0
    - texttable==0.9.1
    - websocket-client==0.56.0
prefix: /home/ubuntu/anaconda3/envs/tensorflow_p36