search

apache / mxnet

Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Scala, Go, Javascript and more
https://mxnet.apache.org
Apache License 2.0
20.77k stars 6.79k forks source link

mxnet.recordio.pack example fails in python3 #16427

Open RoeePeleg opened 4 years ago

RoeePeleg commented 4 years ago

Description

when running the example from mxnet.recordio.pack using python3 it fails: TypeError: can't concat str to bytes

Environment info (Required)

----------Python Info---------- Version : 3.6.7 Compiler : GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2) Build : ('default', 'Jun 19 2019 11:26:37') Arch : ('64bit', '') ------------Pip Info----------- Version : 19.2.3 Directory : /Users/rpeleg/.pyenv/versions/3.6.7/lib/python3.6/site-packages/pip ----------MXNet Info----------- Version : 1.4.1 Directory : /Users/rpeleg/.pyenv/versions/3.6.7/lib/python3.6/site-packages/mxnet Commit Hash : 1a7199691f5cbc6012bb53eecbf884bed5ae6590 Library : ['/Users/rpeleg/.pyenv/versions/3.6.7/lib/python3.6/site-packages/mxnet/libmxnet.so'] Build features: No runtime build feature info available ----------System Info---------- Platform : Darwin-18.7.0-x86_64-i386-64bit system : Darwin node : 38f9d32eb535.ant.amazon.com release : 18.7.0 version : Darwin Kernel Version 18.7.0: Tue Aug 20 16:57:14 PDT 2019; root:xnu-4903.271.2~2/RELEASE_X86_64 ----------Hardware Info---------- machine : x86_64 processor : i386 b'machdep.cpu.brand_string: Intel(R) Core(TM) i7-7660U CPU @ 2.50GHz' b'machdep.cpu.features: FPU VME DE PSE TSC MSR PAE MCE CX8 APIC SEP MTRR PGE MCA CMOV PAT PSE36 CLFSH DS ACPI MMX FXSR SSE SSE2 SS HTT TM PBE SSE3 PCLMULQDQ DTES64 MON DSCPL VMX SMX EST TM2 SSSE3 FMA CX16 TPR PDCM SSE4.1 SSE4.2 x2APIC MOVBE POPCNT AES PCID XSAVE OSXSAVE SEGLIM64 TSCTMR AVX1.0 RDRAND F16C' b'machdep.cpu.leaf7_features: RDWRFSGS TSC_THREAD_OFFSET SGX BMI1 HLE AVX2 SMEP BMI2 ERMS INVPCID RTM FPU_CSDS MPX RDSEED ADX SMAP CLFSOPT IPT MDCLEAR TSXFA IBRS STIBP L1DF SSBD' b'machdep.cpu.extfeatures: SYSCALL XD 1GBPAGE EM64T LAHF LZCNT PREFETCHW RDTSCP TSCI' ----------Network Test---------- Setting timeout: 10 Timing for MXNet: https://github.com/apache/incubator-mxnet, DNS: 0.0007 sec, LOAD: 1.7365 sec. Timing for Gluon Tutorial(en): http://gluon.mxnet.io, DNS: 0.0008 sec, LOAD: 1.1648 sec. Timing for Gluon Tutorial(cn): https://zh.gluon.ai, DNS: 0.0005 sec, LOAD: 0.9769 sec. Timing for FashionMNIST: https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/dataset/fashion-mnist/train-labels-idx1-ubyte.gz, DNS: 0.0004 sec, LOAD: 0.5961 sec. Timing for PYPI: https://pypi.python.org/pypi/pip, DNS: 0.0004 sec, LOAD: 1.7993 sec. Timing for Conda: https://repo.continuum.io/pkgs/free/, DNS: 0.0004 sec, LOAD: 0.4702 sec. ----------Environment----------

Package used (Python/R/Scala/Julia): I'm using Python, Mxnet

Error Message:

Traceback (most recent call last): File "rec_pack_example.py", line 6, in packed_s = mx.recordio.pack(header, s) File "/Users/rpeleg/.pyenv/versions/3.6.7/lib/python3.6/site-packages/mxnet/recordio.py", line 390, in pack s = struct.pack(_IR_FORMAT, *header) + s TypeError: can't concat str to bytes

Minimum reproducible example

i'm running this example (as coded here: https://github.com/apache/incubator-mxnet/blob/master/python/mxnet/recordio.py#L378-L386):

import mxnet as mx label = 4 # label can also be a 1-D array, for example: label = [1,2,3] id = 2574 header = mx.recordio.IRHeader(0, label, id, 0) s = '' packed_s = mx.recordio.pack(header, s)

Steps to reproduce

(Paste the commands you ran that produced the error.)

  1. download the following zip rec_pack_example.py.zip

  2. run: python3 rec_pack_example.py

mxnet-label-bot commented 4 years ago

Hey, this is the MXNet Label Bot. Thank you for submitting the issue! I will try and suggest some labels so that the appropriate MXNet community members can help resolve it. Here are my recommended label(s): Example

ddavydenko commented 4 years ago

@mxnet-label-bot , add [Bug]

tuanzhangCS commented 4 years ago

Use s = b'...' @RoeePeleg You can learn the difference of string between python2 and python3 (encoding).

RoeePeleg commented 4 years ago

Use s = b'...' @RoeePeleg You can learn the difference of string between python2 and python3 (encoding).

@tuanzhangCS i know i can use bytes instead of str in order to make it work. i just wanted to note that the example and the documentation of the method are not correct. https://github.com/apache/incubator-mxnet/blob/master/python/mxnet/recordio.py#L370-L371 https://github.com/apache/incubator-mxnet/blob/master/python/mxnet/recordio.py#L378-L386