Simple Infer pickle.load EOF error

[jaskarannagi19]

(Pose) jaskaran@jaskaran-pc:~/densepose$ python tools/infer_simple.py --cfg configs/DensePose_ResNet101_FPN_s1x-e2e.yaml --output-dir DensePoseData/infer_out/ --image-ext jpg --wts DensePose_ResNet101_FPN_s1x-e2e.pkl DensePoseData/demo_data/demo_im2.jpg /home/jaskaran/.conda/envs/Pose/lib/python2.7/site-packages/h5py/init.py:36: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from ._conv import register_converters as _register_converters /home/jaskaran/.conda/envs/Pose/lib/python2.7/site-packages/h5py/init.py:45: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from . import h5a, h5d, h5ds, h5f, h5fd, h5g, h5r, h5s, h5t, h5p, h5z /home/jaskaran/.conda/envs/Pose/lib/python2.7/site-packages/h5py/_hl/group.py:22: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from .. import h5g, h5i, h5o, h5r, h5t, h5l, h5p /home/jaskaran/.conda/envs/Pose/lib/python2.7/site-packages/scipy/linalg/basic.py:17: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from ._solve_toeplitz import levinson /home/jaskaran/.conda/envs/Pose/lib/python2.7/site-packages/scipy/linalg/init.py:207: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from ._decomp_update import /home/jaskaran/.conda/envs/Pose/lib/python2.7/site-packages/scipy/special/init.py:640: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from ._ufuncs import /home/jaskaran/.conda/envs/Pose/lib/python2.7/site-packages/scipy/special/_ellip_harm.py:7: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from ._ellip_harm_2 import _ellipsoid, _ellipsoid_norm /home/jaskaran/.conda/envs/Pose/lib/python2.7/site-packages/scipy/interpolate/_bsplines.py:10: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from . import _bspl /home/jaskaran/.conda/envs/Pose/lib/python2.7/site-packages/scipy/sparse/lil.py:19: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from . import _csparsetools /home/jaskaran/.conda/envs/Pose/lib/python2.7/site-packages/scipy/sparse/csgraph/init.py:165: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from ._shortest_path import shortest_path, floyd_warshall, dijkstra,\ /home/jaskaran/.conda/envs/Pose/lib/python2.7/site-packages/scipy/sparse/csgraph/_validation.py:5: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from ._tools import csgraph_to_dense, csgraph_from_dense,\ /home/jaskaran/.conda/envs/Pose/lib/python2.7/site-packages/scipy/sparse/csgraph/init.py:167: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from ._traversal import breadth_first_order, depth_first_order, \ /home/jaskaran/.conda/envs/Pose/lib/python2.7/site-packages/scipy/sparse/csgraph/init.py:169: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from ._min_spanning_tree import minimum_spanning_tree /home/jaskaran/.conda/envs/Pose/lib/python2.7/site-packages/scipy/sparse/csgraph/init.py:170: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from ._reordering import reverse_cuthill_mckee, maximum_bipartite_matching, \ /home/jaskaran/.conda/envs/Pose/lib/python2.7/site-packages/scipy/spatial/init.py:95: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from .ckdtree import /home/jaskaran/.conda/envs/Pose/lib/python2.7/site-packages/scipy/spatial/init.py:96: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from .qhull import /home/jaskaran/.conda/envs/Pose/lib/python2.7/site-packages/scipy/spatial/_spherical_voronoi.py:18: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from . import _voronoi /home/jaskaran/.conda/envs/Pose/lib/python2.7/site-packages/scipy/spatial/distance.py:122: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from . import _hausdorff /home/jaskaran/.conda/envs/Pose/lib/python2.7/site-packages/scipy/io/matlab/mio4.py:18: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from .mio_utils import squeeze_element, chars_to_strings /home/jaskaran/.conda/envs/Pose/lib/python2.7/site-packages/scipy/io/matlab/mio5.py:98: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from .mio5_utils import VarReader5 Found Detectron ops lib: /home/jaskaran/.conda/envs/Pose/lib/libcaffe2_detectron_ops_gpu.so E0803 01:04:59.011703 10154 init_intrinsics_check.cc:43] CPU feature avx is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU. E0803 01:04:59.011721 10154 init_intrinsics_check.cc:43] CPU feature avx2 is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU. E0803 01:04:59.011726 10154 init_intrinsics_check.cc:43] CPU feature fma is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU. WARNING cnn.py: 25: [====DEPRECATE WARNING====]: you are creating an object from CNNModelHelper class which will be deprecated soon. Please use ModelHelper object with brew module. For more information, please refer to caffe2.ai and python/brew.py, python/brew_test.py for more information. INFO net.py: 51: Loading weights from: DensePose_ResNet101_FPN_s1x-e2e.pkl Traceback (most recent call last): File "tools/infer_simple.py", line 140, in main(args) File "tools/infer_simple.py", line 91, in main model = infer_engine.initialize_model_from_cfg(args.weights) File "/home/jaskaran/densepose/detectron/core/test_engine.py", line 336, in initialize_model_from_cfg model, weights_file, gpu_id=gpu_id, File "/home/jaskaran/densepose/detectron/utils/net.py", line 54, in initialize_gpu_from_weights_file src_blobs = pickle.load(f) EOFError

Please see if you can help out in this

[siamaksalman]

What is your setup ? OS ? GPU ? CUDA version ? python version ?

[jaskarannagi19]

@siamaksalman OS: Ubuntu 16.04 GPU: Nvidia GTX 1050 CUDA: 8.0 Python: 2.7 from anaconda

[siamaksalman]

@jaskarannagi19 As I mentions in the guide, the official docker file of DensePose uses CUDA 9.0 with cudnn7, I think installing cuda 9 will solve your problem.

Use this command to make sure your caffe2 can access your CUDA:

python2 -c 'from caffe2.python import     workspace; print    (workspace.NumCudaDevices())'

[jaskarannagi19]

@siamaksalman Will I be need to reinstall caffe 2 as well ? If I update my cuda version?

[jaskarannagi19]

Yes caffe 2 is able to access my cuda and shows 1 for GPU command

[siamaksalman]

I don't thing reinstalling caffe2 is needed. did you upgrade your CUDA version?

[jaskarannagi19]

Not yet I will reinstall after 10 hours and then I will reply accordingly.

On Sat 4 Aug, 2018, 2:57 PM Siamak, notifications@github.com wrote:

I don't thing reinstalling caffe2 is needed. did you upgrade your CUDA version?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/siamaksalman/denseposeInstalltion/issues/2#issuecomment-410436553, or mute the thread https://github.com/notifications/unsubscribe-auth/AlbDycyMCxspc3gdIJ-zBdScKRnTYddOks5uNWjsgaJpZM4Vs4hf .

[jaskarannagi19]

@siamaksalman updating cuda didnt help either now cua is 9.0 cudnn is 7 and nccl is 2 i did reinstalled caffe2 like this

conda install -c caffe2 caffe2-cuda9.0-cudnn7

nvcc --version nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2017 NVIDIA Corporation Built on Fri_Sep__1_21:08:03_CDT_2017 Cuda compilation tools, release 9.0, V9.0.176

python2 -c 'from caffe2.python import core' 2>/dev/null && echo "Success" || echo "Failure" this says success

python2 -c 'from caffe2.python import workspace; print(workspace.NumCudaDevices()) says 1

[siamaksalman]

oh my bad, yes also you needed to install caffe2-cuda9.0-cudnn7. do you still get the same error while running the script ?

[jaskarannagi19]

Yes

On Sun 5 Aug, 2018, 4:20 PM Siamak, notifications@github.com wrote:

oh my bad, yes also you needed to install caffe2-cuda9.0-cudnn7. do you still get the same error while running the script ?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/siamaksalman/denseposeInstalltion/issues/2#issuecomment-410511898, or mute the thread https://github.com/notifications/unsubscribe-auth/AlbDyepq8mmJblowRHMvIBzgmk1FTpWIks5uNs4CgaJpZM4Vs4hf .

[siamaksalman]

I think your setup is okey, maybe downloading DensePose latest commit will do the job. Latest commit e118a4c 6 days ago

[jaskarannagi19]

@siamaksalman some one suggested its a memory issue I'll download llighter weight files to confirm but I am 80% sure that when its memory issue pickle says EOFerror ran out of input memory. Running very high fever need to take break before I redo my Ubuntu and follow all steps you listed here