AutoConfigClient.mkdir does not create directory

saurabhsawant commented 8 years ago

AutoConfigClient.mkdir does not create directories.

wouterdebie commented 8 years ago

Can you be a bit more specific? Show how you invoke the client?

saurabhsawant commented 8 years ago

Thank you for followup :)

Here is what I am doing

>>>from snakebite.client import AutoConfigClient
>>> client = AutoConfigClient()
>>> for x in client.ls(['/delphi/next-staging/cmv/2016/07/']):
>>> print x

{'group': u'supergroup', 'permission': 493, 'file_type': 'd', 'access_time': 0L, 'block_replication': 0, 'modification_time': 1468353318963L, 'length': 0L, 'blocksize': 0L, 'owner': u'sparktor', 'path': '/delphi/next-staging/cmv/2016/07/01'}
{'group': u'supergroup', 'permission': 493, 'file_type': 'd', 'access_time': 0L, 'block_replication': 0, 'modification_time': 1468623329768L, 'length': 0L, 'blocksize': 0L, 'owner': u'sparktor', 'path': '/delphi/next-staging/cmv/2016/07/05'}
{'group': u'supergroup', 'permission': 493, 'file_type': 'd', 'access_time': 0L, 'block_replication': 0, 'modification_time': 1468623170408L, 'length': 0L, 'blocksize': 0L, 'owner': u'sparktor', 'path': '/delphi/next-staging/cmv/2016/07/06'}
{'group': u'supergroup', 'permission': 493, 'file_type': 'd', 'access_time': 0L, 'block_replication': 0, 'modification_time': 1468622967724L, 'length': 0L, 'blocksize': 0L, 'owner': u'sparktor', 'path': '/delphi/next-staging/cmv/2016/07/07'}

....

But when I try to create a directory using mkdir, it does not have any effect

>>> client.mkdir(['/delphi/next-staging/cmv/2016/07/29'])
<generator object wrapped at 0x103f5d1e0>

No effect for this

but if i create using "hadoop fs -mkdir, it does"

saurabhsawant commented 8 years ago

I tried with create parent = True flag , still nothing I tried mkdirp but AutoConfigClient does not have it I guess.

saurabhsawant commented 8 years ago

More info:

pip show snakebite
---
Metadata-Version: 2.0
Name: snakebite
Version: 2.10.1
Summary: Pure Python HDFS client
Home-page: http://github.com/spotify/snakebite
Author: Wouter de Bie
Author-email: wouter@spotify.com
License: Apache License 2.0
Location: /Library/Python/2.7/site-packages
Requires: argparse, protobuf
Classifiers:
  Topic :: Utilities
  Programming Language :: Python
  Operating System :: POSIX :: Linux
  Operating System :: MacOS
  Topic :: Software Development :: Libraries :: Application Frameworks
  Environment :: Other Environment

python --version
Python 2.7.11

wouterdebie commented 8 years ago

mkdir return a generator that you have to consume for it to execute. It's in the docs. This is done because you can pass in multiple paths and each path can fail or succeed independently.

On Aug 5, 2016 7:36 PM, "saurabhsawant" notifications@github.com wrote:

More info:

pip show snakebite

Metadata-Version: 2.0 Name: snakebite Version: 2.10.1 Summary: Pure Python HDFS client Home-page: http://github.com/spotify/snakebite Author: Wouter de Bie Author-email: wouter@spotify.com License: Apache License 2.0 Location: /Library/Python/2.7/site-packages Requires: argparse, protobuf Classifiers: Topic :: Utilities Programming Language :: Python Operating System :: POSIX :: Linux Operating System :: MacOS Topic :: Software Development :: Libraries :: Application Frameworks Environment :: Other Environment

python --version Python 2.7.11

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/spotify/snakebite/issues/213#issuecomment-237989578, or mute the thread https://github.com/notifications/unsubscribe-auth/AAKgBmqga2Kk61gtMR_UMCkQlvPx_h0bks5qc8jxgaJpZM4JdPmm .

saurabhsawant commented 8 years ago

Okay I see ! I will try that. Thank you for the prompt reply.

spotify / snakebite

AutoConfigClient.mkdir does not create directory #213

pip show snakebite