dpark调度器异常

[lizixing]

最近想尝试下用dpark写分布式程序，装好mesos后发现dpark运行时出错：

Traceback (most recent call last):
  File "calc-pi.py", line 13, in <module>
    dpark.parallelize(range(0, N), 5).foreach(random_once)
  File "/home/work/local/lib/python2.7/site-packages/Dpark-0.1-py2.7.egg/dpark/rdd.py", line 146, in foreach
    return self.ctx.runJob(self, mf)
  File "/home/work/local/lib/python2.7/site-packages/Dpark-0.1-py2.7.egg/dpark/context.py", line 128, in runJob
    self.start()
  File "/home/work/local/lib/python2.7/site-packages/Dpark-0.1-py2.7.egg/dpark/context.py", line 114, in start
    self.scheduler.start()
  File "/home/work/local/lib/python2.7/site-packages/Dpark-0.1-py2.7.egg/dpark/schedule.py", line 436, in start
    self.getExecutorInfo(), self.master)
  File "/home/work/local/lib/python2.7/site-packages/Dpark-0.1-py2.7.egg/dpark/schedule.py", line 402, in _
    return f(self, *a, **kw)
  File "/home/work/local/lib/python2.7/site-packages/Dpark-0.1-py2.7.egg/dpark/schedule.py", line 473, in getExecutorInfo
    info.uri = os.path.abspath(os.path.join(dir, 'executor'))
AttributeError: 'ExecutorInfo' object has no attribute 'uri'

我看了mesos_pb2.py的代码，发现ExecutorInfo的定义是这样的：

_EXECUTORINFO = descriptor.Descriptor(
  name='ExecutorInfo',
  full_name='mesos.ExecutorInfo',
  filename=None,
  file=DESCRIPTOR,
  containing_type=None,
  fields=[
    descriptor.FieldDescriptor(
      name='executor_id', full_name='mesos.ExecutorInfo.executor_id', index=0,
      number=1, type=11, cpp_type=10, label=2,
      has_default_value=False, default_value=None,
      message_type=None, enum_type=None, containing_type=None,
      is_extension=False, extension_scope=None,
      options=None),
    descriptor.FieldDescriptor(
      name='command', full_name='mesos.ExecutorInfo.command', index=1,
      number=7, type=11, cpp_type=10, label=2,
      has_default_value=False, default_value=None,
      message_type=None, enum_type=None, containing_type=None,
      is_extension=False, extension_scope=None,
      options=None),
    descriptor.FieldDescriptor(
      name='resources', full_name='mesos.ExecutorInfo.resources', index=2,
      number=5, type=11, cpp_type=10, label=3,
      has_default_value=False, default_value=[],
      message_type=None, enum_type=None, containing_type=None,
      is_extension=False, extension_scope=None,
      options=None),
    descriptor.FieldDescriptor(
      name='data', full_name='mesos.ExecutorInfo.data', index=3,
      number=4, type=12, cpp_type=9, label=1,
      has_default_value=False, default_value="",
      message_type=None, enum_type=None, containing_type=None,
      is_extension=False, extension_scope=None,
      options=None),
  ],
  extensions=[
  ],
  nested_types=[],
  enum_types=[
  ],
  options=None,
  is_extendable=False,
  extension_ranges=[],
  serialized_start=417,
  serialized_end=558,
)

里面没有uri。。。 是不是安装dpark需要的mesos必须是指定的某个较老的版本？ 我的mesos是用svn在https://svn.apache.org/repos/asf/incubator/mesos/trunk/下的

Path: .
URL: https://svn.apache.org/repos/asf/incubator/mesos/trunk
Repository Root: https://svn.apache.org/repos/asf
Repository UUID: 13f79535-47bb-0310-9956-ffa450edef68
Revision: 1330115
Node Kind: directory
Schedule: normal
Last Changed Author: benh
Last Changed Rev: 1330079
Last Changed Date: 2012-04-25 09:23:38 +0800 (Wed, 25 Apr 2012)

[davies]

是的，最近 mesos 的接口变动比较大，打算等 mesos 0.9 发布后再调整 DPark。

目前 DPark 可以工作在 Mesos trunk -r @1292597 或者 github 上 的 mirror 的 112ea0400dd3 版本。

可以 svn up -r 1292597 或者 git checkout 112ea0400dd3

[lizixing]

谢谢！终于运行起来了，不容易啊。。。

[davies]

enjoy it :)

On Fri, May 4, 2012 at 5:25 PM, lizixing < reply@reply.github.com

wrote:

谢谢！终于运行起来了，不容易啊。。。

---

Reply to this email directly or view it on GitHub: https://github.com/jackfengji/test_pro/issues/1#issuecomment-5506969

• Davies