crs4 / pydoop

A Python MapReduce and HDFS API for Hadoop
Apache License 2.0
237 stars 59 forks source link

dict-compatible JobConf #320

Closed simleo closed 6 years ago

simleo commented 6 years ago

Fixes #289. Fixes #318.

Rewrites JobConf so that it has the same API as a dict (plus the type-specific getters), with no exceptions. Drops all backwards-compatible functionality (property conversion between Hadoop 1 and 2 and init from a flat sequence). JC objects can now be manipulated as dictionaries at all levels, since 'A' cases in serialize.cc have been rewritten to handle dicts directly.

Side effects: fixes the bugs listed above. I've also fixed error checking in get_rules (command.cc).