ScaleUnlimited / flink-crawler

Continuous scalable web crawler built on top of Flink and crawler-commons
Apache License 2.0
51 stars 18 forks source link

Updated Flink version to 1.7.2. #171

Closed vmagotra closed 5 years ago

vmagotra commented 5 years ago

The flink-shaded-hadoop2 dependency is now included with ‘provided’ scope in flink-hadoop-compatibility so we need to add it explicitly to our pom (as a ‘provided’ dependency). Updated use of the configuration options - ConfigConstants.TASK_MANAGER_NUM_TASK_SLOTS has been depracated, so switched to the new TaskManagerOptions.NUM_TASK_SLOTS, while TaskManagerOptions.MANAGED_MEMORY_SIZE has a default value of “0L” so removed the previous set up with -1L. #170