IKANOW / Aleph2

The IKANOW v2 meta-database and analytics platform
Apache License 2.0
2 stars 1 forks source link

Hadoop (etc) tasks running on DB - file output doesn't work since HDP2.3? #73

Open Alex-Ikanow opened 8 years ago

Alex-Ikanow commented 8 years ago

This isn't confirmed, but it looked like "externalEmit" failed on a DB node and worked on all other nodes

For DB nodes (unless they have the aleph2 service running), they use environment variables to get their FileContext reference, so it seems likely that this no longer works since HDP2.3.

Workaround is either to install /opt/aleph2-home/yarn-config/* or to turn off NodeManagers on the DB nodes

Need to turn off the API NodeManagers on a dev cluster, confirm, and then fix.