search

NeCTAR-RC / tutorials

Nectar Tutorials website
https://tutorials.rc.nectar.org.au
Apache License 2.0
5 stars 8 forks source link

hadoop/spark #108

Open skiptoniam opened 2 years ago

skiptoniam commented 2 years ago

This tutorial seems to be well out of date. I managed to get it running by folking the elasticluster repo and making a lot of changes. I seem to have a few issues actually running the hadoop cluster, which I'm still working through, I expect some changes to the ansible playbooks might have cause issue with the hadoop/spark cluster.

ramson33 commented 2 years ago

Hi there,

Thanks for letting us know, and sorry there were issues with the tutorial!

I will have a look through and make fixes where necessary.

Kind regards, Sonia

skiptoniam commented 2 years ago

All good. I also had to run elasticluster from a python3.7 virtual environment. It didn't work from 3.8.

andybotting commented 1 year ago

hey @skiptoniam We've had a couple of tickets come through recently about this. I've been trying to make this work with at least Python 3.10 and I've fixed up a few things.

We're trying to work out if we can continue to support this tutorial, as it does feel like upstream aren't really maintaining it.

Maybe with the work you've done here and my work fixing some of the newer Python issues, we can keep it going. Are you going to be using Elasticluster longish term?