Only create Spark UI proxy when requested

ukaea / piezo

1 stars 0 forks source link

Only create Spark UI proxy when requested #134

Open robert-clegg-tessella opened 5 years ago

robert-clegg-tessella commented 5 years ago

Spark UI proxies take up resources on the cluster that can help lead to a log jam. By only creating them when requested, the likelihood of this issue can be reduced.

Users should be able to request a Spark UI by setting a flag in the request to submitjob.

The wiki documentation needs to be updated to make this clear.

robert-clegg-tessella commented 5 years ago

Covered in PR 139