Spark UI container is not getting started

[purnima1612]

I am following all the steps to setup local spark UI but getting error . Please help

2023-02-22 14:19:51 Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties 2023-02-22 14:19:51 23/02/22 19:19:51 INFO HistoryServer: Started daemon with process name: 1@e3f348f82a0d 2023-02-22 14:19:51 23/02/22 19:19:51 INFO SignalUtils: Registering signal handler for TERM 2023-02-22 14:19:51 23/02/22 19:19:51 INFO SignalUtils: Registering signal handler for HUP 2023-02-22 14:19:51 23/02/22 19:19:51 INFO SignalUtils: Registering signal handler for INT 2023-02-22 14:19:52 23/02/22 19:19:52 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-22 14:19:52 23/02/22 19:19:52 INFO SecurityManager: Changing view acls to: root 2023-02-22 14:19:52 23/02/22 19:19:52 INFO SecurityManager: Changing modify acls to: root 2023-02-22 14:19:52 23/02/22 19:19:52 INFO SecurityManager: Changing view acls groups to: 2023-02-22 14:19:52 23/02/22 19:19:52 INFO SecurityManager: Changing modify acls groups to: 2023-02-22 14:19:52 23/02/22 19:19:52 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); groups with view permissions: Set(); users with modify permissions: Set(root); groups with modify permissions: Set() 2023-02-22 14:19:52 23/02/22 19:19:52 INFO FsHistoryProvider: History server ui acls disabled; users with admin permissions: ; groups with admin permissions: 2023-02-22 14:19:52 Exception in thread "main" java.lang.reflect.InvocationTargetException 2023-02-22 14:19:52 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 2023-02-22 14:19:52 at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) 2023-02-22 14:19:52 at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 2023-02-22 14:19:52 at java.lang.reflect.Constructor.newInstance(Constructor.java:423) 2023-02-22 14:19:52 at org.apache.spark.deploy.history.HistoryServer$.main(HistoryServer.scala:300) 2023-02-22 14:19:52 at org.apache.spark.deploy.history.HistoryServer.main(HistoryServer.scala) 2023-02-22 14:19:52 Caused by: org.apache.hadoop.fs.UnsupportedFileSystemException: No FileSystem for scheme "s3" 2023-02-22 14:19:52 at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:3281) 2023-02-22 14:19:52 at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3301) 2023-02-22 14:19:52 at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:124) 2023-02-22 14:19:52 at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3352) 2023-02-22 14:19:52 at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3320) 2023-02-22 14:19:52 at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:479) 2023-02-22 14:19:52 at org.apache.hadoop.fs.Path.getFileSystem(Path.java:365) 2023-02-22 14:19:52 at org.apache.spark.deploy.history.FsHistoryProvider.<init>(FsHistoryProvider.scala:116) 2023-02-22 14:19:52 at org.apache.spark.deploy.history.FsHistoryProvider.<init>(FsHistoryProvider.scala:88) 2023-02-22 14:19:52 ... 6 more

[purnima1612]

@moomindani Could you please help us in this ?

[purnima1612]

Issues is resolved . I was getting issue because I was passing

-Dspark.history.fs.logDirectory=s3:// name instead of s3a://

[moomindani]

Thanks for reporting that the issue has been resolved