microsoft / sql-spark-connector

Apache Spark Connector for SQL Server and Azure SQL
Apache License 2.0
273 stars 116 forks source link

not supported for java spark 3.3 scala 1.12? #218

Open yezhengli-Mr9 opened 1 year ago

yezhengli-Mr9 commented 1 year ago

not supported for java spark 3.3 scala 1.12?, seems spark 3.3 scala 1.12 only works for scala/spark

        <!-- https://mvnrepository.com/artifact/com.microsoft.azure/spark-mssql-connector -->
        <dependency>
            <groupId>com.microsoft.azure</groupId>
            <artifactId>spark-mssql-connector_1.12</artifactId>
            <version>1.3.0-BETA</version>
        </dependency>

Need to identify something like

        configMap = new HashMap<>();
        configMap.put(SqlConstants.URL, connectionConfig.getUrl());
        configMap.put(SqlConstants.DATABASE_NAME, connectionConfig.getDatabase());
        configMap.put(SqlConstants.USER, connectionConfig.getUsername());
        configMap.put(SqlConstants.PASSWORD, connectionConfig.getPassword());
        if (!Optional.ofNullable(readerConfig.getQuery()).isPresent()) {
            configMap.put(DB_TABLE, finalTableName);
        } else {
            configMap.put(QUERY_CUSTOM, readerConfig.getQuery());
        }
SqlDBConfigBuilder configBuilder = new SqlDBConfigBuilder(JavaConverters
                .mapAsScalaMapConverter(configMap)
                .asScala().toMap(Predef.<Tuple2<String, Object>>conforms()));

        DataFrameReader dataFrameReader = new DataFrameReader(sparkSession);
        setOptions(dataFrameReader, readerConfig.getOptions());
        DataFrameReaderFunctions dataFrameReaderFunctions =
                new DataFrameReaderFunctions(dataFrameReader);