Norconex Filesystem Collector is a flexible crawler for collecting, parsing, and manipulating data ranging from local hard drives to network locations into various data repositories such as search engines.
I am using norconex filesystem crawler 2.9.1 and want to commit to a postgre database 13.4. The database requires user authentication which is SCRAM-based. In the collector config I specified the following commiter:
[..]
INFO [AbstractCrawler] Crawler For ProSi Reports: Crawler finishing: committing documents.
INFO [AbstractFileQueueCommitter] Committing 266 files
INFO [SQLCommitter] Sending 100 commit operations to SQL database.
INFO [SQLCommitter] Checking if table "crawler.crawl_1" exists...
INFO [SQLCommitter] Table "crawler.crawl_1" does not exist. Attempting to create it...
INFO [AbstractCrawler] Crawler For ProSi Reports: Crawler executed in 4 seconds.
ERROR [JobSuite] Execution failed for job: Crawler For ProSi Reports
com.norconex.committer.core.CommitterException: Could not commit batch to database.
at com.norconex.committer.sql.SQLCommitter.commitBatch(SQLCommitter.java:480)
at com.norconex.committer.core.AbstractBatchCommitter.commitAndCleanBatch(AbstractBatchCommitter.java:179)
at com.norconex.committer.core.AbstractBatchCommitter.cacheOperationAndCommitIfReady(AbstractBatchCommitter.java:208)
at com.norconex.committer.core.AbstractBatchCommitter.commitAddition(AbstractBatchCommitter.java:143)
at com.norconex.committer.core.AbstractFileQueueCommitter.commit(AbstractFileQueueCommitter.java:222)
at com.norconex.committer.sql.SQLCommitter.commit(SQLCommitter.java:425)
at com.norconex.collector.core.crawler.AbstractCrawler.execute(AbstractCrawler.java:274)
at com.norconex.collector.core.crawler.AbstractCrawler.doExecute(AbstractCrawler.java:228)
at com.norconex.collector.core.crawler.AbstractCrawler.startExecution(AbstractCrawler.java:184)
at com.norconex.jef4.job.AbstractResumableJob.execute(AbstractResumableJob.java:49)
at com.norconex.jef4.suite.JobSuite.runJob(JobSuite.java:357)
at com.norconex.jef4.suite.JobSuite.doExecute(JobSuite.java:294)
at com.norconex.jef4.suite.JobSuite.execute(JobSuite.java:167)
at com.norconex.collector.core.AbstractCollector.start(AbstractCollector.java:150)
at com.norconex.collector.core.AbstractCollectorLauncher.launch(AbstractCollectorLauncher.java:95)
at com.norconex.collector.fs.FilesystemCollector.main(FilesystemCollector.java:76)
Caused by: java.sql.SQLException: Cannot create PoolableConnectionFactory (The server requested SCRAM-based authentication, but no password was provided.)
at org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:2294)
at org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:2039)
at org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1533)
at org.apache.commons.dbutils.AbstractQueryRunner.prepareConnection(AbstractQueryRunner.java:204)
at org.apache.commons.dbutils.QueryRunner.update(QueryRunner.java:419)
at com.norconex.committer.sql.SQLCommitter.ensureTable(SQLCommitter.java:607)
at com.norconex.committer.sql.SQLCommitter.commitBatch(SQLCommitter.java:463)
... 15 more
Caused by: org.postgresql.util.PSQLException: The server requested SCRAM-based authentication, but no password was provided.
at org.postgresql.core.v3.ConnectionFactoryImpl.lambda$doAuthentication$4(ConnectionFactoryImpl.java:791)
at org.postgresql.core.v3.AuthenticationPluginManager.withPassword(AuthenticationPluginManager.java:81)
at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:788)
at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:180)
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:235)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
at org.postgresql.jdbc.PgConnection.(PgConnection.java:223)
at org.postgresql.Driver.makeConnection(Driver.java:402)
at org.postgresql.Driver.connect(Driver.java:261)
at org.apache.commons.dbcp2.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:39)
at org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:256)
at org.apache.commons.dbcp2.BasicDataSource.validateConnectionFactory(BasicDataSource.java:2304)
at org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:2290)
... 21 more
INFO [JobSuite] Running Crawler For ProSi Reports: END (Mon May 23 09:41:40 CEST 2022)
It says no password provided, but I did. Could you help me?
I am using norconex filesystem crawler 2.9.1 and want to commit to a postgre database 13.4. The database requires user authentication which is SCRAM-based. In the collector config I specified the following commiter:
And this is the stacktrace I get:
It says no password provided, but I did. Could you help me?
Regards!