Closed olegs closed 2 years ago
According to this stack trace, index on tsp vector was successfully built, however adding multiple indexes shouldn't be performed in a single transaction.
2022-08-20 13:37:23,256 INFO [main] o.j.b.p.s.SemanticScholarLoader [SemanticScholarLoader.kt:47] Config path: /home/ubuntu/.pubtrends/config.prope
rties
2022-08-20 13:37:23,288 INFO [main] o.j.b.p.s.SemanticScholarLoader [SemanticScholarLoader.kt:51] Init Postgresql database connection
2022-08-20 13:37:23,290 INFO [main] o.j.b.p.d.SemanticScholarPostgresWriter [SemanticScholarPostgresWriter.kt:25] Initializing DB connection
2022-08-20 13:37:23,356 INFO [main] o.j.b.p.d.SemanticScholarPostgresWriter [SemanticScholarPostgresWriter.kt:32] Init transaction starting
2022-08-20 13:37:23,366 INFO [main] o.j.b.p.d.SemanticScholarPostgresWriter [SemanticScholarPostgresWriter.kt:34] Creating schema
2022-08-20 13:37:24,203 INFO [main] o.j.b.p.d.SemanticScholarPostgresWriter [SemanticScholarPostgresWriter.kt:37] Adding TSV column
2022-08-20 13:37:24,225 INFO [main] o.j.b.p.d.SemanticScholarPostgresWriter [SemanticScholarPostgresWriter.kt:40] Adding primary key, required for
batch update
2022-08-20 13:37:24,265 INFO [main] o.j.b.p.d.SemanticScholarPostgresWriter [SemanticScholarPostgresWriter.kt:57] Creating index ss_title_abstract_
index
^[[A^[[B^[[C^[[D^[[A^[[B^[[A^[[B^[[C^[[D2022-08-21 09:18:55,067 INFO [main] o.j.b.p.d.SemanticScholarPostgresWriter [SemanticScholarPostgresWriter.
kt:61] Creating citations material view matview_sscitations
2022-08-21 09:52:50,493 WARN [main] Exposed [ThreadLocalTransactionManager.kt:169] Transaction attempt #0 failed: org.postgresql.util.PSQLException
: An I/O error occurred while sending to the backend.. Statement(s):
create materialized view if not exists matview_sscitations as
SELECT ssid_in as ssid, crc32id_in as crc32id, COUNT(*) AS count
FROM SSCitations C
GROUP BY ssid, crc32id
HAVING COUNT(*) >= 3; -- Ignore tail of 0,1,2 cited papers
create index if not exists SSCitation_matview_index on matview_sscitations using hash(crc32id);
org.jetbrains.exposed.exceptions.ExposedSQLException: org.postgresql.util.PSQLException: An I/O error occurred while sending to the backend.
at org.jetbrains.exposed.sql.statements.Statement.executeIn$exposed(Statement.kt:61)
at org.jetbrains.exposed.sql.Transaction.exec(Transaction.kt:184)
at org.jetbrains.exposed.sql.Transaction.exec(Transaction.kt:126)
at org.jetbrains.exposed.sql.Transaction.exec(Transaction.kt:102)
at org.jetbrains.exposed.sql.Transaction.exec(Transaction.kt:93)
Indexes building split into separate transactions is implemented in https://github.com/JetBrains-Research/pubtrends/commit/c90dc89886481904773dc17da994e7e27f416eea
Database log: