search

TeamPiped / Piped

An alternative privacy-friendly YouTube frontend which is efficient by design.
https://piped.video
GNU Affero General Public License v3.0
8.19k stars 664 forks source link

Videos from the same channel are lumped together in feed with identical incorrect date #2489

Open Evilhotdog opened 1 year ago

Evilhotdog commented 1 year ago

Official Instance

Describe the bug

When looking at my feed, often five or more videos from the same channel appear strung together, all of them appearing a few days ago, despite the fact that many are months old (and say they're months old on the actual video page). On the official instance where I just set up an account, they all say "video not available" and have a black screen (#2487), but on my instance, this appears a separate issue.

An example: image

To Reproduce

  1. Open the feed

Expected behavior

The videos appear in chronological order

Logs/Errors

Frontend:

InstallTrigger is deprecated and will be removed in the future. constants.js:50:14
Failed to get subsystem status for purpose 
Object { rejected: true, message: Error }
content-script.js:95:21
Uncaught TypeError: this.videosStore is null
    loadMoreVideos https://piped.solarpunk.moe/assets/FeedPage-ef80efa5.js:1
    handleScroll https://piped.solarpunk.moe/assets/FeedPage-ef80efa5.js:1
FeedPage-ef80efa5.js:1:1973

Backend (unsure if related):

ThrottlingCache: 0 entries
SLF4J: No SLF4J providers were found.
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See https://www.slf4j.org/codes.html#noProviders for further details.
Jun 01, 2023 9:21:16 PM org.hibernate.Version logVersion
INFO: HHH000412: Hibernate ORM core version [WORKING]
Jun 01, 2023 9:21:16 PM org.hibernate.cfg.Environment <clinit>
INFO: HHH000406: Using bytecode reflection optimizer
Jun 01, 2023 9:21:18 PM org.hibernate.engine.jdbc.connections.internal.ConnectionProviderInitiator initiateService
INFO: HHH000130: Instantiating explicit connection provider: org.hibernate.hikaricp.internal.HikariCPConnectionProvider
Logged in as user: null
Room ID: !CfXSiQMnWTEPfnBNuK:matrix.org
Filter ID: null
Jun 01, 2023 9:21:19 PM org.hibernate.bytecode.internal.BytecodeProviderInitiator buildBytecodeProvider
INFO: HHH000021: Bytecode provider name : bytebuddy
Jun 01, 2023 9:21:21 PM org.hibernate.engine.transaction.jta.platform.internal.JtaPlatformInitiator initiateService
INFO: HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform]
Database connection is ready!
Cleanup: Removed 2 old videos
Jun 01, 2023 9:23:30 PM org.hibernate.engine.jdbc.batch.internal.BatchImpl release
INFO: HHH100503: On release of batch it still contained JDBC statements
Jun 01, 2023 9:23:30 PM org.hibernate.engine.jdbc.batch.internal.BatchImpl lambda$performExecution$2
ERROR: HHH100501: Exception executing batch [java.sql.BatchUpdateException: Batch entry 0 insert into videos (uploader_id,duration,is_short,thumbnail,title,uploaded,views,id) values ('UCm2pbaivKA-9poyXZ4rexrw',392,'FALSE','https://i.ytimg.com/vi_webp/oF91plAoADA/maxresdefault.webp','I am a serial killer (My side of the story)',1685654608686,0,'oF91plAoADA') was aborted: ERROR: duplicate key value violates unique constraint "videos_pkey"
  Detail: Key (id)=(oF91plAoADA) already exists.  Call getNextException to see other errors in the batch.], SQL: insert into videos (uploader_id,duration,is_short,thumbnail,title,uploaded,views,id) values (?,?,?,?,?,?,?,?)
Jun 01, 2023 9:23:30 PM org.hibernate.engine.jdbc.spi.SqlExceptionHelper logExceptions
WARN: SQL Error: 0, SQLState: 23505
Jun 01, 2023 9:23:30 PM org.hibernate.engine.jdbc.spi.SqlExceptionHelper logExceptions
ERROR: Batch entry 0 insert into videos (uploader_id,duration,is_short,thumbnail,title,uploaded,views,id) values ('UCm2pbaivKA-9poyXZ4rexrw',392,'FALSE','https://i.ytimg.com/vi_webp/oF91plAoADA/maxresdefault.webp','I am a serial killer (My side of the story)',1685654608686,0,'oF91plAoADA') was aborted: ERROR: duplicate key value violates unique constraint "videos_pkey"
  Detail: Key (id)=(oF91plAoADA) already exists.  Call getNextException to see other errors in the batch.
Jun 01, 2023 9:23:30 PM org.hibernate.engine.jdbc.spi.SqlExceptionHelper logExceptions
ERROR: ERROR: duplicate key value violates unique constraint "videos_pkey"
  Detail: Key (id)=(oF91plAoADA) already exists.
org.hibernate.exception.ConstraintViolationException: could not execute batch [Batch entry 0 insert into videos (uploader_id,duration,is_short,thumbnail,title,uploaded,views,id) values ('UCm2pbaivKA-9poyXZ4rexrw',392,'FALSE','https://i.ytimg.com/vi_webp/oF91plAoADA/maxresdefault.webp','I am a serial killer (My side of the story)',1685654608686,0,'oF91plAoADA') was aborted: ERROR: duplicate key value violates unique constraint "videos_pkey"
  Detail: Key (id)=(oF91plAoADA) already exists.  Call getNextException to see other errors in the batch.] [insert into videos (uploader_id,duration,is_short,thumbnail,title,uploaded,views,id) values (?,?,?,?,?,?,?,?)]
    at org.hibernate.exception.internal.SQLStateConversionDelegate.convert(SQLStateConversionDelegate.java:95)
    at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:56)
    at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:108)
    at org.hibernate.engine.jdbc.batch.internal.BatchImpl.lambda$performExecution$2(BatchImpl.java:289)
    at org.hibernate.engine.jdbc.mutation.internal.PreparedStatementGroupSingleTable.forEachStatement(PreparedStatementGroupSingleTable.java:59)
    at org.hibernate.engine.jdbc.batch.internal.BatchImpl.performExecution(BatchImpl.java:262)
    at org.hibernate.engine.jdbc.batch.internal.BatchImpl.execute(BatchImpl.java:241)
    at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.executeBatch(JdbcCoordinatorImpl.java:188)
    at org.hibernate.internal.StatelessSessionImpl.managedFlush(StatelessSessionImpl.java:465)
    at org.hibernate.internal.StatelessSessionImpl.flushBeforeTransactionCompletion(StatelessSessionImpl.java:627)
    at org.hibernate.internal.StatelessSessionImpl.beforeTransactionCompletion(StatelessSessionImpl.java:600)
    at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.beforeTransactionCompletion(JdbcCoordinatorImpl.java:439)
    at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.beforeCompletionCallback(JdbcResourceLocalTransactionCoordinatorImpl.java:169)
    at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:267)
    at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:101)
    at me.kavin.piped.utils.VideoHelpers.handleNewVideo(VideoHelpers.java:52)
    at me.kavin.piped.utils.VideoHelpers.handleNewVideo(VideoHelpers.java:23)
    at me.kavin.piped.utils.matrix.SyncRunner.lambda$run$3(SyncRunner.java:177)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.sql.BatchUpdateException: Batch entry 0 insert into videos (uploader_id,duration,is_short,thumbnail,title,uploaded,views,id) values ('UCm2pbaivKA-9poyXZ4rexrw',392,'FALSE','https://i.ytimg.com/vi_webp/oF91plAoADA/maxresdefault.webp','I am a serial killer (My side of the story)',1685654608686,0,'oF91plAoADA') was aborted: ERROR: duplicate key value violates unique constraint "videos_pkey"
  Detail: Key (id)=(oF91plAoADA) already exists.  Call getNextException to see other errors in the batch.
    at org.postgresql.jdbc.BatchResultHandler.handleError(BatchResultHandler.java:165)
    at org.postgresql.core.ResultHandlerDelegate.handleError(ResultHandlerDelegate.java:52)
    at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2402)
    at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:574)
    at org.postgresql.jdbc.PgStatement.internalExecuteBatch(PgStatement.java:896)
    at org.postgresql.jdbc.PgStatement.executeBatch(PgStatement.java:919)
    at org.postgresql.jdbc.PgPreparedStatement.executeBatch(PgPreparedStatement.java:1685)
    at com.zaxxer.hikari.pool.ProxyStatement.executeBatch(ProxyStatement.java:127)
    at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeBatch(HikariProxyPreparedStatement.java)
    at org.hibernate.engine.jdbc.batch.internal.BatchImpl.lambda$performExecution$2(BatchImpl.java:275)
    ... 19 more
Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "videos_pkey"
  Detail: Key (id)=(oF91plAoADA) already exists.
    at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2713)
    at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2401)
    ... 26 more
ThrottlingCache: 36 entries
Cleanup: Removed 0 old videos
ThrottlingCache: 34 entries
Cleanup: Removed 0 old videos
ThrottlingCache: 36 entries
Cleanup: Removed 0 old videos
ThrottlingCache: 7 entries
Cleanup: Removed 0 old videos
ThrottlingCache: 13 entries
Cleanup: Removed 0 old videos
ThrottlingCache: 4 entries
Cleanup: Removed 0 old videos
ThrottlingCache: 6 entries
Cleanup: Removed 0 old videos
ThrottlingCache: 6 entries
Cleanup: Removed 0 old videos
ThrottlingCache: 2 entries
Cleanup: Removed 0 old videos
ThrottlingCache: 2 entries
Cleanup: Removed 0 old videos
ThrottlingCache: 62 entries
Cleanup: Removed 1 old videos
ThrottlingCache: 0 entries
Cleanup: Removed 0 old videos
ThrottlingCache: 0 entries
Cleanup: Removed 0 old videos
ThrottlingCache: 12 entries
Cleanup: Removed 0 old videos
ThrottlingCache: 10 entries
Cleanup: Removed 0 old videos
ThrottlingCache: 12 entries
Cleanup: Removed 0 old videos
com.fasterxml.jackson.core.JsonParseException: Unexpected character ('<' (code 60)): expected a valid value (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
 at [Source: (okio.RealBufferedSource$inputStream$1); line: 1, column: 2]
    at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:2477)
    at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:750)
    at com.fasterxml.jackson.core.base.ParserMinimalBase._reportUnexpectedChar(ParserMinimalBase.java:674)
    at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._handleUnexpectedValue(UTF8StreamJsonParser.java:2790)
    at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._nextTokenNotInObject(UTF8StreamJsonParser.java:908)
    at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.nextToken(UTF8StreamJsonParser.java:794)
    at com.fasterxml.jackson.databind.ObjectMapper._readTreeAndClose(ObjectMapper.java:4854)
    at com.fasterxml.jackson.databind.ObjectMapper.readTree(ObjectMapper.java:3199)
    at me.kavin.piped.utils.RequestUtils.getJsonNode(RequestUtils.java:38)
    at me.kavin.piped.utils.matrix.SyncRunner.run(SyncRunner.java:97)
    at java.base/java.lang.Thread.run(Thread.java:833)
java.lang.RuntimeException: Failed to parse JSON
    at me.kavin.piped.utils.RequestUtils.getJsonNode(RequestUtils.java:42)
    at me.kavin.piped.utils.matrix.SyncRunner.run(SyncRunner.java:97)
    at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: com.fasterxml.jackson.core.JsonParseException: Unexpected character ('<' (code 60)): expected a valid value (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
 at [Source: (okio.RealBufferedSource$inputStream$1); line: 1, column: 2]
    at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:2477)
    at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:750)
    at com.fasterxml.jackson.core.base.ParserMinimalBase._reportUnexpectedChar(ParserMinimalBase.java:674)
    at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._handleUnexpectedValue(UTF8StreamJsonParser.java:2790)
    at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._nextTokenNotInObject(UTF8StreamJsonParser.java:908)
    at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.nextToken(UTF8StreamJsonParser.java:794)
    at com.fasterxml.jackson.databind.ObjectMapper._readTreeAndClose(ObjectMapper.java:4854)
    at com.fasterxml.jackson.databind.ObjectMapper.readTree(ObjectMapper.java:3199)
    at me.kavin.piped.utils.RequestUtils.getJsonNode(RequestUtils.java:38)
    ... 2 more
ThrottlingCache: 14 entries
Cleanup: Removed 0 old videos
ThrottlingCache: 42 entries
Cleanup: Removed 0 old videos

Browser, and OS with Version.

Firefox 113.0/2 on latest Arch Linux

Additional context

This also happens through Libretube

Ammako commented 1 year ago

It's been a thing #2040 #1130

Evilhotdog commented 1 year ago

I've looked through each, and can't find an immediate way of fixing it. For instance, opening the channel in a new tab does not change anything. Is there something in those threads that I'm missing?

FireMasterK commented 1 year ago

I'm pretty sure this is caused by #2487

We need checks in NewPipeExtractor to fail hard when this happens... Otherwise, the feed gets affected like this

Evilhotdog commented 1 year ago

Wait, even if the cases of this happening on my own instance don't have "Video not available"?

Ammako commented 1 year ago

Is there something in those threads that I'm missing?

I'm more so mentioning that this is a recurrent issue that's been happening from time to time; the exact cause has never really been clear though. Your case is a lot worse than mine was though.

DrCode32 commented 1 year ago

![Uploading Screenshot_20230603-082358_Chrome.png…]() Me too!