Song analysis are missing from Platform graphQL in Prod.

[lindaxiang]

Prod RDPC graphQL does not get song index properly?

https://api.rdpc-prod.cumulus.genomeinformatics.org/graphql 

The analysis exist on https://song.rdpc-prod.cumulus.genomeinformatics.org/, for example the following analysis from P1000-US: a3aca246-9aef-4110-aca2-469aefb11084 a977a85a-6e35-425c-b7a8-5a6e35225ccf

While using the following graphQL call can't get back the analysis:

query {
  analyses(filter: { analysisId: "a3aca246-9aef-4110-aca2-469aefb11084"}) {
    info {
      contentCount
    }
  }
}

[joneubank]

Both of these files are being tracked by our file manager. They both correspond to the same donor and that donor is not core complete so the analysis's files have not started embargo yet. This is correctly not available on the site or through the api.

[joneubank]

I misread the issue believing the problem to be the missing document in the platform.

On further review it is clear that this analysis was not indexed into the RDPC ES and is therefore not queryable throught the RDPC Gateway.

This analysis is known by the platform's file service, therefore Kafka messaging is working. The error is therefore between kafkfa, maestro, and ElasticSearch. I will investigate further.

[joneubank]

Restarted Maestro and it was able to fix its connection with kafka - Maestro indexed all the analyses and files that were missing.

There are a lot of logs that relate to some sort of kafka consumer issue, but it seems to not be functionally related to this bug. Just mentioning here for reference. Hopefully this bug doesn't occur again but if it does maybe this is related (some sort of kafka disconneciton issue?).

The maestro logs are spitting out this message once every minute:

2024-01-10 | 18:02:14.766 | 1 | pool-5-thread-2 | INFO | org.apache.kafka.clients.consumer.internals.ConsumerCoordinator | [Consumer clientId=consumer-requestsConsumerGrp-5, groupId=requestsConsumerGrp] Found no committed offset for partition maestro_index_requests-0