nasa-gcn / gcn.nasa.gov

General Coordinates Network (GCN) web site
https://gcn.nasa.gov
Other
184 stars 44 forks source link

backfill synonyms for all circulars once eventId has been backfilled #2662

Open Courey opened 3 weeks ago

Courey commented 3 weeks ago
import { tables } from '@architect/functions'
import { ReturnValue } from '@aws-sdk/client-dynamodb'
import type { DynamoDBDocument } from '@aws-sdk/lib-dynamodb'
import { paginateScan, UpdateCommand } from '@aws-sdk/lib-dynamodb'

import {
  type Circular,
  parseEventFromSubject,
} from '~/routes/circulars/circulars.lib'

export async function backfill() {
  console.log('Starting backfill...')
  const db = await tables()
  const client = db._doc as unknown as DynamoDBDocument
  const TableName = db.name('circulars')
  const pages = paginateScan(
    { client },
    {
      TableName,
    }
  )
  for await (const page of pages) {
    for (const record of page.Items || []) {
      const circular = record as unknown as Circular
      const validEvent =
        circular.eventId || parseEventFromSubject(circular.subject)
      if (!circular.eventId && validEvent) {
         tryInitSynonym(validEvent)
      }
    }
  }
  console.log('... End backfill')
}
await backfill()
Courey commented 1 week ago

Blocked until eventIds are backfilled

lpsinger commented 1 week ago

@Courey, would you please post that code snippet on a gist so that we can make line comments on it?

Courey commented 3 days ago

https://gist.github.com/Courey/7d0424b9a3cd0d30b54e88d6a1053960 This is the synonym only backfill