search

biglocalnews / warn-transformer

Consolidate, enrich and republish the data gathered by warn-scraper
https://warn-transformer.readthedocs.io
Apache License 2.0
5 stars 3 forks source link

Fix California transformation error #109

Closed palewire closed 2 years ago

palewire commented 2 years ago

Seen here

Traceback (most recent call last):
  File "/home/runner/.local/share/virtualenvs/warn-github-flow-R1xICqqL/bin/warn-transformer", line 8, in <module>
    sys.exit(cli())
  File "/home/runner/.local/share/virtualenvs/warn-github-flow-R1xICqqL/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/runner/.local/share/virtualenvs/warn-github-flow-R1xICqqL/lib/python3.9/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/runner/.local/share/virtualenvs/warn-github-flow-R1xICqqL/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/runner/.local/share/virtualenvs/warn-github-flow-R1xICqqL/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/runner/.local/share/virtualenvs/warn-github-flow-R1xICqqL/lib/python3.9/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/runner/.local/share/virtualenvs/warn-github-flow-R1xICqqL/lib/python3.9/site-packages/warn_transformer/cli.py", line 79, in consolidate
    consolidate_runner.run(input_dir, source)
  File "/home/runner/.local/share/virtualenvs/warn-github-flow-R1xICqqL/lib/python3.9/site-packages/warn_transformer/consolidate.py", line 41, in run
    source_list = module.Transformer(input_dir).transform()
  File "/home/runner/.local/share/virtualenvs/warn-github-flow-R1xICqqL/lib/python3.9/site-packages/warn_transformer/schema.py", line 83, in transform
    transformed_list = [self.transform_row(r) for r in row_list]
  File "/home/runner/.local/share/virtualenvs/warn-github-flow-R1xICqqL/lib/python3.9/site-packages/warn_transformer/schema.py", line 83, in <listcomp>
    transformed_list = [self.transform_row(r) for r in row_list]
  File "/home/runner/.local/share/virtualenvs/warn-github-flow-R1xICqqL/lib/python3.9/site-packages/warn_transformer/schema.py", line 132, in transform_row
    jobs=self.transform_jobs(self.get_raw_value(row, self.fields["jobs"])),
  File "/home/runner/.local/share/virtualenvs/warn-github-flow-R1xICqqL/lib/python3.9/site-packages/warn_transformer/schema.py", line 304, in transform_jobs
    clean_value = self.jobs_corrections[value]
KeyError: 'Layoff Permanent'
palewire commented 2 years ago

Fixed upstream with our scraper.