crocs-muni / sec-certs

Tool for analysis of security certificates and their security targets (Common Criteria, NIST FIPS140-2...).
https://sec-certs.org
MIT License
9 stars 7 forks source link

FIPS algo download is broken #419

Open J08nY opened 1 week ago

J08nY commented 1 week ago

See https://github.com/crocs-muni/sec-certs/pull/418#issuecomment-2182677529.

KeyError: 'Validation Number'
  File "pandas/core/indexes/base.py", line 3802, in get_loc
    return self._engine.get_loc(casted_key)
  File "pandas/_libs/index.pyx", line 138, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 165, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 5745, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 5753, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'Validation Number'
  File "dramatiq/worker.py", line 485, in process_message
    res = actor(*message.args, **message.kwargs)
  File "dramatiq/actor.py", line 177, in __call__
    return self.fn(*args, **kwargs)
  File "sec_certs_page/common/tasks.py", line 503, in wrapper
    return f(*args, **kwargs)
  File "sec_certs_page/fips/tasks.py", line 171, in update_data
    updater.update()
  File "sec_certs_page/common/tasks.py", line 337, in update
    raise e
  File "sec_certs_page/common/tasks.py", line 253, in update
    to_reindex = self.process(dset, paths)
  File "sec_certs_page/fips/tasks.py", line 127, in process
    dset.process_auxiliary_datasets(update_json=False)
  File "sec_certs/serialization/json.py", line 108, in _serialize
    result = func(*args, **kwargs)
  File "sec_certs/dataset/fips.py", line 253, in process_auxiliary_datasets
    self.auxiliary_datasets.algorithm_dset = self._prepare_algorithm_dataset(download_fresh)
  File "sec_certs/utils/profiling.py", line 41, in wrapper
    return func(*args, **kwargs)
  File "sec_certs/dataset/fips.py", line 258, in _prepare_algorithm_dataset
    alg_dset = FIPSAlgorithmDataset.from_web(self.algorithm_dataset_path)
  File "sec_certs/dataset/fips_algorithm.py", line 58, in from_web
    algs = set(itertools.chain.from_iterable(FIPSAlgorithmDataset.parse_algorithms_from_html(x) for x in htmls))
  File "sec_certs/dataset/fips_algorithm.py", line 58, in <genexpr>
    algs = set(itertools.chain.from_iterable(FIPSAlgorithmDataset.parse_algorithms_from_html(x) for x in htmls))
  File "sec_certs/dataset/fips_algorithm.py", line 99, in parse_algorithms_from_html
    df["alg_type"] = df["Validation Number"].map(lambda x: re.sub(r"[0-9\s]", "", x))
  File "pandas/core/frame.py", line 3807, in __getitem__
    indexer = self.columns.get_loc(key)
  File "pandas/core/indexes/base.py", line 3804, in get_loc
    raise KeyError(key) from err
sentry-io[bot] commented 1 week ago

Sentry Issue: SECCERTS-1KB