petl-developers / petl

Python Extract Transform and Load Tables of Data
MIT License
1.22k stars 190 forks source link

Fix other functions to conform with PEP 479 #645

Closed augustomen closed 10 months ago

augustomen commented 11 months ago

As noted in https://github.com/petl-developers/petl/issues/575, there are still many places where RuntimeError could be raised as a result of an unhandled StopIteration when calling next(), due to PEP 479, which was implemented in Python 3.5.

This PR has the goal of properly handling headerless tables. This differs from empty tables in that an empty table at least has a header row (which could have no items in it - see petl.util.base.empty()). The real world use case for headerless tables are empty data-only CSVs (where the header is previously agreed between two systems), or development starting from an empty list - as is the case with all added tests.

A previous PR (https://github.com/petl-developers/petl/pull/613) solved the issue for the function iterrowslice. This PR expands this fix to (almost) all such calls. Although a little bit repetitive, I've decided to try..except all calls and handle them individually.

Changes

  1. Handled every call to hdr = next(it) that could potentially raise an exception.
  2. Added test cases for all these calls.

Notes

In some places, it made sense to let the exception be raised regardless, and in others it made sense to include a header row where one wasn't before. Namely:

Checklist

Use this checklist to ensure the quality of pull requests that include new code and/or make changes to existing code.

juarezr commented 10 months ago

@augustomen,

Sorry for the delay. I have to fix the GitHub actions before evaluating these changes.

Thanks for the PR.

juarezr commented 10 months ago

@augustomen,

As the CI was broken when you submitted this PR, I would like to check if everything is ok now after the fixes.

Could you rebase this PR before we proceed, please?

juarezr commented 10 months ago

Rebased and merged in #653