search

singularity-energy / open-grid-emissions

Tools for producing high-quality hourly generation and emissions data for U.S. electric grids
MIT License
67 stars 4 forks source link

Format and Lint existing python files #317

Closed rouille closed 6 months ago

rouille commented 7 months ago

Purpose

Format existing Python files using ruff. Use ruff to automate and standardize formatting. Rules are listed in the newly created pyproject.toml. The configuration for formatting are the same as black. A GitHub workflow (Lint) is created to check that python files and notebooks are well formatted on push and pr (if not already triggered by push).

What the code is doing

N/A

Testing

N/A

Where to look

Usage Example/Visuals

The following instances have been formated/linted manually (not fixable by ruff):

error: Failed to format notebooks/explore_data/manually_import_uncontrolled_emission_factors.ipynb: source contains syntax errors: ParseError { error: UnrecognizedToken(String { value: "boiler_bottom_type", kind: String, triple_quoted: false }, None), offset: 1011, source_path: "<filename>" }
error: Failed to format notebooks/validation/hourly_validation.ipynb: source contains syntax errors: ParseError { error: UnrecognizedToken(Newline, None), offset: 6, source_path: "<filename>" }

src/column_checks.py:463:9: F601 Dictionary key literal `"nox_control_id"` repeated
src/data_cleaning.py:3:8: F401 [*] `os` imported but unused
src/eia930.py:15:1: E402 Module level import not at top of file
src/output_data.py:465:16: E721 Do not compare types, use `isinstance()`
src/validation.py:223:35: E741 Ambiguous variable name: `l`
test/test_logging.py:9:1: E402 Module level import not at top of file
test/test_logging.py:9:22: F401 [*] `src.eia930` imported but unused
test/test_logging.py:10:1: E402 Module level import not at top of file
test/test_logging.py:12:1: E402 Module level import not at top of file

Review estimate

5min

Future work

Do we want to enforce W501 for comments, docstrings and strings? If set it generates around 1000 violations

Checklist

grgmiller commented 7 months ago

Do we want to enforce E501 for comments, docstrings and strings? If set it generates around 1000 violations

This may be a good practice. Is this something that could be auto-formatted if enforced, or would we need to manually edit all of the comments? If we can auto-format, we should probably do it.

rouille commented 7 months ago

Do we want to enforce W501 for comments, docstrings and strings? If set it generates around 1000 violations

This may be a good practice. Is this something that could be auto-formatted if enforced, or would we need to manually edit all of the comments? If we can auto-format, we should probably do it.

These are the issues when we add E501:

(open-grid-emissions) [~/Singularity/open-grid-emissions] (ben/format) brdo$ ruff check .
src/column_checks.py:4:89: E501 Line too long (90 > 88)
src/column_checks.py:488:89: E501 Line too long (106 > 88)
src/column_checks.py:498:89: E501 Line too long (95 > 88)
src/consumed.py:108:89: E501 Line too long (105 > 88)
src/consumed.py:125:89: E501 Line too long (132 > 88)
src/consumed.py:243:89: E501 Line too long (91 > 88)
src/consumed.py:281:89: E501 Line too long (93 > 88)
src/consumed.py:332:89: E501 Line too long (91 > 88)
src/consumed.py:460:89: E501 Line too long (94 > 88)
src/consumed.py:510:89: E501 Line too long (95 > 88)
src/consumed.py:520:89: E501 Line too long (99 > 88)
src/data_cleaning.py:51:89: E501 Line too long (118 > 88)
src/data_cleaning.py:111:89: E501 Line too long (111 > 88)
src/data_cleaning.py:182:89: E501 Line too long (93 > 88)
src/data_cleaning.py:198:89: E501 Line too long (116 > 88)
src/data_cleaning.py:199:89: E501 Line too long (112 > 88)
src/data_cleaning.py:203:89: E501 Line too long (105 > 88)
src/data_cleaning.py:204:89: E501 Line too long (113 > 88)
src/data_cleaning.py:205:89: E501 Line too long (104 > 88)
src/data_cleaning.py:206:89: E501 Line too long (111 > 88)
src/data_cleaning.py:207:89: E501 Line too long (105 > 88)
src/data_cleaning.py:210:89: E501 Line too long (108 > 88)
src/data_cleaning.py:212:89: E501 Line too long (101 > 88)
src/data_cleaning.py:213:89: E501 Line too long (97 > 88)
src/data_cleaning.py:219:89: E501 Line too long (105 > 88)
src/data_cleaning.py:220:89: E501 Line too long (111 > 88)
src/data_cleaning.py:221:89: E501 Line too long (113 > 88)
src/data_cleaning.py:222:89: E501 Line too long (109 > 88)
src/data_cleaning.py:223:89: E501 Line too long (118 > 88)
src/data_cleaning.py:224:89: E501 Line too long (112 > 88)
src/data_cleaning.py:225:89: E501 Line too long (116 > 88)
src/data_cleaning.py:226:89: E501 Line too long (119 > 88)
src/data_cleaning.py:227:89: E501 Line too long (114 > 88)
src/data_cleaning.py:229:89: E501 Line too long (111 > 88)
src/data_cleaning.py:231:89: E501 Line too long (115 > 88)
src/data_cleaning.py:232:89: E501 Line too long (119 > 88)
src/data_cleaning.py:233:89: E501 Line too long (123 > 88)
src/data_cleaning.py:234:89: E501 Line too long (129 > 88)
src/data_cleaning.py:235:89: E501 Line too long (104 > 88)
src/data_cleaning.py:236:89: E501 Line too long (137 > 88)
src/data_cleaning.py:237:89: E501 Line too long (128 > 88)
src/data_cleaning.py:241:89: E501 Line too long (115 > 88)
src/data_cleaning.py:244:89: E501 Line too long (127 > 88)
src/data_cleaning.py:248:89: E501 Line too long (102 > 88)
src/data_cleaning.py:254:89: E501 Line too long (110 > 88)
src/data_cleaning.py:255:89: E501 Line too long (105 > 88)
src/data_cleaning.py:262:89: E501 Line too long (112 > 88)
src/data_cleaning.py:263:89: E501 Line too long (103 > 88)
src/data_cleaning.py:264:89: E501 Line too long (92 > 88)
src/data_cleaning.py:274:89: E501 Line too long (96 > 88)
src/data_cleaning.py:286:89: E501 Line too long (106 > 88)
src/data_cleaning.py:287:89: E501 Line too long (107 > 88)
src/data_cleaning.py:299:89: E501 Line too long (117 > 88)
src/data_cleaning.py:305:89: E501 Line too long (98 > 88)
src/data_cleaning.py:308:89: E501 Line too long (89 > 88)
src/data_cleaning.py:314:89: E501 Line too long (109 > 88)
src/data_cleaning.py:326:89: E501 Line too long (102 > 88)
src/data_cleaning.py:329:89: E501 Line too long (158 > 88)
src/data_cleaning.py:364:89: E501 Line too long (128 > 88)
src/data_cleaning.py:391:89: E501 Line too long (158 > 88)
src/data_cleaning.py:423:89: E501 Line too long (102 > 88)
src/data_cleaning.py:425:89: E501 Line too long (98 > 88)
src/data_cleaning.py:468:89: E501 Line too long (89 > 88)
src/data_cleaning.py:577:89: E501 Line too long (114 > 88)
src/data_cleaning.py:579:89: E501 Line too long (92 > 88)
src/data_cleaning.py:589:89: E501 Line too long (92 > 88)
src/data_cleaning.py:590:89: E501 Line too long (98 > 88)
src/data_cleaning.py:659:89: E501 Line too long (96 > 88)
src/data_cleaning.py:662:89: E501 Line too long (100 > 88)
src/data_cleaning.py:771:89: E501 Line too long (100 > 88)
src/data_cleaning.py:833:89: E501 Line too long (93 > 88)
src/data_cleaning.py:904:89: E501 Line too long (106 > 88)
src/data_cleaning.py:905:89: E501 Line too long (114 > 88)
src/data_cleaning.py:906:89: E501 Line too long (127 > 88)
src/data_cleaning.py:920:89: E501 Line too long (113 > 88)
src/data_cleaning.py:933:89: E501 Line too long (110 > 88)
src/data_cleaning.py:959:89: E501 Line too long (103 > 88)
src/data_cleaning.py:1030:89: E501 Line too long (92 > 88)
src/data_cleaning.py:1031:89: E501 Line too long (136 > 88)
src/data_cleaning.py:1061:89: E501 Line too long (106 > 88)
src/data_cleaning.py:1071:89: E501 Line too long (97 > 88)
src/data_cleaning.py:1154:89: E501 Line too long (95 > 88)
src/data_cleaning.py:1165:89: E501 Line too long (90 > 88)
src/data_cleaning.py:1166:89: E501 Line too long (94 > 88)
src/data_cleaning.py:1218:89: E501 Line too long (91 > 88)
src/data_cleaning.py:1220:89: E501 Line too long (98 > 88)
src/data_cleaning.py:1221:89: E501 Line too long (94 > 88)
src/data_cleaning.py:1243:89: E501 Line too long (105 > 88)
src/data_cleaning.py:1244:89: E501 Line too long (89 > 88)
src/data_cleaning.py:1302:89: E501 Line too long (107 > 88)
src/data_cleaning.py:1304:89: E501 Line too long (127 > 88)
src/data_cleaning.py:1335:89: E501 Line too long (132 > 88)
src/data_cleaning.py:1350:89: E501 Line too long (113 > 88)
src/data_cleaning.py:1351:89: E501 Line too long (131 > 88)
src/data_cleaning.py:1352:89: E501 Line too long (89 > 88)
src/data_cleaning.py:1375:89: E501 Line too long (104 > 88)
src/data_cleaning.py:1435:89: E501 Line too long (90 > 88)
src/data_cleaning.py:1436:89: E501 Line too long (93 > 88)
src/data_cleaning.py:1438:89: E501 Line too long (91 > 88)
src/data_cleaning.py:1440:89: E501 Line too long (94 > 88)
src/data_cleaning.py:1441:89: E501 Line too long (93 > 88)
src/data_cleaning.py:1443:89: E501 Line too long (92 > 88)
src/data_cleaning.py:1444:89: E501 Line too long (91 > 88)
src/data_cleaning.py:1445:89: E501 Line too long (92 > 88)
src/data_cleaning.py:1447:89: E501 Line too long (91 > 88)
src/data_cleaning.py:1448:89: E501 Line too long (95 > 88)
src/data_cleaning.py:1486:89: E501 Line too long (91 > 88)
src/data_cleaning.py:1505:89: E501 Line too long (98 > 88)
src/data_cleaning.py:1525:89: E501 Line too long (96 > 88)
src/data_cleaning.py:1529:89: E501 Line too long (97 > 88)
src/data_cleaning.py:1537:89: E501 Line too long (93 > 88)
src/data_cleaning.py:1538:89: E501 Line too long (91 > 88)
src/data_cleaning.py:1555:89: E501 Line too long (89 > 88)
src/data_cleaning.py:1556:89: E501 Line too long (90 > 88)
src/data_cleaning.py:1577:89: E501 Line too long (93 > 88)
src/data_cleaning.py:1623:89: E501 Line too long (105 > 88)
src/data_cleaning.py:1628:89: E501 Line too long (89 > 88)
src/data_cleaning.py:1642:89: E501 Line too long (92 > 88)
src/data_cleaning.py:1671:89: E501 Line too long (110 > 88)
src/data_cleaning.py:1686:89: E501 Line too long (91 > 88)
src/data_cleaning.py:1709:89: E501 Line too long (106 > 88)
src/data_cleaning.py:1776:89: E501 Line too long (151 > 88)
src/data_cleaning.py:1816:89: E501 Line too long (92 > 88)
src/data_cleaning.py:1916:89: E501 Line too long (112 > 88)
src/data_cleaning.py:1965:89: E501 Line too long (114 > 88)
src/data_cleaning.py:2052:89: E501 Line too long (110 > 88)
src/data_cleaning.py:2078:89: E501 Line too long (97 > 88)
src/data_cleaning.py:2106:89: E501 Line too long (97 > 88)
src/data_cleaning.py:2109:89: E501 Line too long (109 > 88)
src/data_cleaning.py:2112:89: E501 Line too long (112 > 88)
src/data_cleaning.py:2147:89: E501 Line too long (110 > 88)
src/data_cleaning.py:2148:89: E501 Line too long (118 > 88)
src/data_cleaning.py:2149:89: E501 Line too long (106 > 88)
src/data_pipeline.py:36:89: E501 Line too long (143 > 88)
src/data_pipeline.py:42:89: E501 Line too long (113 > 88)
src/data_pipeline.py:83:89: E501 Line too long (135 > 88)
src/data_pipeline.py:87:89: E501 Line too long (94 > 88)
src/data_pipeline.py:132:89: E501 Line too long (90 > 88)
src/data_pipeline.py:143:89: E501 Line too long (107 > 88)
src/data_pipeline.py:406:89: E501 Line too long (117 > 88)
src/data_pipeline.py:409:89: E501 Line too long (117 > 88)
src/data_pipeline.py:416:89: E501 Line too long (94 > 88)
src/data_pipeline.py:470:89: E501 Line too long (103 > 88)
src/data_pipeline.py:473:89: E501 Line too long (98 > 88)
src/data_pipeline.py:554:89: E501 Line too long (104 > 88)
src/data_pipeline.py:556:89: E501 Line too long (97 > 88)
src/download_data.py:131:89: E501 Line too long (101 > 88)
src/eia930.py:117:89: E501 Line too long (93 > 88)
src/eia930.py:129:89: E501 Line too long (115 > 88)
src/eia930.py:401:89: E501 Line too long (96 > 88)
src/eia930.py:443:89: E501 Line too long (90 > 88)
src/eia930.py:475:89: E501 Line too long (89 > 88)
src/emissions.py:24:89: E501 Line too long (136 > 88)
src/emissions.py:85:89: E501 Line too long (91 > 88)
src/emissions.py:164:89: E501 Line too long (96 > 88)
src/emissions.py:172:89: E501 Line too long (103 > 88)
src/emissions.py:180:89: E501 Line too long (96 > 88)
src/emissions.py:284:89: E501 Line too long (114 > 88)
src/emissions.py:286:89: E501 Line too long (129 > 88)
src/emissions.py:320:89: E501 Line too long (94 > 88)
src/emissions.py:344:89: E501 Line too long (100 > 88)
src/emissions.py:355:89: E501 Line too long (123 > 88)
src/emissions.py:412:89: E501 Line too long (92 > 88)
src/emissions.py:413:89: E501 Line too long (95 > 88)
src/emissions.py:449:89: E501 Line too long (168 > 88)
src/emissions.py:450:89: E501 Line too long (134 > 88)
src/emissions.py:475:89: E501 Line too long (111 > 88)
src/emissions.py:521:89: E501 Line too long (89 > 88)
src/emissions.py:550:89: E501 Line too long (100 > 88)
src/emissions.py:555:89: E501 Line too long (119 > 88)
src/emissions.py:663:89: E501 Line too long (137 > 88)
src/emissions.py:695:89: E501 Line too long (118 > 88)
src/emissions.py:761:89: E501 Line too long (117 > 88)
src/emissions.py:763:89: E501 Line too long (115 > 88)
src/emissions.py:766:89: E501 Line too long (104 > 88)
src/emissions.py:767:89: E501 Line too long (99 > 88)
src/emissions.py:785:89: E501 Line too long (91 > 88)
src/emissions.py:819:89: E501 Line too long (95 > 88)
src/emissions.py:854:89: E501 Line too long (179 > 88)
src/emissions.py:857:89: E501 Line too long (113 > 88)
src/emissions.py:985:89: E501 Line too long (93 > 88)
src/emissions.py:1009:89: E501 Line too long (93 > 88)
src/emissions.py:1010:89: E501 Line too long (105 > 88)
src/emissions.py:1018:89: E501 Line too long (92 > 88)
src/emissions.py:1063:89: E501 Line too long (96 > 88)
src/emissions.py:1064:89: E501 Line too long (96 > 88)
src/emissions.py:1078:89: E501 Line too long (108 > 88)
src/emissions.py:1124:89: E501 Line too long (107 > 88)
src/emissions.py:1131:89: E501 Line too long (98 > 88)
src/emissions.py:1160:89: E501 Line too long (119 > 88)
src/emissions.py:1177:89: E501 Line too long (168 > 88)
src/emissions.py:1178:89: E501 Line too long (141 > 88)
src/emissions.py:1210:89: E501 Line too long (111 > 88)
src/emissions.py:1276:89: E501 Line too long (100 > 88)
src/emissions.py:1281:89: E501 Line too long (119 > 88)
src/emissions.py:1363:89: E501 Line too long (94 > 88)
src/emissions.py:1384:89: E501 Line too long (137 > 88)
src/emissions.py:1413:89: E501 Line too long (137 > 88)
src/emissions.py:1534:89: E501 Line too long (97 > 88)
src/emissions.py:1614:89: E501 Line too long (93 > 88)
src/emissions.py:1687:89: E501 Line too long (92 > 88)
src/emissions.py:1688:89: E501 Line too long (105 > 88)
src/emissions.py:1690:89: E501 Line too long (109 > 88)
src/emissions.py:1692:89: E501 Line too long (116 > 88)
src/emissions.py:1705:89: E501 Line too long (89 > 88)
src/emissions.py:1714:89: E501 Line too long (98 > 88)
src/emissions.py:1736:89: E501 Line too long (90 > 88)
src/emissions.py:1781:89: E501 Line too long (93 > 88)
src/emissions.py:1814:89: E501 Line too long (101 > 88)
src/emissions.py:1837:89: E501 Line too long (100 > 88)
src/gross_to_net_generation.py:25:89: E501 Line too long (116 > 88)
src/gross_to_net_generation.py:29:89: E501 Line too long (126 > 88)
src/gross_to_net_generation.py:96:89: E501 Line too long (99 > 88)
src/gross_to_net_generation.py:279:89: E501 Line too long (96 > 88)
src/gross_to_net_generation.py:311:89: E501 Line too long (101 > 88)
src/gross_to_net_generation.py:400:89: E501 Line too long (97 > 88)
src/gross_to_net_generation.py:468:89: E501 Line too long (100 > 88)
src/gross_to_net_generation.py:475:89: E501 Line too long (123 > 88)
src/gross_to_net_generation.py:488:89: E501 Line too long (97 > 88)
src/gross_to_net_generation.py:489:89: E501 Line too long (98 > 88)
src/gross_to_net_generation.py:494:89: E501 Line too long (109 > 88)
src/gross_to_net_generation.py:505:89: E501 Line too long (114 > 88)
src/gross_to_net_generation.py:506:89: E501 Line too long (94 > 88)
src/gross_to_net_generation.py:507:89: E501 Line too long (94 > 88)
src/gross_to_net_generation.py:516:89: E501 Line too long (107 > 88)
src/gross_to_net_generation.py:527:89: E501 Line too long (113 > 88)
src/gross_to_net_generation.py:627:89: E501 Line too long (96 > 88)
src/gross_to_net_generation.py:649:89: E501 Line too long (98 > 88)
src/gross_to_net_generation.py:662:89: E501 Line too long (109 > 88)
src/gross_to_net_generation.py:683:89: E501 Line too long (121 > 88)
src/gross_to_net_generation.py:684:89: E501 Line too long (99 > 88)
src/gross_to_net_generation.py:686:89: E501 Line too long (92 > 88)
src/gross_to_net_generation.py:708:89: E501 Line too long (118 > 88)
src/gross_to_net_generation.py:815:89: E501 Line too long (121 > 88)
src/gross_to_net_generation.py:828:89: E501 Line too long (136 > 88)
src/impute_hourly_profiles.py:91:89: E501 Line too long (94 > 88)
src/impute_hourly_profiles.py:120:89: E501 Line too long (91 > 88)
src/impute_hourly_profiles.py:165:89: E501 Line too long (89 > 88)
src/impute_hourly_profiles.py:169:89: E501 Line too long (116 > 88)
src/impute_hourly_profiles.py:192:89: E501 Line too long (112 > 88)
src/impute_hourly_profiles.py:278:89: E501 Line too long (123 > 88)
src/impute_hourly_profiles.py:295:89: E501 Line too long (111 > 88)
src/impute_hourly_profiles.py:301:89: E501 Line too long (120 > 88)
src/impute_hourly_profiles.py:328:89: E501 Line too long (90 > 88)
src/impute_hourly_profiles.py:377:89: E501 Line too long (98 > 88)
src/impute_hourly_profiles.py:378:89: E501 Line too long (100 > 88)
src/impute_hourly_profiles.py:423:89: E501 Line too long (108 > 88)
src/impute_hourly_profiles.py:479:89: E501 Line too long (96 > 88)
src/impute_hourly_profiles.py:523:89: E501 Line too long (96 > 88)
src/impute_hourly_profiles.py:611:89: E501 Line too long (93 > 88)
src/impute_hourly_profiles.py:719:89: E501 Line too long (119 > 88)
src/impute_hourly_profiles.py:793:89: E501 Line too long (90 > 88)
src/impute_hourly_profiles.py:794:89: E501 Line too long (90 > 88)
src/impute_hourly_profiles.py:795:89: E501 Line too long (95 > 88)
src/impute_hourly_profiles.py:863:89: E501 Line too long (99 > 88)
src/impute_hourly_profiles.py:909:89: E501 Line too long (104 > 88)
src/impute_hourly_profiles.py:910:89: E501 Line too long (122 > 88)
src/impute_hourly_profiles.py:993:89: E501 Line too long (91 > 88)
src/impute_hourly_profiles.py:1113:89: E501 Line too long (89 > 88)
src/impute_hourly_profiles.py:1177:89: E501 Line too long (90 > 88)
src/impute_hourly_profiles.py:1244:89: E501 Line too long (94 > 88)
src/impute_hourly_profiles.py:1265:89: E501 Line too long (104 > 88)
src/impute_hourly_profiles.py:1294:89: E501 Line too long (126 > 88)
src/impute_hourly_profiles.py:1295:89: E501 Line too long (117 > 88)
src/impute_hourly_profiles.py:1318:89: E501 Line too long (102 > 88)
src/impute_hourly_profiles.py:1332:89: E501 Line too long (117 > 88)
src/impute_hourly_profiles.py:1348:89: E501 Line too long (106 > 88)
src/impute_hourly_profiles.py:1353:89: E501 Line too long (101 > 88)
src/impute_hourly_profiles.py:1390:89: E501 Line too long (95 > 88)
src/impute_hourly_profiles.py:1416:89: E501 Line too long (106 > 88)
src/impute_hourly_profiles.py:1453:89: E501 Line too long (89 > 88)
src/impute_hourly_profiles.py:1468:89: E501 Line too long (103 > 88)
src/impute_hourly_profiles.py:1486:89: E501 Line too long (94 > 88)
src/impute_hourly_profiles.py:1503:89: E501 Line too long (94 > 88)
src/impute_hourly_profiles.py:1516:89: E501 Line too long (105 > 88)
src/impute_hourly_profiles.py:1548:89: E501 Line too long (211 > 88)
src/load_data.py:20:89: E501 Line too long (100 > 88)
src/load_data.py:42:89: E501 Line too long (101 > 88)
src/load_data.py:75:89: E501 Line too long (100 > 88)
src/load_data.py:189:89: E501 Line too long (102 > 88)
src/load_data.py:227:89: E501 Line too long (89 > 88)
src/load_data.py:256:89: E501 Line too long (105 > 88)
src/load_data.py:278:89: E501 Line too long (113 > 88)
src/load_data.py:287:89: E501 Line too long (119 > 88)
src/load_data.py:342:89: E501 Line too long (90 > 88)
src/load_data.py:360:89: E501 Line too long (91 > 88)
src/load_data.py:402:89: E501 Line too long (101 > 88)
src/load_data.py:441:89: E501 Line too long (113 > 88)
src/load_data.py:531:89: E501 Line too long (93 > 88)
src/load_data.py:533:89: E501 Line too long (110 > 88)
src/load_data.py:544:89: E501 Line too long (107 > 88)
src/load_data.py:545:89: E501 Line too long (107 > 88)
src/load_data.py:561:89: E501 Line too long (97 > 88)
src/load_data.py:586:89: E501 Line too long (109 > 88)
src/load_data.py:597:89: E501 Line too long (94 > 88)
src/load_data.py:685:89: E501 Line too long (160 > 88)
src/load_data.py:696:89: E501 Line too long (106 > 88)
src/load_data.py:731:89: E501 Line too long (92 > 88)
src/load_data.py:782:89: E501 Line too long (98 > 88)
src/load_data.py:795:89: E501 Line too long (117 > 88)
src/load_data.py:819:89: E501 Line too long (90 > 88)
src/load_data.py:820:89: E501 Line too long (107 > 88)
src/load_data.py:834:89: E501 Line too long (107 > 88)
src/load_data.py:867:89: E501 Line too long (90 > 88)
src/load_data.py:868:89: E501 Line too long (110 > 88)
src/load_data.py:885:89: E501 Line too long (95 > 88)
src/output_data.py:64:89: E501 Line too long (101 > 88)
src/output_data.py:75:89: E501 Line too long (92 > 88)
src/output_data.py:307:89: E501 Line too long (106 > 88)
src/output_data.py:323:89: E501 Line too long (106 > 88)
src/output_data.py:374:89: E501 Line too long (113 > 88)
src/output_data.py:406:89: E501 Line too long (107 > 88)
src/output_data.py:467:89: E501 Line too long (101 > 88)
src/output_data.py:482:89: E501 Line too long (94 > 88)
src/output_data.py:483:89: E501 Line too long (89 > 88)
src/output_data.py:484:89: E501 Line too long (94 > 88)
src/output_data.py:498:89: E501 Line too long (96 > 88)
src/validation.py:19:89: E501 Line too long (89 > 88)
src/validation.py:27:89: E501 Line too long (118 > 88)
src/validation.py:37:89: E501 Line too long (90 > 88)
src/validation.py:38:89: E501 Line too long (91 > 88)
src/validation.py:141:89: E501 Line too long (224 > 88)
src/validation.py:161:89: E501 Line too long (93 > 88)
src/validation.py:169:89: E501 Line too long (92 > 88)
src/validation.py:192:89: E501 Line too long (90 > 88)
src/validation.py:233:89: E501 Line too long (203 > 88)
src/validation.py:248:89: E501 Line too long (181 > 88)
src/validation.py:257:89: E501 Line too long (105 > 88)
src/validation.py:259:89: E501 Line too long (111 > 88)
src/validation.py:266:89: E501 Line too long (186 > 88)
src/validation.py:275:89: E501 Line too long (92 > 88)
src/validation.py:277:89: E501 Line too long (96 > 88)
src/validation.py:279:89: E501 Line too long (94 > 88)
src/validation.py:297:89: E501 Line too long (127 > 88)
src/validation.py:304:89: E501 Line too long (131 > 88)
src/validation.py:327:89: E501 Line too long (200 > 88)
src/validation.py:340:89: E501 Line too long (96 > 88)
src/validation.py:346:89: E501 Line too long (95 > 88)
src/validation.py:358:89: E501 Line too long (161 > 88)
src/validation.py:395:89: E501 Line too long (161 > 88)
src/validation.py:428:89: E501 Line too long (99 > 88)
src/validation.py:434:89: E501 Line too long (103 > 88)
src/validation.py:452:89: E501 Line too long (130 > 88)
src/validation.py:454:89: E501 Line too long (97 > 88)
src/validation.py:511:89: E501 Line too long (145 > 88)
src/validation.py:519:89: E501 Line too long (111 > 88)
src/validation.py:536:89: E501 Line too long (122 > 88)
src/validation.py:546:89: E501 Line too long (115 > 88)
src/validation.py:559:89: E501 Line too long (147 > 88)
src/validation.py:573:89: E501 Line too long (108 > 88)
src/validation.py:637:89: E501 Line too long (112 > 88)
src/validation.py:645:89: E501 Line too long (124 > 88)
src/validation.py:653:89: E501 Line too long (119 > 88)
src/validation.py:660:89: E501 Line too long (124 > 88)
src/validation.py:668:89: E501 Line too long (119 > 88)
src/validation.py:676:89: E501 Line too long (118 > 88)
src/validation.py:681:89: E501 Line too long (128 > 88)
src/validation.py:689:89: E501 Line too long (95 > 88)
src/validation.py:715:89: E501 Line too long (163 > 88)
src/validation.py:727:89: E501 Line too long (101 > 88)
src/validation.py:737:89: E501 Line too long (136 > 88)
src/validation.py:753:89: E501 Line too long (98 > 88)
src/validation.py:754:89: E501 Line too long (95 > 88)
src/validation.py:767:89: E501 Line too long (112 > 88)
src/validation.py:771:89: E501 Line too long (95 > 88)
src/validation.py:785:89: E501 Line too long (113 > 88)
src/validation.py:789:89: E501 Line too long (95 > 88)
src/validation.py:794:89: E501 Line too long (138 > 88)
src/validation.py:805:89: E501 Line too long (89 > 88)
src/validation.py:925:89: E501 Line too long (92 > 88)
src/validation.py:972:89: E501 Line too long (118 > 88)
src/validation.py:973:89: E501 Line too long (107 > 88)
src/validation.py:1013:89: E501 Line too long (122 > 88)
src/validation.py:1113:89: E501 Line too long (93 > 88)
src/validation.py:1136:89: E501 Line too long (94 > 88)
src/validation.py:1215:89: E501 Line too long (95 > 88)
src/validation.py:1232:89: E501 Line too long (133 > 88)
src/validation.py:1312:89: E501 Line too long (105 > 88)
src/validation.py:1328:89: E501 Line too long (91 > 88)
src/validation.py:1330:89: E501 Line too long (105 > 88)
src/validation.py:1370:89: E501 Line too long (93 > 88)
src/validation.py:1461:89: E501 Line too long (93 > 88)
src/validation.py:1575:89: E501 Line too long (176 > 88)
src/validation.py:1585:89: E501 Line too long (126 > 88)
src/validation.py:1594:89: E501 Line too long (133 > 88)
src/validation.py:1619:89: E501 Line too long (178 > 88)
src/validation.py:1626:89: E501 Line too long (136 > 88)
src/validation.py:1674:89: E501 Line too long (197 > 88)
src/validation.py:1677:89: E501 Line too long (160 > 88)
src/validation.py:1707:89: E501 Line too long (130 > 88)
src/validation.py:1716:89: E501 Line too long (142 > 88)
src/validation.py:1766:89: E501 Line too long (103 > 88)
src/validation.py:1767:89: E501 Line too long (103 > 88)
src/validation.py:1768:89: E501 Line too long (103 > 88)
src/validation.py:1769:89: E501 Line too long (130 > 88)
src/validation.py:1855:89: E501 Line too long (129 > 88)
src/validation.py:1856:89: E501 Line too long (128 > 88)
src/validation.py:1933:89: E501 Line too long (106 > 88)
src/validation.py:2112:89: E501 Line too long (130 > 88)
src/validation.py:2130:89: E501 Line too long (116 > 88)
src/visualization.py:8:89: E501 Line too long (96 > 88)
src/visualization.py:10:89: E501 Line too long (90 > 88)
Found 402 errors.

None are fixable using --fix option.

grgmiller commented 7 months ago

These are the issues when we add E501:

That's a long list. Would be nice to fix this over time, but not sure that it's a priority right now. Maybe we ignore this error for now. I'm going to add a vertical ruler at 88 characters to my vscode settings so that I can notice when a line is too long and fix over time.

rouille commented 7 months ago

These are the issues when we add E501:

That's a long list. Would be nice to fix this over time, but not sure that it's a priority right now. Maybe we ignore this error for now. I'm going to add a vertical ruler at 88 characters to my vscode settings so that I can notice when a line is too long and fix over time.

Docstring and comments are easily fixable. The hard work is breaking (f-)strings in print statements and loggers

rouille commented 7 months ago

@grgmiller , running python data_pipeline.py --year 2021 --small leads to the following error:

FileNotFoundError: [Errno 2] No such file or directory: '/Users/brdo/Singularity/open-grid-emissions/data/downloads/eia923/f923_2021/EIA923_Schedule_8_Annual_Environmental_Information_2021_Final.xlsx'

I fixed this issue in 277a74a52004a14728aefc994d5e6056b4e3ef02 as part of PR #313. Three options:

What do you prefer?

grgmiller commented 7 months ago

What do you prefer?

It looks like that was a one-line code change. Can you just make the same change in this branch and commit it here as well?

I just left some comments on the other PR to try and get that merged in soon as well.

rouille commented 6 months ago

What do you prefer?

It looks like that was a one-line code change. Can you just make the same change in this branch and commit it here as well?

I just left some comments on the other PR to try and get that merged in soon as well.

@grgmiller, I have rebased this branch onto development that now includes the fix and run the full pipeline successfully. However, the pipeline is not running with the small argument for the reason mentioned in #320.

I believe it is ready to be merged.

rouille commented 6 months ago

Thanks Ben, I think we may want to save merging this PR until after we've refactored the code with all the 2022 updates- I'm just afraid of a bunch of conflicts that will have to be resolved manually since we're doing some major updates. We'll want to re-apply the auto linting to the new code once refactored. Maybe as next steps:

1. We merge the `update_pudl_dependency` branch into `development` once that is done

2. We close this PR and create a new branch off of development, and copy over `lint.yml` and `pyproject.toml` and then apply all the linting

3. We merge all the formatting changes in, and then continue with our refactoring.

Let's merge update_pudl_dependency first. We don't need to close this PR. I will rebase onto development and reformat the files with ruff and update this PR

rouille commented 6 months ago

@grgmiller , the branch is rebased onto the latest development and the whole pipeline has been run successfully. I believe it is ready to be merged.