This PR adds coverage of the decoding pipeline to our test suite. The tests themselves could use greater specificity - these are generic checks of the right object type or column names in a dataframe. Maybe a future PR could use this infrastructure to ensure the data are processed correctly?
This motivated a couple edits to the pipeline itself
decoding_merge: previously had a couple places that ran fetch1 on unrestricted/uninstanced class objects. Swapping out for merge_restrict_class fixed these issues
decoding_merge.create_decoding_view: previously expected a tuple from fetch_linear_position_info, which caused errors because it returns just the dataframe
clusterless.py and core.py: swapped out raising error for logging error to align with broader package insert patterns
core.py: Partially addressing #1158 with a check of whether or not vars will work on the passed params.
`unit_annotation
Checklist:
[x] No. This PR should be accompanied by a release: (yes/no/unsure)
[x] N/a. If release, I have updated the CITATION.cff
[x] No. This PR makes edits to table definitions: (yes/no)
[x] N/a. If table edits, I have included an alter snippet for release notes.
[x] N/a. If this PR makes changes to position, I ran the relevant tests locally.
[x] I have updated the CHANGELOG.md with PR number and description.
[x] N/a. I have added/edited docs/notebooks to reflect the changes
Description
This PR adds coverage of the decoding pipeline to our test suite. The tests themselves could use greater specificity - these are generic checks of the right object type or column names in a dataframe. Maybe a future PR could use this infrastructure to ensure the data are processed correctly?
Coverage
``` ---------- coverage: platform linux, python 3.9.19-final-0 ----------- Name Stmts Miss Cover Missing ---------------------------------------------------------------------------------------- src/spyglass/common/common_behav.py 276 42 85% 48-52, 85, 200, 228, 314-318, 322-351, 354, 398-399, 440, 453, 465-471, 501, 531, 544-546, 564, 642-645, 648-649, 655, 678-682 src/spyglass/common/common_dandi.py 122 122 0% 1-332 src/spyglass/common/common_device.py 224 42 81% 91, 124, 292-304, 308, 402-403, 422-452, 464, 573-611, 656, 710-713, 737, 770-774, 816-834 src/spyglass/common/common_dio.py 52 3 94% 41-45, 129 src/spyglass/common/common_ephys.py 313 178 43% 74, 114, 184, 194-203, 228-270, 305-310, 313, 327, 400-404, 439, 474-565, 569-578, 582-583, 648-734, 759-958, 962-963 src/spyglass/common/common_filter.py 188 19 90% 23-24, 92-96, 99-102, 138-139, 341-344, 381, 392-400, 474 src/spyglass/common/common_interval.py 195 16 92% 52-53, 120-125, 140-145, 163-166, 536, 546 src/spyglass/common/common_lab.py 111 11 90% 62, 69-70, 154-155, 269, 277-278, 310, 315-316 src/spyglass/common/common_nwbfile.py 246 107 57% 214, 228, 245-246, 289-309, 353-356, 363, 370, 449-525, 552-619, 636-655, 676-677, 691, 696-701, 709-714, 723, 727-731, 764, 784-798 src/spyglass/common/common_position.py 256 25 90% 28-30, 184, 354-355, 362-363, 674-740 src/spyglass/common/common_region.py 14 0 100% src/spyglass/common/common_sensors.py 24 2 92% 38-39 src/spyglass/common/common_session.py 70 8 89% 155-159, 171, 177, 189-192 src/spyglass/common/common_subject.py 21 3 86% 40-41, 63 src/spyglass/common/common_task.py 108 33 69% 34-40, 122-123, 139-142, 166, 196-225, 241, 251-258 src/spyglass/common/common_usage.py 148 32 78% 108, 129-130, 197, 206, 214, 251, 263-267, 275-279, 313, 338-342, 346-357 src/spyglass/common/errors.py 2 0 100% src/spyglass/common/populate_all_common.py 49 12 76% 40-46, 89-92, 161-163, 166-172 src/spyglass/common/prepopulate/prepopulate.py 44 31 30% 19, 24-74, 86-95 src/spyglass/common/signal_processing.py 16 16 0% 1-57 src/spyglass/decoding/decoding_merge.py 77 11 86% 44, 63-64, 82-83, 119-121, 139-147 src/spyglass/decoding/utils.py 15 8 47% 30-39 src/spyglass/decoding/v1/clusterless.py 169 20 88% 239-242, 271, 343, 553-601 src/spyglass/decoding/v1/core.py 94 12 87% 59, 72-89, 254, 260, 262, 277 src/spyglass/decoding/v1/dj_decoder_conversion.py 73 5 93% 32, 168-169, 188-189 src/spyglass/decoding/v1/sorted_spikes.py 156 29 81% 199-202, 230, 302, 356-372, 411, 421, 480-526 src/spyglass/decoding/v1/utils.py 6 0 100% src/spyglass/decoding/v1/waveform_features.py 108 10 91% 119, 132-134, 189, 272-273, 293-296, 358 src/spyglass/lfp/analysis/v1/lfp_band.py 136 10 93% 107, 242, 244, 264, 278-282, 361-365, 435, 463 src/spyglass/lfp/lfp_electrode.py 24 0 100% src/spyglass/lfp/lfp_imported.py 12 0 100% src/spyglass/lfp/lfp_merge.py 22 0 100% src/spyglass/lfp/v1/lfp.py 66 4 94% 128-132, 192-193 src/spyglass/lfp/v1/lfp_artifact.py 56 25 55% 139-210 src/spyglass/lfp/v1/lfp_artifact_MAD_detection.py 37 29 22% 42-63, 81, 107, 129-137, 157-176 src/spyglass/lfp/v1/lfp_artifact_difference_detection.py 79 69 13% 72-224, 253-280 src/spyglass/linearization/merge.py 14 0 100% src/spyglass/linearization/utils.py 0 0 100% src/spyglass/linearization/v0/main.py 49 24 51% 62-64, 71-74, 88-92, 127-192, 196 src/spyglass/linearization/v1/main.py 55 6 89% 66-69, 83-87, 203 src/spyglass/position/position_merge.py 22 0 100% src/spyglass/position/v1/dlc_reader.py 112 24 79% 26, 40, 46-47, 53, 59-60, 63, 72, 76, 82-83, 141-143, 154, 158-171, 223, 227 src/spyglass/position/v1/dlc_utils.py 371 152 59% 58, 61, 69, 72, 97-100, 104, 149-163, 186, 212-225, 230-237, 249-263, 278-296, 315, 334, 415, 424, 430, 465, 478-509, 513-540, 559-574, 588-634, 639-664, 734, 807, 818 src/spyglass/position/v1/dlc_utils_makevid.py 246 118 52% 70, 80, 118-119, 168-169, 195-199, 221, 236-244, 259, 318-340, 346-480, 483-490, 498-505, 508-557 src/spyglass/position/v1/position_dlc_centroid.py 113 8 93% 185, 208-216, 222, 239, 262 src/spyglass/position/v1/position_dlc_cohort.py 47 1 98% 122 src/spyglass/position/v1/position_dlc_model.py 122 27 78% 36-40, 104, 207, 222, 236, 296-340 src/spyglass/position/v1/position_dlc_orient.py 75 10 87% 76, 140-160, 172 src/spyglass/position/v1/position_dlc_pose_estimation.py 139 4 97% 75, 82, 128, 266 src/spyglass/position/v1/position_dlc_position.py 180 32 82% 57, 107, 217-218, 224-230, 367-369, 375, 398, 401, 423, 462-482 src/spyglass/position/v1/position_dlc_project.py 214 90 58% 92, 94, 130-203, 270, 299, 311-316, 328, 344-362, 406-407, 438-445, 452-455, 479-504 src/spyglass/position/v1/position_dlc_selection.py 125 2 98% 236, 393 src/spyglass/position/v1/position_dlc_training.py 89 5 94% 49, 130-131, 190-191 src/spyglass/position/v1/position_trodes_position.py 105 3 97% 70, 307-308 src/spyglass/spikesorting/analysis/v1/group.py 92 34 63% 78-80, 109-128, 178, 189-197, 201, 216, 234-249, 279, 294 src/spyglass/spikesorting/analysis/v1/unit_annotation.py 41 2 95% 75, 138 src/spyglass/spikesorting/imported.py 64 35 45% 49-58, 65, 72, 94-113, 121-134, 139-150 src/spyglass/spikesorting/spikesorting_merge.py 91 19 79% 106, 134-142, 214-229, 260 src/spyglass/spikesorting/utils.py 97 48 51% 76-78, 89, 102-103, 111-115, 120-124, 146-152, 161-204, 237, 243-248, 250-255, 260-278, 282 src/spyglass/spikesorting/v1/artifact.py 86 25 71% 69, 101-102, 256, 297-331, 350-372 src/spyglass/spikesorting/v1/curation.py 157 32 80% 144-162, 241-266, 357-364, 388, 414, 439-443, 478-480 src/spyglass/spikesorting/v1/figurl_curation.py 86 55 36% 47-58, 69-103, 125-169, 175-176, 188, 206-273 src/spyglass/spikesorting/v1/metric_curation.py 191 29 85% 88, 134, 139-141, 162, 193-194, 260, 313, 403, 448, 457-458, 462, 507-526, 585 src/spyglass/spikesorting/v1/metric_utils.py 19 5 74% 43-50, 58 src/spyglass/spikesorting/v1/recording.py 202 29 86% 82, 85, 122, 157-158, 245, 384-395, 417-432, 453-460, 481, 486, 509-510 src/spyglass/spikesorting/v1/sorting.py 118 20 83% 100, 131-132, 205, 209, 256, 272-278, 322-351, 387 src/spyglass/spikesorting/v1/utils.py 27 15 44% 62-109 src/spyglass/utils/database_settings.py 99 15 85% 147, 162, 173-175, 179-183, 196, 212-215, 223, 226 src/spyglass/utils/dj_graph.py 488 34 93% 38-39, 55-56, 125, 263, 306, 375, 407, 473, 482, 522-531, 629, 646-648, 663, 716, 929, 949, 955-956, 992, 1020, 1111-1113, 1126, 1136, 1138, 1158, 1182 src/spyglass/utils/dj_helper_fn.py 195 94 52% 78, 101, 167-177, 182-191, 196-199, 292, 301-302, 313, 341-342, 372-436, 452-459, 479-495, 511-523, 538-539, 555-569 src/spyglass/utils/dj_merge_tables.py 301 76 75% 35, 37, 43-47, 74-75, 91, 153, 179, 235, 241, 323-325, 345, 351-352, 361, 413, 419, 457-463, 484-503, 533, 614, 620, 631, 678, 686, 708-714, 718, 725, 748, 767, 803-804, 814, 824-830, 834-844, 851-854, 879, 884, 914-925 src/spyglass/utils/dj_mixin.py 368 66 82% 238-249, 391-431, 452-453, 461, 513, 549, 589, 605, 617-618, 626-636, 650-656, 675-678, 761, 768, 911, 937-938, 958-1026, 1037-1038 src/spyglass/utils/logging.py 17 2 88% 27-28 src/spyglass/utils/nwb_helper_fn.py 175 36 79% 53-90, 109, 154-159, 202, 205, 214, 238, 262-263, 307, 325, 435, 548, 565, 574-592 src/spyglass/utils/position.py 13 2 85% 27-28 src/spyglass/utils/spikesorting.py 9 2 78% 13, 18 src/spyglass/utils/sql_helper_fn.py 84 17 80% 45, 58-62, 91, 150, 184-185, 195-205 ---------------------------------------------------------------------------------------- TOTAL 8707 2132 76% ===================== 279 passed, 12 skipped in 202.96s (0:03:22) ===================== ```This motivated a couple edits to the pipeline itself
decoding_merge: previously had a couple places that ranfetch1on unrestricted/uninstanced class objects. Swapping out formerge_restrict_classfixed these issuesdecoding_merge.create_decoding_view: previously expected a tuple fromfetch_linear_position_info, which caused errors because it returns just the dataframeclusterless.pyandcore.py: swapped out raising error for logging error to align with broader package insert patternscore.py: Partially addressing #1158 with a check of whether or notvarswill work on the passed params.Checklist:
CITATION.cffaltersnippet for release notes.CHANGELOG.mdwith PR number and description.