BCHSI / philter-ucsf

Open source clinical text de-identification
BSD 3-Clause "New" or "Revised" License
111 stars 49 forks source link

Update regular expressions to move global flags at the front of regular expressions #23

Open fbraun4358 opened 5 months ago

fbraun4358 commented 5 months ago

In python 3.10, global flags, like (?i) will throw a FutureWarning about global flags not being at the front. In python 3.11+, this now becomes an exception.

See this issue: https://github.com/python/cpython/issues/83575

We have done this on our end as we have moved to python 3.12, but due to corporate policy I can't easily propose those changes in a PR so I wanted to let you know.

The easy solution is to just move all global flags in all regular expressions to the front. As far as I can tell, (?i) is the only used global flag. We wrote a script to just replace (?i) in the regexs and then prepend (?i) in those cases where we removed it.

These are the regexes which need updating. Sorry I can't actually easily share the fixes. I wish I could.

addresses\at_street_dash_street_transformed.txt
addresses\at_street_number_dash_street_transformed.txt
addresses\box_room_transformed.txt
addresses\box_transformed.txt
addresses\corner_of_street_&_street_transformed.txt
addresses\county_name_transformed.txt
addresses\desk_#_transformed.txt
addresses\floor_box_transformed.txt
addresses\full_street_address_transformed.txt
addresses\full_street_address_with_concatenated_indicator_transformed.txt
addresses\num_streetname_city_transformed.txt
addresses\num_streetname_extension_transformed.txt
addresses\num_streetname_transformed.txt
addresses\number_streetname_noindicator_suite_transformed.txt
addresses\room_#_transformed.txt
addresses\room_box_transformed.txt
addresses\short_street_name_transformed.txt
addresses\state_indicator_transformed.txt
addresses\street_and_street_transformed.txt
addresses\streetname_floor_number_transformed.txt
addresses\streetname_only_transformed.txt
addresses\to_state_transformed.txt
addresses\waiting_room_transformed.txt
age\x_year_old.txt
age\x_yo.txt
contact\call_#.txt
dates\as_of_date.txt
mrn_id\accession_#.txt
mrn_id\activation_code.txt
mrn_id\cassette_#.txt
mrn_id\file_indicator.txt
mrn_id\id_verbose.txt
mrn_id\lot_#.txt
mrn_id\order_number.txt
mrn_id\specimen_#.txt
mrn_id\ssn.txt
mrn_id\tape_number.txt
safe\a1c_safe.txt
safe\able_to_safe.txt
safe\active_safe.txt
safe\age_safe.txt
safe\airway_safe.txt
safe\alert_safe.txt
safe\AM_safe.txt
safe\arms_safe.txt
safe\assessment_&_plan_safe.txt
safe\assessment_safe.txt
safe\at_measurement_safe.txt
safe\attending_safe.txt
safe\axis_safe.txt
safe\baby_safe.txt
safe\back_safe.txt
safe\balance_safe.txt
safe\barretts_esophagus_safe.txt
safe\base_safe.txt
safe\be_safe.txt
safe\beats_safe.txt
safe\bee_safe.txt
safe\below_safe.txt
safe\bile_safe.txt
safe\birth_safe.txt
safe\bp_safe.txt
safe\bruits_safe.txt
safe\call_safe.txt
safe\cancer_safe.txt
safe\case_safe.txt
safe\cava_safe.txt
safe\central_safe.txt
safe\chief_safe.txt
safe\child_safe.txt
safe\code_safe.txt
safe\colon_safe.txt
safe\commonly_safe.txt
safe\contact_safe.txt
safe\contractions_safe.txt
safe\cord_safe.txt
safe\crohns_disease_safe.txt
safe\day_safe.txt
safe\days_safe.txt
safe\dial_safe.txt
safe\disposition_safe.txt
safe\distance_safe.txt
safe\distensable_safe.txt
safe\do_safe.txt
safe\doppler_safe.txt
safe\down_syndrome_safe.txt
safe\drop_safe.txt
safe\due_safe.txt
safe\edge_safe.txt
safe\effort_safe.txt
safe\est_safe.txt
safe\fall_safe.txt
safe\fax_safe.txt
safe\few_safe.txt
safe\fiber_safe.txt
safe\file_safe.txt
safe\flexion_extension_safe.txt
safe\floor_safe.txt
safe\found_safe.txt
safe\go_safe.txt
safe\grade_safe.txt
safe\gross_safe.txt
safe\hearing_safe.txt
safe\hour_safe.txt
safe\how_safe.txt
safe\id_safe.txt
safe\independence_safe.txt
safe\index_safe.txt
safe\intake_safe.txt
safe\key_safe.txt
safe\knee_safe.txt
safe\lab_safe.txt
safe\last_safe.txt
safe\lb_safe.txt
safe\learn_safe.txt
safe\loop_safe.txt
safe\male_safe.txt
safe\max_safe.txt
safe\may_safe.txt
safe\measurement_safe.txt
safe\medical_safe.txt
safe\medication_safe.txt
safe\micro_safe.txt
safe\morning_safe.txt
safe\mri_safe.txt
safe\na_safe.txt
safe\night_safe.txt
safe\non_safe.txt
safe\not_safe.txt
safe\onset_safe.txt
safe\oral_safe.txt
safe\order_md_safe.txt
safe\ordering_md_safe.txt
safe\other_safe.txt
safe\pain_safe.txt
safe\pap_safe.txt
safe\parkinson_safe.txt
safe\peak_safe.txt
safe\pounds_safe.txt
safe\prn_safe.txt
safe\reviewed_by_safe.txt
safe\sci_notation_safe.txt
safe\scope_safe.txt
safe\smoking_safe.txt
safe\tablet_safe.txt
safe\the_safe.txt
safe\units_safe.txt
safe\young_safe.txt
salutations\code_status_name.txt
salutations\confirmed_by_name.txt
salutations\dr_ambiguous.txt
salutations\editor_name.txt
salutations\name_age.txt
salutations\name_indicator.txt
salutations\ordering_md.txt
salutations\sent_note_to.txt
salutations\wrote.txt
ucsf_regex\ucsf_apex_safe.txt
ucsf_regex\ucsf_bay_area.txt
ucsf_regex\ucsf_neighborhoods.txt