sfu-db / dataprep

Open-source low code data preparation library in python. Collect, clean and visualization your data in python with a few lines of code.
http://dataprep.ai
MIT License
1.98k stars 203 forks source link

Clean: clean_phone for international prefixes that are not +1 when user select E164 #876

Open yixuy opened 2 years ago

yixuy commented 2 years ago

Describe the bug A clear and concise description of what the bug is.

When the user inputs the E164 as output_format while they are using clean_phone

If some phone number is not in Canada or US(prefix with +1) it will fail

To Reproduce Steps to reproduce the behavior:

  1. Go to Jupyter notebook
  2. Click on Phone number in the clean module selection
  3. Click on e164 in the selection and select the column(contains phone number) the user would like to clean
  4. Click on the OK
  5. See error

Or:

paste your code here
import pandas as pd
import numpy as np
df = pd.DataFrame({"Name":
                   ["Abby", "Scott", "Scott", "Scott2", np.nan, "NULL"],
                   "AGE":
                   [12, 33, 33, 56,  np.nan, "NULL"],
                   "weight__":
                   [32.5, 47.1, 47.1, 55.2, np.nan, "NULL"],
                   "Admission Date":
                   ["2020-01-01", "2020-01-15", "2020-01-15",
                    "2020-09-01", pd.NaT, "NULL"],
                   "email_address":
                   ["abby@gmail.com","scott@gmail.com", "scott@gmail.com", "test@abc.com", np.nan, "NULL"],
                   "Country of Birth":
                   ["CA","Canada", "Canada", "NULL", np.nan, "NULL"],
                  "Contact (Numbers)":
                   ["1-789-456-0123","1-123-456-7890","1-123-456-7890","86-456-123-7890", np.nan, "NULL" ],

})
df

from dataprep.clean import clean_df_gui
# import matplotlib
# %matplotlib ipympl
clean_df_gui(df)

Expected behavior A clear and concise description of what you expected to happen.

We expect the 86-456-123-7890 should be +864561237890 after cleaned

Screenshots If applicable, add screenshots to help explain your problem.

image

Desktop (please complete the following information):

Additional context Add any other context about the problem here.