'Dataframe' object has no attribute 'distance'

Dataset used: openpath_prod_usaid_laos_ev

The below error was encountered while running script for notebook generic_metrics_sensed.

More information:

We recently introduced determination of 80th percentile computation code block for generic_metrics_sensed notebook.

cutoff = expanded_ct.distance.quantile(0.8)

The above block of code is outside an try block, therefore, whenever we come across a scenario where expanded_ct doesn't have an attribute of distance. Likely in case, where Loaded all confirmed trips of length 0 .

Details of execution:

root@6f9d9e5e6995:/usr/src/app# source setup/activate.sh
(emission) root@6f9d9e5e6995:/usr/src/app# cd saved-notebooks
(emission) root@6f9d9e5e6995:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/update_mappings.py mapping_dictionaries.ipynb
(emission) root@6f9d9e5e6995:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/generate_plots.py generic_metrics_sensed.ipynb default
/usr/src/app/saved-notebooks/bin/generate_plots.py:30: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  if r.status_code is not 200:
About to download config from https://raw.githubusercontent.com/e-mission/nrel-openpath-deploy-configs/main/configs/usaid-laos-ev.nrel-op.json
Successfully downloaded config with version 1 for USAID-NREL Support for Electric Vehicle Readiness and data collection URL https://USAID-laos-EV-openpath.nrel.gov/api/
Dynamic labels download was successful for nrel-openpath-deploy-configs: usaid-laos-ev
Running at 2024-02-20T16:57:43.469878+00:00 with args Namespace(plot_notebook='generic_metrics_sensed.ipynb', program='default', date=None) for range (<Arrow [2023-05-01T00:00:00+00:00]>, <Arrow [2024-02-01T00:00:00+00:00]>)
Running at 2024-02-20T16:57:43.508571+00:00 with params [Parameter('year', int), Parameter('month', int), Parameter('program', str, value='default'), Parameter('study_type', str, value='study'), Parameter('include_test_users', bool, value=True), Parameter('use_imperial', bool, value=False), Parameter('sensed_algo_prefix', str, value='cleaned')]
Running at 2024-02-20T16:57:52.489014+00:00 with params [Parameter('year', int, value=2023), Parameter('month', int, value=5), Parameter('program', str, value='default'), Parameter('study_type', str, value='study'), Parameter('include_test_users', bool, value=True), Parameter('use_imperial', bool, value=False), Parameter('sensed_algo_prefix', str, value='cleaned')]
Traceback (most recent call last):
  File "/usr/src/app/saved-notebooks/bin/generate_plots.py", line 108, in <module>
    compute_for_date(month_year.month, month_year.year)
  File "/usr/src/app/saved-notebooks/bin/generate_plots.py", line 101, in compute_for_date
    nbclient.execute(new_nb)
  File "/root/miniconda-23.5.2/envs/emission/lib/python3.9/site-packages/nbclient/client.py", line 1305, in execute
    return NotebookClient(nb=nb, resources=resources, km=km, **kwargs).execute()
  File "/root/miniconda-23.5.2/envs/emission/lib/python3.9/site-packages/jupyter_core/utils/__init__.py", line 165, in wrapped
    return loop.run_until_complete(inner)
  File "/root/miniconda-23.5.2/envs/emission/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete
    return future.result()
  File "/root/miniconda-23.5.2/envs/emission/lib/python3.9/site-packages/nbclient/client.py", line 705, in async_execute
    await self.async_execute_cell(
  File "/root/miniconda-23.5.2/envs/emission/lib/python3.9/site-packages/nbclient/client.py", line 1058, in async_execute_cell
    await self._check_raise_for_error(cell, cell_index, exec_reply)
  File "/root/miniconda-23.5.2/envs/emission/lib/python3.9/site-packages/nbclient/client.py", line 914, in _check_raise_for_error
    raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)
nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
------------------
#determine 80th percentile
cutoff = expanded_ct.distance.quantile(0.8)
dist_threshold = expanded_ct[distance_col].quantile(0.8).round(1)
dist_threshold = str(dist_threshold)

plot_title_no_quality="Number of trips under " + dist_threshold + " " + label_units_lower + " for each primary mode"
plot_title_no_quality=plot_title_no_quality + "\n(inferred by OpenPATH from phone sensors)" 
plot_title_no_quality=plot_title_no_quality + "\n["+dist_threshold + " " + label_units_lower+" represents 80th percentile of trip length]"
file_name ='ntrips_under10miles_sensed_mode%s' % file_suffix

try:
    labels_d10 = expanded_ct.loc[(expanded_ct['distance'] <= cutoff)].primary_mode.value_counts(dropna=True).keys().tolist()
    values_d10 = expanded_ct.loc[(expanded_ct['distance'] <= cutoff)].primary_mode.value_counts(dropna=True).tolist()
    d10_quality_text = scaffolding.get_quality_text(expanded_ct, expanded_ct[expanded_ct['distance'] <= cutoff], "< " + dist_threshold + " " + label_units_lower, include_test_users)
    plot_title= plot_title_no_quality+"\n"+d10_quality_text
    pie_chart_sensed_mode(plot_title,labels_d10,values_d10,file_name)
    alt_text = store_alt_text_pie(pd.DataFrame(values_d10, labels_d10), file_name, plot_title)
    print(expanded_ct.loc[(expanded_ct['distance'] <= cutoff)].primary_mode.value_counts(dropna=True))
except:
    d10_df = expanded_ct.query("distance <= " + cutoff) if "distance" in expanded_ct.columns else expanded_ct
    debug_df.loc["Trips_less_than_80th_pct"] = scaffolding.trip_label_count("Mode_confirm", d10_df)
    generate_missing_plot(plot_title_no_quality,debug_df,file_name)
    alt_text = store_alt_text_missing(debug_df, file_name, plot_title_no_quality)
------------------

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[5], line 2
      1 #determine 80th percentile
----> 2 cutoff = expanded_ct.distance.quantile(0.8)
      3 dist_threshold = expanded_ct[distance_col].quantile(0.8).round(1)
      4 dist_threshold = str(dist_threshold)

File ~/miniconda-23.5.2/envs/emission/lib/python3.9/site-packages/pandas/core/generic.py:5902, in NDFrame.__getattr__(self, name)
   5895 if (
   5896     name not in self._internal_names_set
   5897     and name not in self._metadata
   5898     and name not in self._accessors
   5899     and self._info_axis._can_hold_identifiers_and_holds_name(name)
   5900 ):
   5901     return self[name]
-> 5902 return object.__getattribute__(self, name)

AttributeError: 'DataFrame' object has no attribute 'distance'

(emission) root@6f9d9e5e6995:/usr/src/app/saved-notebooks#

This has been handled in https://github.com/e-mission/em-public-dashboard/pull/121

Solution: Shifted the computation in try block, that way we would generate a table stating chart could not be generated but would not lead to error in notebook's charts generation.

e-mission / em-public-dashboard

'Dataframe' object has no attribute 'distance' #122