Open jomoengineer opened 4 months ago
This is what I see with Pandas 2.x:
Retrieving weather data for canazel
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
Cell In[5], line 6
3 location_list = ['canazel']
5 # retrieve_hist_data returns a list of dataframe(s)
----> 6 hist_df = retrieve_hist_data(api_key,
7 location_list,
8 '01-JAN-2011',
9 '31-DEC-2020',
10 frequency,
11 location_label = False,
12 export_csv = False,
13 store_df = True)
15 # Extract temperature, humidity and precipitation
16 t_list = hist_df[0].tempC.astype(float).to_list()
File ~/develop/machine_leaning/ml_testing/lib/python3.11/site-packages/wwo_hist/__init__.py:120, in retrieve_hist_data(api_key, location_list, start_date, end_date, frequency, location_label, export_csv, store_df, response_cache_path)
118 for location in location_list:
119 print('\n\nRetrieving weather data for ' + location + '\n\n')
--> 120 df_this_city = retrieve_this_location(api_key, location, start_date, end_date, frequency, response_cache_path)
122 if (location_label == True):
123 # add city name as prefix to the colnames
124 df_this_city = df_this_city.add_prefix(location + '_')
File ~/develop/machine_leaning/ml_testing/lib/python3.11/site-packages/wwo_hist/__init__.py:75, in retrieve_this_location(api_key, location, start_date, end_date, frequency, response_cache_path)
72 list_mon_begin = pd.concat([pd.Series(pd.to_datetime(start_date)), pd.Series(list_mon_begin)], ignore_index=True)
74 # create list of month end dates for range between start and end dates non-inclusive (open)
---> 75 list_mon_end = pd.date_range(start_date, end_date, freq='M', closed='left')
76 # convert to Series and add end_date at end
77 list_mon_end = pd.concat([pd.Series(list_mon_end), pd.Series(pd.to_datetime(end_date))], ignore_index=True)
File ~/develop/machine_leaning/ml_testing/lib/python3.11/site-packages/pandas/core/indexes/datetimes.py:1008, in date_range(start, end, periods, freq, tz, normalize, name, inclusive, unit, **kwargs)
1005 if freq is None and com.any_none(periods, start, end):
1006 freq = "D"
-> 1008 dtarr = DatetimeArray._generate_range(
1009 start=start,
1010 end=end,
1011 periods=periods,
1012 freq=freq,
1013 tz=tz,
1014 normalize=normalize,
1015 inclusive=inclusive,
1016 unit=unit,
1017 **kwargs,
1018 )
1019 return DatetimeIndex._simple_new(dtarr, name=name)
TypeError: DatetimeArray._generate_range() got an unexpected keyword argument 'closed'
I was able to get passed the Pandas 2.x issue by making changes in WorldWeatherOnline.
https://github.com/ekapope/WorldWeatherOnline/pull/22
However, I am still seeing an issue with KeyError 'weather'.
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
Cell In[5], line 6
3 location_list = ['canazel']
5 # retrieve_hist_data returns a list of dataframe(s)
----> 6 hist_df = retrieve_hist_data(api_key,
7 location_list,
8 '01-JAN-2011',
9 '31-DEC-2020',
10 frequency,
11 location_label = False,
12 export_csv = False,
13 store_df = True)
15 # Extract temperature, humidity and precipitation
16 t_list = hist_df[0].tempC.astype(float).to_list()
File ~/develop/machine_leaning/WorldWeatherOnline/wwo_hist/__init__.py:119, in retrieve_hist_data(api_key, location_list, start_date, end_date, frequency, location_label, export_csv, store_df, response_cache_path)
117 for location in location_list:
118 print('\n\nRetrieving weather data for ' + location + '\n\n')
--> 119 df_this_city = retrieve_this_location(api_key, location, start_date, end_date, frequency, response_cache_path)
121 if (location_label == True):
122 # add city name as prefix to the colnames
123 df_this_city = df_this_city.add_prefix(location + '_')
File ~/develop/machine_leaning/WorldWeatherOnline/wwo_hist/__init__.py:101, in retrieve_this_location(api_key, location, start_date, end_date, frequency, response_cache_path)
99 with open(f'{response_cache_path}/{location}_{start_d}_{end_d}', 'w') as f:
100 json.dump(json_data, f)
--> 101 data = json_data['data']['weather']
102 # call function to extract json object
103 df_this_month = extract_monthly_data(data)
KeyError: 'weather'
Looks like the 'weather' error was due to me misspelling the name 'canazei'. Method 1 works now after making the changes to 'wwo_hist' with changing 'close' to 'inclusive' There are still some deprecated warnings though.
However, I am still seeing the 'vectorNumElems' error in 'tensorflow'.
2024-02-14 22:56:05.435587: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:378] Ignored output_format.
2024-02-14 22:56:05.435642: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:381] Ignored drop_control_dependency.
2024-02-14 22:56:05.436237: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: snow_forecast
2024-02-14 22:56:05.437412: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve }
2024-02-14 22:56:05.437455: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: snow_forecast
2024-02-14 22:56:05.439953: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:388] MLIR V1 optimization pass is not enabled
2024-02-14 22:56:05.440956: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle.
2024-02-14 22:56:05.479323: I tensorflow/cc/saved_model/loader.cc:217] Running initialization op on SavedModel bundle at path: snow_forecast
2024-02-14 22:56:05.491198: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 54967 microseconds.
2024-02-14 22:56:05.501956: I tensorflow/compiler/mlir/tensorflow/utils/dump_mlir_util.cc:269] disabling MLIR crash reproducer, set env var `MLIR_CRASH_REPRODUCER_DIRECTORY` to enable.
Summary on the non-converted ops:
---------------------------------
* Accepted dialects: tfl, builtin, func
* Non-Converted Ops: 4, Total Ops 10, % non-converted = 40.00 %
* 4 ARITH ops
- arith.constant: 4 occurrences (f32: 4)
(f32: 2)
(f32: 1)
fully_quantize: 0, inference_type: 6, input_inference_type: INT8, output_inference_type: INT8
WARNING:absl:Buffer deduplication procedure will be skipped when flatbuffer library is not properly loaded
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
/tmp/ipykernel_47203/1373129489.py in ?()
----> 1 tflite_model_quant = converter.convert()
2 open(TFL_MODEL_FILE, "wb").write(tflite_model_quant)
~/develop/machine_leaning/ml_testing/lib/python3.11/site-packages/tensorflow/lite/python/lite.py in ?(self, *args, **kwargs)
1136 @functools.wraps(convert_func)
1137 def wrapper(self, *args, **kwargs):
1138 # pylint: disable=protected-access
-> 1139 return self._convert_and_export_metrics(convert_func, *args, **kwargs)
1140 # pylint: enable=protected-access
~/develop/machine_leaning/ml_testing/lib/python3.11/site-packages/tensorflow/lite/python/lite.py in ?(self, convert_func, *args, **kwargs)
1107 # TODO(b/202090541): Collects sparsity block size information.
1108 sparsity_modes = _get_sparsity_modes(model_object)
1109 self._metadata.options.modelOptimizationModes.extend(sparsity_modes)
1110 model_object = _populate_conversion_metadata(model_object, self._metadata)
-> 1111 return flatbuffer_utils.convert_object_to_bytearray(model_object)
~/develop/machine_leaning/ml_testing/lib/python3.11/site-packages/tensorflow/lite/tools/flatbuffer_utils.py in ?(model_object, extra_buffer)
89 def convert_object_to_bytearray(model_object, extra_buffer=b''):
90 """Converts a tflite model from an object to a immutable bytearray."""
91 # Initial size of the buffer, which will grow automatically if needed
92 builder = flatbuffers.Builder(1024)
---> 93 model_offset = model_object.Pack(builder)
94 builder.Finish(model_offset, file_identifier=_TFLITE_FILE_IDENTIFIER)
95 model_bytearray = bytes(builder.Output())
96 model_bytearray = model_bytearray + extra_buffer
~/develop/machine_leaning/ml_testing/lib/python3.11/site-packages/tensorflow/lite/python/schema_py_generated.py in ?(self, builder)
18097 operatorCodeslist.append(self.operatorCodes[i].Pack(builder))
18098 ModelStartOperatorCodesVector(builder, len(self.operatorCodes))
18099 for i in reversed(range(len(self.operatorCodes))):
18100 builder.PrependUOffsetTRelative(operatorCodeslist[i])
> 18101 operatorCodes = builder.EndVector()
18102 if self.subgraphs is not None:
18103 subgraphslist = []
18104 for i in range(len(self.subgraphs)):
TypeError: Builder.EndVector() missing 1 required positional argument: 'vectorNumElems'
I was able to get passed the TypeError for 'vectorNumElems' buy downgrading flatbuffers to ver 23.5.26.
Ex:
pip3 uninstall flatbuffers
pip3 install flatbuffers=23.5.25
There appears to be a conflict with Tensorflow 2.15.0.
I was able to run the complete notebook on a Raspberry 5 running Bookworm.
I am seeing errors when running the Method 1 prepare_model.ipynb. With Pandas 1.5.x I see this:
Then at :"Convert the model to TensorFlow Lite file format (FlatBuffers) as save it as .tflite", I get this.