search

slacgismo / solar-data-tools

Some data analysis tools for working with historical PV solar time-series data sets.
https://solar-data-tools.readthedocs.io
BSD 2-Clause "Simplified" License
57 stars 22 forks source link

Handling missing Mosek license file #61

Closed alejandro350 closed 1 year ago

alejandro350 commented 3 years ago

When a user does not have the Mosek license, running the pipeline fails with an error that is difficult to trace. Need to handle the error in a way that tells the user that the Mosek license file is missing.

Matrix filling failed.
Daily quality scoring failed.
Daily quality flagging failed.
Clear day detection failed.
clipping check failed: 'NoneType' object has no attribute 'no_errors'
Data set summary scoring failed.

Traceback (most recent call last):
  File "/Users/mac/Documents/github/solar-data-tools/solardatatools/data_handler.py", line 193, in run_pipeline
    ss.run_optimizer(self.raw_data_matrix, plot=False, solver=solver)
  File "/Users/mac/Documents/github/solar-data-tools/solardatatools/algorithms/sunrise_sunset_estimation.py", line 177, in run_optimizer
    sunrises, train_msk_sr, tau=0.05, solver=solver
  File "/Users/mac/Documents/github/solar-data-tools/solardatatools/signal_decompositions.py", line 164, in tl1_l2d2p365
    prob.solve(solver=solver, verbose=verbose)
  File "/opt/miniconda3/envs/pvi-dev/lib/python3.7/site-packages/cvxpy/problems/problem.py", line 462, in solve
    return solve_func(self, *args, **kwargs)
  File "/opt/miniconda3/envs/pvi-dev/lib/python3.7/site-packages/cvxpy/problems/problem.py", line 959, in _solve
    self, data, warm_start, verbose, kwargs)
  File "/opt/miniconda3/envs/pvi-dev/lib/python3.7/site-packages/cvxpy/reductions/solvers/solving_chain.py", line 344, in solve_via_data
    solver_opts, problem._solver_cache)
  File "/opt/miniconda3/envs/pvi-dev/lib/python3.7/site-packages/cvxpy/reductions/solvers/conic_solvers/mosek_conif.py", line 235, in solve_via_data
    task.optimize()
  File "/opt/miniconda3/envs/pvi-dev/lib/python3.7/site-packages/mosek/__init__.py", line 7627, in optimize
    raise Error(rescode(res),msg)
mosek.Error: rescode.err_missing_license_file(1008): License cannot be located. The default search path is ':/Users/mac/mosek/mosek.lic:'.
Traceback (most recent call last):
  File "/Users/mac/Documents/github/solar-data-tools/solardatatools/data_handler.py", line 210, in run_pipeline
    interp_day=interp_day)
  File "/Users/mac/Documents/github/solar-data-tools/solardatatools/data_handler.py", line 607, in make_filled_data_matrix
    night_mask=~self.boolean_masks.daytime)
TypeError: bad operand type for unary ~: 'NoneType'
Traceback (most recent call last):
  File "/Users/mac/Documents/github/solar-data-tools/solardatatools/data_handler.py", line 261, in run_pipeline
    self.get_daily_scores(threshold=0.2, solver=solver)
  File "/Users/mac/Documents/github/solar-data-tools/solardatatools/data_handler.py", line 640, in get_daily_scores
    self.get_density_scores(threshold=threshold, solver=solver) # CVXPY
  File "/Users/mac/Documents/github/solar-data-tools/solardatatools/data_handler.py", line 680, in get_density_scores
    return_density_signal=True, return_fit=True, solver=solver
  File "/Users/mac/Documents/github/solar-data-tools/solardatatools/data_quality.py", line 54, in daily_missing_data_advanced
    solver=solver
  File "/Users/mac/Documents/github/solar-data-tools/solardatatools/signal_decompositions.py", line 164, in tl1_l2d2p365
    prob.solve(solver=solver, verbose=verbose)
  File "/opt/miniconda3/envs/pvi-dev/lib/python3.7/site-packages/cvxpy/problems/problem.py", line 462, in solve
    return solve_func(self, *args, **kwargs)
  File "/opt/miniconda3/envs/pvi-dev/lib/python3.7/site-packages/cvxpy/problems/problem.py", line 959, in _solve
    self, data, warm_start, verbose, kwargs)
  File "/opt/miniconda3/envs/pvi-dev/lib/python3.7/site-packages/cvxpy/reductions/solvers/solving_chain.py", line 344, in solve_via_data
    solver_opts, problem._solver_cache)
  File "/opt/miniconda3/envs/pvi-dev/lib/python3.7/site-packages/cvxpy/reductions/solvers/conic_solvers/mosek_conif.py", line 235, in solve_via_data
    task.optimize()
  File "/opt/miniconda3/envs/pvi-dev/lib/python3.7/site-packages/mosek/__init__.py", line 7627, in optimize
    raise Error(rescode(res),msg)
mosek.Error: rescode.err_missing_license_file(1008): License cannot be located. The default search path is ':/Users/mac/mosek/mosek.lic:'.
Traceback (most recent call last):
  File "/Users/mac/Documents/github/solar-data-tools/solardatatools/data_handler.py", line 272, in run_pipeline
    linearity_threshold=linearity_threshold)
  File "/Users/mac/Documents/github/solar-data-tools/solardatatools/data_handler.py", line 647, in get_daily_flags
    self.daily_scores.density > density_lower_threshold,
AttributeError: 'NoneType' object has no attribute 'density'
Traceback (most recent call last):
  File "/Users/mac/Documents/github/solar-data-tools/solardatatools/data_handler.py", line 286, in run_pipeline
    solver=solver
  File "/Users/mac/Documents/github/solar-data-tools/solardatatools/data_handler.py", line 823, in detect_clear_days
    solver=solver)
  File "/Users/mac/Documents/github/solar-data-tools/solardatatools/clear_day_detection.py", line 52, in find_clear_days
    tc, tau=0.9, c1=1e3, yearly_periodic=False, solver=solver
  File "/Users/mac/Documents/github/solar-data-tools/solardatatools/signal_decompositions.py", line 164, in tl1_l2d2p365
    prob.solve(solver=solver, verbose=verbose)
  File "/opt/miniconda3/envs/pvi-dev/lib/python3.7/site-packages/cvxpy/problems/problem.py", line 462, in solve
    return solve_func(self, *args, **kwargs)
  File "/opt/miniconda3/envs/pvi-dev/lib/python3.7/site-packages/cvxpy/problems/problem.py", line 959, in _solve
    self, data, warm_start, verbose, kwargs)
  File "/opt/miniconda3/envs/pvi-dev/lib/python3.7/site-packages/cvxpy/reductions/solvers/solving_chain.py", line 344, in solve_via_data
    solver_opts, problem._solver_cache)
  File "/opt/miniconda3/envs/pvi-dev/lib/python3.7/site-packages/cvxpy/reductions/solvers/conic_solvers/mosek_conif.py", line 221, in solve_via_data
    task = MOSEK._build_dualized_task(task, data)
  File "/opt/miniconda3/envs/pvi-dev/lib/python3.7/site-packages/cvxpy/reductions/solvers/conic_solvers/mosek_conif.py", line 272, in _build_dualized_task
    task.putclist(np.arange(c.size, dtype=int), c)
  File "/opt/miniconda3/envs/pvi-dev/lib/python3.7/site-packages/mosek/__init__.py", line 10289, in putclist
    raise Error(rescode(res),msg)
mosek.Error: rescode.err_nan_in_c(1470): The objective vector c contains an invalid value for variable '' (0).
Traceback (most recent call last):
  File "/Users/mac/Documents/github/solar-data-tools/solardatatools/data_handler.py", line 297, in run_pipeline
    self.clipping_check(solver=solver)
  File "/Users/mac/Documents/github/solar-data-tools/solardatatools/data_handler.py", line 734, in clipping_check
    self.filled_data_matrix, no_error_flag=self.daily_flags.no_errors,
AttributeError: 'NoneType' object has no attribute 'no_errors'
Traceback (most recent call last):
  File "/Users/mac/Documents/github/solar-data-tools/solardatatools/data_handler.py", line 307, in run_pipeline
    self.score_data_set()
  File "/Users/mac/Documents/github/solar-data-tools/solardatatools/data_handler.py", line 721, in score_data_set
    self.data_quality_score = np.sum(self.daily_flags.no_errors) / num_days
AttributeError: 'NoneType' object has no attribute 'no_errors'

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
/var/folders/fx/3njyl4t16fl1qjwfnv_71str0000gn/T/ipykernel_31300/756290427.py in <module>
----> 1 dh.run_pipeline(power_col=pow_cols[0], fix_shifts=fix)

~/Documents/github/solar-data-tools/solardatatools/data_handler.py in run_pipeline(self, power_col, min_val, max_val, zero_night, interp_day, fix_shifts, density_lower_threshold, density_upper_threshold, linearity_threshold, clear_day_smoothness_param, clear_day_energy_param, verbose, start_day_ix, end_day_ix, c1, c2, solar_noon_estimator, correct_tz, extra_cols, daytime_threshold, units, solver)
    317         try:
    318             # CVXPY
--> 319             self.capacity_clustering(solver=solver)
    320         except TypeError:
    321             msg = 'Capacity clustering failed.'

~/Documents/github/solar-data-tools/solardatatools/data_handler.py in capacity_clustering(self, solver, plot, figsize, show_clusters)
    752             self.capacity_analysis = CapacityChange()
    753             self.capacity_analysis.run(
--> 754                 self.filled_data_matrix, filter=self.daily_flags.no_errors,
    755                 quantile=1.00, c1=15, c2=100, c3=300, reweight_eps=0.5,
    756                 reweight_niter=5, dbscan_eps=.02, dbscan_min_samples='auto',

AttributeError: 'NoneType' object has no attribute 'no_errors'
bmeyers commented 3 years ago

I gave this a bit of thought, and the simplest solution I can think of is to explicitly catch the mosek errors: mosek.Error: rescode.err_missing_license_file(1008). There are already try/except clauses around each of the subroutines, which is why the top of the error output has all the "such and such failed" lines.

I think we just want to stop the pipeline and warn the user, the moment a mosek license error is encountered.

bmeyers commented 1 year ago

In current release, there is an improved error message for this