Use estimate_orientation without Mosek solver?

[bt-]

I ran run_pipeline with solver_convex='OSQP' and solver=QSS.

When I tried to use estimate_orientation on the resulting datahandler, I get a Mosek license error:

Error: rescode.err_missing_license_file(1008): License cannot be located. The default search path is ':/home/ben/mosek/mosek.lic:'.

Is it possible to use this method without Mosek?

Thanks!

[pluflou]

Hi @bt- , that's strange, I am able to run this without MOSEK. Could you provide an example to reproduce what you're seeing?

[bt-]

@pluflou, here is what I'm seeing when I start with the example from the General Usage page. This gives the same error that I get when I run it on my actual data.

from solardatatools import DataHandler
from solardatatools.dataio import get_pvdaq_data

pv_system_data = get_pvdaq_data(sysid=35, api_key='DEMO_KEY', year=[2011, 2012, 2013])

dh = DataHandler(pv_system_data)
dh.run_pipeline(power_col='dc_power')

dh.setup_location_and_orientation_estimation(-5) # not sure what the correct GMT offset is for sysid=35
dh.estimate_orientation(latitude=41.356985, longitude=-71.479387) # not sure what correct lat / long is for sysid=35

Gives the following stack trace. It looks like the issue may be in line 25 of ~/miniconda3/envs/pvi-user/lib/python3.10/site-packages/pvsystemprofiler/algorithms/performance_model_estimation.py.

--------------------------------------------------------------------------- Error Traceback (most recent call last) [/tmp/ipykernel_308263/1122878322.py](http://localhost:8888/tmp/ipykernel_308263/1122878322.py) in ?() ----> 1 dh.estimate_orientation(latitude=41.356985, longitude=-71.479387) [~/miniconda3/envs/pvi-user/lib/python3.10/site-packages/solardatatools/data_handler.py](http://localhost:8888/lab/tree/Signal/amherst/~/miniconda3/envs/pvi-user/lib/python3.10/site-packages/solardatatools/data_handler.py) in ?(self, latitude, longitude, tilt, azimuth, day_interval, x1, x2) 1299 x2=0.9, 1300 ): 1301 ready = self.__help_param_est() 1302 if ready: -> 1303 self.parameter_estimation.estimate_orientation( 1304 longitude=longitude, 1305 latitude=latitude, 1306 tilt=tilt, [~/miniconda3/envs/pvi-user/lib/python3.10/site-packages/pvsystemprofiler/estimator.py](http://localhost:8888/lab/tree/Signal/amherst/~/miniconda3/envs/pvi-user/lib/python3.10/site-packages/pvsystemprofiler/estimator.py) in ?(self, longitude, latitude, tilt, azimuth, day_interval, x1, x2) 151 self.num_days = dh.num_days 152 self.omega = calculate_omega(self.data_sampling, self.num_days, self.longitude, self.day_of_year, 153 self.gmt_offset) 154 --> 155 self.tilt, self.azimuth = self._cal_orientation_helper() [~/miniconda3/envs/pvi-user/lib/python3.10/site-packages/pvsystemprofiler/estimator.py](http://localhost:8888/lab/tree/Signal/amherst/~/miniconda3/envs/pvi-user/lib/python3.10/site-packages/pvsystemprofiler/estimator.py) in ?(self) 188 day_range = (self.day_of_year > self.day_interval[0]) & (self.day_of_year < self.day_interval[1]) 189 else: 190 day_range = np.ones(self.day_of_year.shape, dtype=bool) 191 --> 192 scale_factor_costheta, costheta_fit = find_fit_costheta(self.data_matrix, self.days) 193 194 boolean_filter = filter_data(self.data_matrix, self.daytime_threshold, self.x1, self.x2) 195 [~/miniconda3/envs/pvi-user/lib/python3.10/site-packages/pvsystemprofiler/algorithms/performance_model_estimation.py](http://localhost:8888/lab/tree/Signal/amherst/~/miniconda3/envs/pvi-user/lib/python3.10/site-packages/pvsystemprofiler/algorithms/performance_model_estimation.py) in ?(data_matrix, clear_index) 21 data == s1 + s2, 22 s1[365:] == s1[:-365] 23 ] 24 problem = cvx.Problem(objective, constraints) ---> 25 problem.solve(solver='MOSEK') 26 scale_factor_costheta = s1.value 27 costheta_fit = data_matrix [/](http://localhost:8888/) np.max(s1.value) 28 return scale_factor_costheta, costheta_fit [~/miniconda3/envs/pvi-user/lib/python3.10/site-packages/cvxpy/problems/problem.py](http://localhost:8888/lab/tree/Signal/amherst/~/miniconda3/envs/pvi-user/lib/python3.10/site-packages/cvxpy/problems/problem.py) in ?(self, *args, **kwargs) 499 if func_name is not None: 500 solve_func = Problem.REGISTERED_SOLVE_METHODS[func_name] 501 else: 502 solve_func = Problem._solve --> 503 return solve_func(self, *args, **kwargs) [~/miniconda3/envs/pvi-user/lib/python3.10/site-packages/cvxpy/problems/problem.py](http://localhost:8888/lab/tree/Signal/amherst/~/miniconda3/envs/pvi-user/lib/python3.10/site-packages/cvxpy/problems/problem.py) in ?(self, solver, warm_start, verbose, gp, qcp, requires_grad, enforce_dpp, ignore_dpp, canon_backend, **kwargs) 1078 s.LOGGER.info( 1079 'Invoking solver %s to obtain a solution.', 1080 solving_chain.reductions[-1].name()) 1081 start = time.time() -> 1082 solution = solving_chain.solve_via_data( 1083 self, data, warm_start, verbose, kwargs) 1084 end = time.time() 1085 self._solve_time = end - start [~/miniconda3/envs/pvi-user/lib/python3.10/site-packages/cvxpy/reductions/solvers/solving_chain.py](http://localhost:8888/lab/tree/Signal/amherst/~/miniconda3/envs/pvi-user/lib/python3.10/site-packages/cvxpy/reductions/solvers/solving_chain.py) in ?(self, problem, data, warm_start, verbose, solver_opts) 449 raw solver solution 450 The information returned by the solver; this is not necessarily 451 a Solution object. 452 """ --> 453 return self.solver.solve_via_data(data, warm_start, verbose, 454 solver_opts, problem._solver_cache) [~/miniconda3/envs/pvi-user/lib/python3.10/site-packages/cvxpy/reductions/solvers/conic_solvers/mosek_conif.py](http://localhost:8888/lab/tree/Signal/amherst/~/miniconda3/envs/pvi-user/lib/python3.10/site-packages/cvxpy/reductions/solvers/conic_solvers/mosek_conif.py) in ?(self, data, warm_start, verbose, solver_opts, solver_cache) 293 if save_file: 294 task.writedata(save_file) 295 296 # Optimize the Mosek Task, and return the result. --> 297 rescode = task.optimize() 298 299 if rescode == mosek.rescode.trm_max_time: 300 warnings.warn( [~/miniconda3/envs/pvi-user/lib/python3.10/site-packages/mosek/__init__.py](http://localhost:8888/lab/tree/Signal/amherst/~/miniconda3/envs/pvi-user/lib/python3.10/site-packages/mosek/__init__.py) in ?(self, *args, **kwds) 10777 10778 optimize() -> (trmcode) 10779 [trmcode : mosek.rescode] Is either OK or a termination response code. 10780 """ > 10781 return self.__optimizetrm__1(*args,**kwds) [~/miniconda3/envs/pvi-user/lib/python3.10/site-packages/mosek/__init__.py](http://localhost:8888/lab/tree/Signal/amherst/~/miniconda3/envs/pvi-user/lib/python3.10/site-packages/mosek/__init__.py) in ?(self) 10766 def __optimizetrm__1(self): 10767 _res_optimizetrm,_retargs_optimizetrm = self.__obj.optimizetrm__1() 10768 if _res_optimizetrm != 0: 10769 _,_msg_optimizetrm = self.__getlasterror(_res_optimizetrm) > 10770 raise Error(rescode(_res_optimizetrm),_msg_optimizetrm) 10771 else: 10772 (trmcode) = _retargs_optimizetrm 10773 return (rescode(trmcode)) Error: rescode.err_missing_license_file(1008): License cannot be located. The default search path is ':[/home/ben/mosek/mosek.lic](http://localhost:8888/home/ben/mosek/mosek.lic):'.

[bmeyers]

Hi @bt-, thanks for bringing this to our attention. We have a traced this to a conda ~error~ (we've been informed by conda maintainers that this is expected behavior), in which an extremely old version of SDT is being selected (v0.5.0 vs. v1.3.1).

Can you confirm that conda installed v0.5.0 on your system?

We are trying to fix this in our conda distribution workflow, but we don't have a fix ready to go. For now, we recommend either installation with pip (pip install solar-data-tools). Alternatively, if you wish to install from conda, you will need to specify more channels to get the correct install behavior:

conda install -c stanfordcvxgrp -c slacgismo -c conda-forge solar-data-tools

Thank you again for documenting this for us!

[bt-]

@bmeyers, thanks for looking into this. It looks to me like I do have v1.3.0 installed.

I am using a env yml file that I modified starting from the one in the repo. So, it has those channels included still:

[bmeyers]

@bt- hmm that doesn't quite track. Your traceback shows code that hasn't existed in the codebase since v0.7.1. In v1.3.0+, it's a completely different function. Referring to this part of the traceback you shared:

[~/miniconda3/envs/pvi-user/lib/python3.10/site-packages/pvsystemprofiler/algorithms/performance_model_estimation.py](http://localhost:8888/lab/tree/Signal/amherst/~/miniconda3/envs/pvi-user/lib/python3.10/site-packages/pvsystemprofiler/algorithms/performance_model_estimation.py) in ?(data_matrix, clear_index)
21 data == s1 + s2,
22 s1[365:] == s1[:-365]
23 ]
24 problem = cvx.Problem(objective, constraints)
---> 25 problem.solve(solver='MOSEK')
26 scale_factor_costheta = s1.value
27 costheta_fit = data_matrix / np.max(s1.value)
28 return scale_factor_costheta, costheta_fit

If you could, please try to make the environment fresh, either from a minimal conda install or a minimal pip install. If you'd like to use the yaml file, please try removing all dependencies except for solar-data-tools.

[bt-]

@bmeyers,hmm, agreed that doesn't make sense. I was short on time yesterday, so I'll spend a few more minutes investigating today.

I tried to import the solar-data-tools module and check the version there but that said there was no __version__.

[bmeyers]

Yes, sorry. I don't believe we are currently setting that attribute. Still this seems like an environment/installation issue. Sorry to pull an "IT Crowd" on you, but could you try a fresh install in a new virtual environment? Also please note that we no longer have a conda enviroment yaml file hosted in the repo (and don't really recommend that method except as needed for advanced environment management).

[pluflou]

Hi @bt- , were you able to identify whether this was an environment issue or do you still need help?

[bt-]

Hi @pluflou, I worked around this by using a trial license of Mosek. I haven't had a chance to get back to this and look into the environment issue in more detail. I'm still open to trying a new install following the latest guidance (new since I opened this?), but I'm not sure when I'll be able to try it.

[bmeyers]

As a final note on this issue, the request to not require MOSEK has already been included in a prior release. We're happy to assist if you're still having environment issues, @bt-.