Closed rok-cesnovar closed 3 years ago
This is a checklist attempt for the next release so we get a bit more organized. The first draft was done by me and @bbbales2, feel free to comment/edit/add/remove.
Tagging @stan-dev/math @stan-dev/maintainers
Looks good and makes it more clear how much work making a release needs!
The list of Math pulls that had gone in as of Dec 7th are here. I'm not sure the value of maintaining this list but I'll put this one here since @rok-cesnovar and I discussed it:
custom_vari
class.(#2144)symmetrize(matrix m)
function to stan::math
.(#2209)reduce_sum
parallelism.(#2162)ordered_probit_lpmf
for increased numerical stability(#2229)var_value<Eigen::Matrix<T, R, C>>
specializations for columns_dot_self, determinant, dot_self, inverse, log_determinant, matrix_power, multiply_lower_tri_self_transpose, and tcrossprod.(#2106)var<mat>
(#2163)var<mat>
versions of quad_form
and trace_quad_form
(#2164)variance
modified to work with var<mat>
(#2168)sd
, log_softmax
, and log_sum_exp
to work fully with var<mat>
(#2169)var<mat>
support to divide
(#2171)var<matrix>
(#2172)operator-
for var<matrix>
types(#2196)var<mat>
versions of columns_dot_product
and rows_dot_product
(#2199)from_matrix_cl
to correctly works with triangular matrices.(#2117)bernoulli_lpmf
using OpenCL, enabling it to use GPUs. Fixed a bug that caused a generated kernel not to compile if it used both check_cl
and any colwise reduction.(#2155)bernoulli_logit_lpmf
using OpenCL, enabling it to use GPUs.(#2173)beta_lpdf
using OpenCL, enabling it to use GPUs.(#2174)lgamma_stirling
, lgamma_stirling_diff
, lbeta
and binomial_coefficient_log
.(#2176)beta_proportion_lpdf
function.(#2180)sign
, cauchy_lpdf
, chi_square_lpdf
and double_exponential_lpdf
.(#2181)exp_mod_normal_lpdf
, exponential_lpdf
and frechet_lpdf
and support for pow
function in kernel generator.(#2184)gamma_lpdf
and gumbel_lpdf
.(#2185)check_cl
, which in case of throwing an exception, caused it to report wrong indices of error.(#2185)inv_chi_square_lpdf
, inv_gamma_lpdf
and logistic_lpdf
.(#2191)lbeta
and binomial_coefficient_log
.(#2201)c = a + b
and ˙c = a + a
).(#2215)lognormal_lpdf
, normal_lpdf
and pareto_lpdf
.(#2216)pareto_type_2_lpdf
, poisson_log_plmf
and poisson_log_lpmf
.(#2217)multiply_log
to kernel generator.(#2217)rayleigh_lpdf
, scaled_inv_chi_square
and skew_normal_lpdf
.(#2221)std_normal_lpdf
, student_t_lpdf
and uniform_lpdf
.(#2222)weibull_lpdf
, neg_binomial_2_log_lpmf
and neg_binomial_2_lpmf
.(#2231)binomial_lpmf
and neg_binomail_lpmf
.(#2236)dot_product
handles expressions.(#2150)multiply_lower_tri_self_transpose
, block
, map_rect
, log_determinant
, matrix_exp_multiply
, sd
).(#2151)"Generalized student t distribution related functions to accept Eigen expressions" etc is confusing as it looks like we would now have new distributions: generalized student t, generalized uniform, generalized Weibull, etc which all are existing distributions. Only the line "Generalizes Wishart distribution" breaks this pattern, and I assume the distributions were not generalized but just the functions.
Oh yeah lol. This is correct. "Generalized" means the function supports Eigen expressions, not a new distribution. Let me make a note at the top to edit this on the final versions.
This is the unedited version which is just a grouped dump of the release notes written in PRs. In the final version all the “generalize” notes will be something like:
Generalized all remaining C++ functions to accept Eigen expressions.
EDIT: Ben beat me too it :)
in the last release, were there hiccups post-release w/r/t CmdStan makefile?
Rok recently reminded me of https://github.com/stan-dev/cmdstan/issues/929, which get rid of use of a Boost non-header only library - will try to get this in ASAP
The first attempt for the RC tarball was missing CMDSTAN_VERSION somehow, which is why I noted that prominently above. Other than that there were no new issues.
Also are expressions just done? The compiler knows how to use them. There are no language changes to account for them. They are used in Math. They are tested, and I remember we added some doc on how to use those tests somewhere.
If anything doesn't exist, it's higher level C++ docs (like are also missing for varmat).
Also are expressions just done?
In terms of code I think they are done, at least what was meant for 2.26. At some point, maybe for 2.27, we may attempt also returning and handling expressions in Stan user-defined functions, but that is too big of a project for 2.26.
Sounds good!
@bbbales2 asked me to look different lists, and I'm very happy to see this level of planning and clear checklists. I added couple minor comments in other issues, and here I ask clarification
Is it possible to add new open projects before January 13th as long as there is corresponding checklist for "what needs to be done and how it will be verified that it is done" that is considered to be feasible? Or do you know that there are not such projects that someone might want to include?
If someone has a project that can be done by Jan 13th then I'm fine with releasing it this version
We're calling off varmat for the 2.26 release. There are too many pull requests stacked up at the last minute. Specifically, calling off varmat means we're not going to turn on any varmat stuff in the compiler in 2.26 (there's plenty of varmat stuff in Math).
Since varmat was the reason we were doing the early freeze (today), Rok & I would like to move that freeze to the same day as everything else (the 18th). This does not change the regularly scheduled freeze & release timings (18th and 25th, respectively). We just don't want to do the early Math freeze without a reason. The early Math freeze was to give time for varmat compiler development. Since that is off, no reason to keep our Math repository frozen longer than it needs to be.
If anyone has objections to canceling the early Math freeze, let us know by the end of the day today (13th), otherwise we'll just keep reviewing and merging stuff as planned.
Looks like no comments so we'll delay the Math freeze until the 18th (with everything else)
Getting varmat off the ground is definitely worth a week extra time (also two weeks)!
We're just gonna delay it to 2.27. Doesn't feel great, but it would be rushed at this point. And no need to clog up the release as is (which still has lots of good stuff in it).
@serban-nicusor-toptal https://jenkins.mc-stan.org/blue/organizations/jenkins/Math%20Pipeline/detail/develop/1021/pipeline should be the final develop test before the freeze. Once that finishes feel free to make a Math RC (12-13h from now) and once that propagates upstream you can do RCs for Stan/CmdStan. Current Stanc3 nightly is what will be the RC stanc3. Thanks!
Hey @rok-cesnovar RCs are out now!
Thanks Nic! I think the tarball is missing binaries for stanc3?
I started the cmdstanr tests with the tarball for a first sanity check of the tarballs https://github.com/stan-dev/cmdstanr/actions/runs/495905352
It should contain the stanc3 binaries from nightly release, all together inside /bin
I am not seeing it here: https://github.com/stan-dev/cmdstan/releases/download/v2.26.0-rc1/cmdstan-2.26.0-rc1.tar.gz
The tarball is also smaller than for 2.25 which made me check that.
@rok-cesnovar you are right, turns out git-archive-all
does not include /bin
in the final archive. I've re-created it manually and uploaded it again. Should be fine now.
Yeah, I think because /bin is in gitignore. Thanks!
Hey @rok-cesnovar is everything ready for the release ? Asking just to be sure before starting the proces.
Hey @serban-nicusor-toptal, I think everything is ready, but given that we released the RC on Tuesday I think we should do the actual release tomorrow. Does that work for you?
Sure that's fine, we'll leave it for tomorrow! Thanks for the quick reply, have a great week!
@serban-nicusor-toptal here are all the release notes for the 4 repositories: https://gist.github.com/rok-cesnovar/28750bf5b79026bb2ae4dc37190c744b
Let me know if you need anything from me. We can now proceed with the release whenever you have time.
Ther was a fix for the gradients of the multi variate normal cholesky lpdf...not sure I saw that one.
Oh right, good call. Forgot to include the post-freeze stuff (ran the script the day after the freeze). Thanks! Updated now.
@serban-nicusor-toptal also listing the version numbers:
Math version should be updated to 4.0 Stan is 2.26 (we need to update the version numbers, we forgot the last time, see https://github.com/stan-dev/stan/pull/2990/files) Cmdstan and stanc3 also 2.26
Thanks!
If there are no objections I will start the release in 1 hour from now.
Hey @rok-cesnovar release is now done across all projects!
Thanks Nic, will check the tarball and make the announcement. If all is good, will close this issue.
Checklist completed, everything released. Closing.
Thanks @serban-nicusor-toptal !
With the last release, there were a couple straggling issues that took a long time after the formal release to resolve. In particular the release notes and the docs for
lupdf
andlupmf
came a couple/few weeks after the formal release. Since this release will have a lot of new features, let's plan more explicitly what needs done when so things are not forgotten.So we've started a checklist here, and part of this checklist will be more checklists. But the idea is that the checkbox is due on the date it is categorized under.
~6 weeks before the next release (halfway between releases -- this is 14th December):
varmatpostponed to 2.27Feature freeze (January 18th -- this is the regular 1 week freeze):
Release candidate stage:
Release (January 25th):
Ideas for next release: