search

libprima / prima

PRIMA is a package for solving general nonlinear optimization problems without using derivatives. It provides the reference implementation for Powell's derivative-free optimization methods, i.e., COBYLA, UOBYQA, NEWUOA, BOBYQA, and LINCOA. PRIMA means Reference Implementation for Powell's methods with Modernization and Amelioration, P for Powell.
http://libprima.net
BSD 3-Clause "New" or "Revised" License
307 stars 42 forks source link

C interface #28

Closed jschueller closed 1 year ago

jschueller commented 1 year ago

This adds a C interface to the Fortran library thanks to the iso_c_binding module:

prima.h:

typedef void (*prima_function)(const double *x, double *f);
typedef void (*prima_function_con)(const double *x, double *f, double *con);

int prima_cobyla(prima_function_con calcfc, int n, int m, double *x, double rhobeg, double rhoend, int *maxfun, int iprint, void *state);
int prima_bobyqa(prima_function calfun, int n, double *x, double *xl, double *xu, double rhobeg, double rhoend, int *maxfun, int iprint, void *state);
int prima_newuoa(prima_function calfun, int n, double *x, double rhobeg, double rhoend, int *maxfun, int iprint, void *state);
int prima_uobyqa(prima_function calfun, int n, double *x, double rhobeg, double rhoend, int *maxfun, int iprint, void *state);

Some fortran modules have to be renamed to allow building everything into one single fortran library.

This also introduces a CMake build system to build both the Fortran library and the C library in a cross-platform manner.

Comments welcome

zaikunzhang commented 1 year ago

ok, I applied all remarks, except for the array types, this is needed for C interoperability (actually it works too with gfortran but not ifort)

also we do need the _loc variables to avoid compiler error on the conversion since C types and Fortran types may differ

I've added comments for these issues

I understand and agree with what you said about the _loc variables. I will check other points. Thank you.

zaikunzhang commented 1 year ago

except for the array types, this is needed for C interoperability (actually it works too with gfortran but not ifort)

@jschueller I remember you mentioned this issue before. What about other compilers? Isn't it a bug of ifort (I would not be surprised)? Would you mind creating a minimal working example and asking a question on Fortran Discourse and Intel Community? With you example, I can also try other compilers, e.g., nagfor, nvfortran, ... I constantly test PRIMA on ten compilers and make sure that they are all happy.

Explicit-shape arrays are not ideal. It is worthwhile looking a bit deeper into this issue. Thank you.

zaikunzhang commented 1 year ago

https://github.com/jschueller/prima/blob/763e43d911bec6e797faf4eabcfba9d0f71a87f5/c/bobyqa_c.f90#L42-L49

Type conversion should be explicit, or many compilers will complain.

For example,

x_loc = real(x, kind(x_loc))
...
x = real(x_loc, kind(x))

or

x_loc = real(x, RP)
...
x = real(x_loc, C_DOUBLE)
jschueller commented 1 year ago

it's not a compiler bug, assumed-shape array are not compatible because the compiler passes a descriptor instead of the address of the data, it's a known fact, but information is scarce: https://community.intel.com/t5/Intel-Fortran-Compiler/C-interoperability-with-assumed-size-arrays/td-p/1252532

FrancoisGallard commented 1 year ago

@zaikunzhang do you think the C interface is ok for you ? I was thinking maybe it could be merged. The Python part probably still needs discussion though, that's why I removed it for now, but I can still propose it in a later PR if needed, waiting for a pure Python implementation (#37). 0001-Add-Python-binding.txt

Sorry for the delayed reply.

I have been working on some revisions to lincoa and cobyla, which are motivated by your interface. The revisions would imply changes to the interfaces.

In short, lincoa and cobyla will support more types of constraints natively. See

https://github.com/libprima/prima/blob/dev_cobyla/fortran/lincoa/lincoa.f90 https://github.com/libprima/prima/blob/dev_cobyla/fortran/cobyla/cobyla.f90

for details. Sorry but this takes a lot of time to code.

Hi Zaikun, This is awesome, thanks a lot. There is a side effect of handling bounds, the initial DOE may go above the upper bounds. In the Nlopt version they fixed it by doikg a negative step when needed and the DOE is performed with a step rhobeg*(upperbound-lowerbound) because variables may have very different scales. How did you handle these issues?

zaikunzhang commented 1 year ago

https://github.com/jschueller/prima/blob/c35c2beff7de56a226664b1f18d9e2de13a954f6/c/bobyqa_c.f90#L61C1-L62C1

"allows to avoids" should be "allow to avoid".

zaikunzhang commented 1 year ago

Hi @FrancoisGallard

Currently, lincoa and cobyla accept bound constraints but do not respect them. Users do not need to write their own wrappers to translate bound constraints into linear/nonlinear constraints. Internally, however, bounds are handled in the same way as other linear/nonlinear constraints and hence can be violated.

In future versions (it needs time), the bounds will be respected. I will use the approach already introduced by Powell in BOBYQA. It either moves the starting point or shrinks rhobeg to make sure that the initial interpolation set stays within the bounds.

Scaling the variables is a related but different topic. It is nontrivial to code the scaling on the Fortran side due to the lack of lambda functions. It should be regarded as a preprocessing procedure that should be conducted by MATLAB/Python/R/Julia interfaces, as has been done in PDFO. In other words, we assume that the problem is well-scaled before being passed to the Fortran code from MATLAB/Python/R/Julia. I do not think this is an unreasonable requirement, because the scaling is much easier to implement in the latter languages.

The MATLAB interface of PRIMA does the scaling if the user requests so by setting scale = true in the options. To have the same in the Python interface, we may do so by inheriting the interface of PDFO, which is well coded (by @ragonneau Tom).

Thanks.

Zaikun

jschueller commented 1 year ago

https://github.com/jschueller/prima/blob/c35c2beff7de56a226664b1f18d9e2de13a954f6/c/bobyqa_c.f90#L61C1-L62C1

"allows to avoids" should be "allow to avoid".

ok

zaikunzhang commented 1 year ago

Hi @jschueller ,

A new release has been made. It enables lincoa and cobyla to support more types of constraints natively. The signatures of lincoa.f90 and cobyla.f90 are adapted accordingly. It would be great if you could incorporate the changes.

I do not foresee any other significant changes to the signatures of the principle Fortran subroutines (cobyla.f90, uobyqa.f90, newuoa.f90, bobyqa.f90, lincoa.f90) in the near future, if ever. That said, further updates to the Fortran implementation will unlikely affect the C/Python interface.

Many thanks for your efforts.

Best regards, Zaikun

zaikunzhang commented 1 year ago

https://github.com/jschueller/prima/blob/cport/c/cobyla_c.f90 https://github.com/jschueller/prima/blob/cport/c/lincoa_c.f90

  1. It will be better to output cstrv as well.
  2. For COBYLA, nlconstr (constr in v0.5.1) must be outputted.
zaikunzhang commented 1 year ago

https://github.com/jschueller/prima/blob/be30e74eeafc919b095c498007d08388f718f66b/c/bobyqa_c.f90#L38-L56

I suggest modifying these lines as follows. 

! Local variables
integer(IK) :: info_loc
integer(IK) :: iprint_loc
integer(IK) :: maxfun_loc
integer(IK) :: nf_loc
real(RP) :: f_loc
real(RP) :: rhobeg_loc
real(RP) :: rhoend_loc
real(RP) :: x_loc(n)
real(RP) :: xl_loc(n)
real(RP) :: xu_loc(n)

! Read the inputs and convert them to the Fortran side types
x_loc = real(x, kind(x_loc))
xl_loc = real(xl, kind(xl_loc))
xu_loc = real(xu, kind(xu_loc))
rhobeg_loc = real(rhobeg, kind(rhobeg))
rhoend_loc = real(rhoend, kind(rhoend))
maxfun_loc = int(maxfun, kind(maxfun_loc))
iprint_loc = int(iprint, kind(iprint_loc))

! Call the Fortran code
call bobyqa(calfun, x_loc, f_loc, xl=xl_loc, xu=xu_loc, nf=nf_loc, rhobeg=rhobeg_loc, rhoend=rhoend_loc, &
    & maxfun=maxfun_loc, iprint=iprint_loc, info=info_loc)

! Write the outputs
x = real(x_loc, kind(x))
f = real(f_loc, kind(f))
nf = int(nf_loc, kind(nf))
info = int(info_loc, kind(info))

Note the following.

  1. Each local variable takes a line, and they are ordered alphabetically. This will be beneficial in case other local variables are added in the future.
  2. For continued lines, there should be a & at the beginning.
  3. The same applies to cobyla.f90 etc.

Thanks.

jschueller commented 1 year ago

Ok, I updated to master, with the new interface.

zaikunzhang commented 1 year ago

Thank you @jschueller for the quick response.

Some quick comments on https://github.com/jschueller/prima/blob/cport/c/lincoa_c.f90 https://github.com/jschueller/prima/blob/cport/c/cobyla_c.f90 Some of them are applicable to other files.

  1. Sorry, but is there a specific reason why the new interface is not completely adapted, e.g., cobyla_c does not accept linear constraints? The very reason for my last month's work is to enable cobyla to accept such constraints. Without doing so, the user would have to wrap linear constraints as a "black-box nonlinear constraint", which is tedious and error-prone; in addition, it hurts the performance, because such a wrapped constraint hides the linear structure of the problem. Also, lincoa_c is not accepting linear equality constraints. See the signatures of scipy.optimize.minimize.cobyla, pdfo.cobyla, and pdfo.lincoa for your reference.
  2. nlconstr should be an output of cobyla.
  3. https://github.com/jschueller/prima/blob/10a35df678dcb0911653a4d0ba7e344ecfbfe43b/c/cobyla_c.f90#L39-L51 Could we sort the local variables alphabetically? For cobyla_c.f90, it would be the following. This applies to other files as well. 
    ! Local variables
integer(IK) :: info_loc
integer(IK) :: iprint_loc
integer(IK) :: m_loc
integer(IK) :: maxfun_loc
integer(IK) :: nf_loc
real(RP) :: cstrv_loc
real(RP) :: f_loc
real(RP) :: rhobeg_loc
real(RP) :: rhoend_loc
real(RP) :: x_loc(n)
real(RP) :: xl_loc(n)
real(RP) :: xu_loc(n)
  4. https://github.com/jschueller/prima/blob/10a35df678dcb0911653a4d0ba7e344ecfbfe43b/c/cobyla_c.f90#L64 This line seems quite long. For .f90 files, PRIMA generally sets a soft limit of 100 characters for each line, and a hard limit of 132 letters. Since this line is continued anyway, let us apply the soft limit (approximately). This applies to other lines and files as well.
jschueller commented 1 year ago

ok I've done the cosmetic changes (3 & 4)

but (1) I dont know if we want to expose linear constraints for the sake of simplicity for the same idea I dont think nlconstr is necessary since I already added cstrv (2)

zaikunzhang commented 1 year ago

ok I've done the cosmetic changes (3 & 4)

but (1) I dont know if we want to expose linear constraints for the sake of simplicity for the same idea I dont think nlconstr is necessary since I already added cstrv (2)

OK. I may have to give some comments from the viewpoint of numerical optimization algorithms (beyond coding). Most of them have been mentioned above, but it might be better to reiterate and clarify them.

  1. If linear constraints are not exposed, then they will have to be passed as "black-box nonlinear constraints". The obvious disadvantages are as follows.
    1.1. It hurts the performance, because the algorithms will not be able to explore the linear structure. 1.2. The users will have to wrap the linear constraints into "black-box nonlinear constraints". This is tedious and error-prone.

  2. Why is it a must to return the value of the nonlinear constraint? Powell's algorithms are designed for derivative-free optimization problems. Objective/constraint functions of such problems are generally expensive to evaluate --- it is not uncommon that a single function evaluation takes minutes or hours (sometimes days or even more). In addition to the constraint violation cstrv, it is natural that the user also wants to know the value of the nonlinear constraint at the solution. If we do not return this value, then the user would have to perform an expensive function evaluation (minutes, hours, days, or more) to calculate it, which is an obvious waste because this value was already evaluated.

I believe that François @FrancoisGallard will agree with these two points. He is a practitioner that uses these algorithms in real projects. Maybe François could give some comments.

Thanks.

zaikunzhang commented 1 year ago

Hi @jschueller ,

It would be great to write some tests.

  1. Some linting scripts that compile the interfaces with as many compilers as possible, making sure that everything works without a warning. For your reference, here is what I do for the MATLAB interfaces: https://github.com/libprima/prima/blob/main/matlab/mex_gateways/tests/makefiles/Makefile.common https://github.com/libprima/prima/blob/main/fortran/common/mlint You may try 
    cd fortran/cobyla && bash ./mlint -g

    I am very bad at Makefiles and bash. What I do is probably clumsy, but it works.

  2. Some tests to verify that interfaces work correctly.
  3. Some tests to benchmark the solvers against other standard libraries, e.g., PDFO and NLopt, which provide Powell's methods based on the Fortran 77 implementation.

See https://github.com/orgs/libprima/discussions/39 for more comments on tests. I do not mean that these tests should be implemented right now, but we need to have them gradually implemented to some extent. They are all done for the MATLAB interface.

Thanks.

jschueller commented 1 year ago

I understand its best to explicit linear constraints, lets keep things simple for now because I think its safe to assume most usages will involve implicit constraints.

for 1. I only tested gcc and the intel compilers, I can try to extend it to the other compilers for 2. I have already CI tests running that verify the interface works

  1. is out of scope of this PR, and anyway I dont see the point since it will give the same results as the pure fortran layer
zaikunzhang commented 1 year ago

I understand its best to explicit linear constraints, lets keep things simple for now because I think its safe to assume most usages will involve implicit constraints.

Thank you. However, I don't think it will complicate the interface a lot to accept linear constraints in COBYLA and linear equality constraints in LINCOA, whereas the benefits are essential.

For COBYLA, returning the nonlinear constraint value is a must, and it is trivial.

for 1. I only tested gcc and the intel compilers, I can try to extend it to the other compilers

Great!

for 2. I have already CI tests running that verify the interface works

Excellent!

  1. is out of scope of this PR, and anyway I dont see the point since it will give the same results as the pure fortran layer

This is a complement of 2. If the problems tested in 2 are sufficiently diversified, then 3 may be skipped for the moment.

The diversity of tests in 2 is sufficient if we test at least tens of problems with different sizes, types, features, and difficulties, and they are better randomised. If we test only one or two problems, we will not be sure whether the interface always works correctly or we are just lucky.

Thank you very much for your efforts and consideration.

github-actions[bot] commented 1 year ago

@check-spelling-bot Report

:red_circle: Please review

See the :open_file_folder: files view, the :scroll:action log or :angel: SARIF report for details.

Unrecognized words (4)

cfun FUNPTR hexval PROCPOINTER

To accept :heavy_check_mark: these unrecognized words as correct, run the following commands ... in a clone of the [git@github.com:jschueller/prima.git](https://github.com/jschueller/prima.git) repository on the `cport` branch ([:information_source: how do I use this?]( https://github.com/check-spelling/check-spelling/wiki/Accepting-Suggestions)): ``` sh curl -s -S -L 'https://raw.githubusercontent.com/check-spelling/check-spelling/main/apply.pl' | perl - 'https://github.com/libprima/prima/actions/runs/5771488930/attempts/1' ```
If the flagged items are :exploding_head: false positives If items relate to a ... * binary file (or some other file you wouldn't want to check at all). Please add a file path to the `excludes.txt` file matching the containing file. File paths are Perl 5 Regular Expressions - you can [test]( https://www.regexplanet.com/advanced/perl/) yours before committing to verify it will match your files. `^` refers to the file's path from the root of the repository, so `^README\.md$` would exclude [README.md]( ../tree/HEAD/README.md) (on whichever branch you're using). * well-formed pattern. If you can write a [pattern](https://github.com/check-spelling/check-spelling/wiki/Configuration-Examples:-patterns) that would match it, try adding it to the `patterns.txt` file. Patterns are Perl 5 Regular Expressions - you can [test]( https://www.regexplanet.com/advanced/perl/) yours before committing to verify it will match your lines. Note that patterns can't match multiline strings.
github-actions[bot] commented 1 year ago

@check-spelling-bot Report

:red_circle: Please review

See the :open_file_folder: files view, the :scroll:action log or :angel: SARIF report for details.

Unrecognized words (4)

cfun FUNPTR hexval PROCPOINTER

To accept :heavy_check_mark: these unrecognized words as correct, run the following commands ... in a clone of the [git@github.com:jschueller/prima.git](https://github.com/jschueller/prima.git) repository on the `cport` branch ([:information_source: how do I use this?]( https://github.com/check-spelling/check-spelling/wiki/Accepting-Suggestions)): ``` sh curl -s -S -L 'https://raw.githubusercontent.com/check-spelling/check-spelling/main/apply.pl' | perl - 'https://github.com/libprima/prima/actions/runs/5771521192/attempts/1' ```
If the flagged items are :exploding_head: false positives If items relate to a ... * binary file (or some other file you wouldn't want to check at all). Please add a file path to the `excludes.txt` file matching the containing file. File paths are Perl 5 Regular Expressions - you can [test]( https://www.regexplanet.com/advanced/perl/) yours before committing to verify it will match your files. `^` refers to the file's path from the root of the repository, so `^README\.md$` would exclude [README.md]( ../tree/HEAD/README.md) (on whichever branch you're using). * well-formed pattern. If you can write a [pattern](https://github.com/check-spelling/check-spelling/wiki/Configuration-Examples:-patterns) that would match it, try adding it to the `patterns.txt` file. Patterns are Perl 5 Regular Expressions - you can [test]( https://www.regexplanet.com/advanced/perl/) yours before committing to verify it will match your lines. Note that patterns can't match multiline strings.
github-actions[bot] commented 1 year ago

@check-spelling-bot Report

:red_circle: Please review

See the :open_file_folder: files view, the :scroll:action log or :angel: SARIF report for details.

Unrecognized words (3)

cfun FUNPTR PROCPOINTER

To accept :heavy_check_mark: these unrecognized words as correct, run the following commands ... in a clone of the [git@github.com:jschueller/prima.git](https://github.com/jschueller/prima.git) repository on the `cport` branch ([:information_source: how do I use this?]( https://github.com/check-spelling/check-spelling/wiki/Accepting-Suggestions)): ``` sh curl -s -S -L 'https://raw.githubusercontent.com/check-spelling/check-spelling/main/apply.pl' | perl - 'https://github.com/libprima/prima/actions/runs/5771581696/attempts/1' ```
If the flagged items are :exploding_head: false positives If items relate to a ... * binary file (or some other file you wouldn't want to check at all). Please add a file path to the `excludes.txt` file matching the containing file. File paths are Perl 5 Regular Expressions - you can [test]( https://www.regexplanet.com/advanced/perl/) yours before committing to verify it will match your files. `^` refers to the file's path from the root of the repository, so `^README\.md$` would exclude [README.md]( ../tree/HEAD/README.md) (on whichever branch you're using). * well-formed pattern. If you can write a [pattern](https://github.com/check-spelling/check-spelling/wiki/Configuration-Examples:-patterns) that would match it, try adding it to the `patterns.txt` file. Patterns are Perl 5 Regular Expressions - you can [test]( https://www.regexplanet.com/advanced/perl/) yours before committing to verify it will match your lines. Note that patterns can't match multiline strings.
zaikunzhang commented 1 year ago

Hi @jschueller,

I understand that you are still debugging, so cintrf.f90 is not in its final form. Here is a version I suggest when you finalize it.

module cintrf_mod
!--------------------------------------------------------------------------------------------------!
! cintrf_mod provides interoperability with C objective functions
!
! Dedicated to the late Professor M. J. D. Powell FRS (1936--2015).
!--------------------------------------------------------------------------------------------------!

implicit none
private
public :: COBJ, COBJCON, evalcobj, evalcobjcon

abstract interface

    subroutine COBJ(x, f) bind(c)
    use, intrinsic :: iso_c_binding, only : C_DOUBLE
    implicit none
    ! We cannot use assumed-shape arrays for C interoperability
    real(C_DOUBLE), intent(in) :: x(*)
    real(C_DOUBLE), intent(out) :: f
    end subroutine COBJ

    subroutine COBJCON(x, f, constr) bind(c)
    use, intrinsic :: iso_c_binding, only : C_DOUBLE
    implicit none
    ! We cannot use assumed-shape arrays for C interoperability
    real(C_DOUBLE), intent(in) :: x(*)
    real(C_DOUBLE), intent(out) :: f
    real(C_DOUBLE), intent(out) :: constr(*)
    end subroutine COBJCON

end interface

contains

subroutine evalcobj(cobj_ptr, x, f)
use, non_intrinsic :: consts_mod, only : RP
use, intrinsic :: iso_c_binding, only : C_DOUBLE, C_FUNPTR, C_F_PROCPOINTER
implicit none
type(C_FUNPTR), intent(in), value :: cobj_ptr
real(RP), intent(in) :: x(:)
real(RP), intent(out) :: f

! Local variables
procedure(COBJ), pointer :: obj_ptr
real(C_DOUBLE) :: f_loc
real(C_DOUBLE) :: x_loc(size(x))

! Read the inputs and convert them to the types specified in COBJ
x_loc = real(x, kind(x_loc))
call C_F_PROCPOINTER(cobj_ptr, obj_ptr)

! Call the C objective function
call obj_ptr(x_loc, f_loc)

! Write the output
f = real(f_loc, kind(f))

end subroutine evalcobj

subroutine evalcobjcon(cobjcon_ptr, x, f, constr)
use, non_intrinsic :: consts_mod, only : RP
use, intrinsic :: iso_c_binding, only : C_DOUBLE, C_FUNPTR, C_F_PROCPOINTER
implicit none
type(C_FUNPTR), intent(in), value :: cobjcon_ptr
real(RP), intent(in) :: x(:)
real(RP), intent(out) :: f
real(RP), intent(out) :: constr(:)

! Local variables
procedure(COBJCON), pointer :: objcon_ptr
real(C_DOUBLE) :: constr_loc(size(constr))
real(C_DOUBLE) :: f_loc
real(C_DOUBLE) :: x_loc(size(x))

! Read the inputs and convert them to the types specified in COBJCON
x_loc = real(x, kind(x_loc))
call C_F_PROCPOINTER(cobjcon_ptr, objcon_ptr)

call objcon_ptr(x_loc, f_loc, constr_loc)

! Write the outputs
f = real(f_loc, kind(f))
constr = real(constr_loc, kind(constr))

end subroutine evalcobjcon

N.B.:

  1. obj_ptr and objcon_ptr seem more expressive and more consistent with COBJ and COBJCON.

  2. Initialization of f/f_loc and constr/constr_loc is not needed.

  3. I understand that the +10 in the size of constr_loc is for debugging, and I am sure that you will not include it in the final version. This kind of code would hide bugs. It is better to make the code crash when there is a bug rather than covering it.

  4. I ordered the local variables alphabetically.

Many thanks for your efforts and consideration.

Zaikun

zaikunzhang commented 1 year ago

https://github.com/jschueller/prima/actions/runs/5771947549/workflow#L41-L46

  1. Compilers supported by the Fortran implementation of PRIMA are gfrotran, ifort, ifx, nagfor, nvfortran, Classic Flang, AOCC Flang, Arm Flang. The latest versions of these compilers are constantly tested using GitHub actions. Absoft af95, Oracle sunf95, and g95 are not supported, because they are discontinued without sufficient support for Fortran 2008 constructs.

  2. For tests and verification, -Wall is not sufficient for gfortran. -Wextra (and even Werror) should be included as well. Warnings should not be tolerated except for those due to compiler bugs. Below are the compiler options I use for tests and debugging. See https://github.com/libprima/prima/blob/main/fortran/tests/makefiles/Makefile.common for more detail.

    • AOCC Flang: -pedantic -Weverything -Wall -Wextra -Minform=warn -Mstandard -Mbounds -Mrecursive
    • Classic Flang: -pedantic -Weverything -Wall -Wextra -Minform=warn -Mstandard -Mbounds -Mrecursive -Mchkptr
    • gfortran: -Wall -Wextra -Wpedantic -pedantic -fimplicit-none -frecursive -fcheck=all -fstack-check
    • ifort and ifx: -warn all -debug extended -fimplicit-none -standard-semantics -assume recursion
    • nagfor: -u -C -C=alias -C=dangling -C=intovf -C=undefined -kind=unique -Warn=constant_coindexing -Warn=subnormal
    • nvfortran: -C -Wall -Wextra -Minform=warn -Mstandard -Mrecursive -Mbounds -Mchkstk -Mchkptr -Mchkstk
    • Arm Flang: -pedantic -Weverything -Wall -Wextra -Minform=warn -Mstandard -Mrecursive -Mbounds -Mchkptr

  3. Up to v0.6, the Fortran implementation of COBYLA does not work with AOCC 4.0 Flang. I believe it is a bug of the compiler. I note that AOCC 4.1 is released yesterday. ~To be tested.~ The problem remains.

Thanks.

zaikunzhang commented 1 year ago

Hi @jschueller ,

It would be great to write some tests.

  1. Some linting scripts that compile the interfaces with as many compilers as possible, making sure that everything works without a warning. For your reference, here is what I do for the MATLAB interfaces: https://github.com/libprima/prima/blob/main/matlab/mex_gateways/tests/makefiles/Makefile.common https://github.com/libprima/prima/blob/main/fortran/common/mlint You may try
cd fortran/cobyla && bash ./mlint -g

I am very bad at Makefiles and bash. What I do is probably clumsy, but it works. 2. Some tests to verify that interfaces work correctly. 3. Some tests to benchmark the solvers against other standard libraries, e.g., PDFO and NLopt, which provide Powell's methods based on the Fortran 77 implementation.

See https://github.com/orgs/libprima/discussions/39 for more comments on tests. I do not mean that these tests should be implemented right now, but we need to have them gradually implemented to some extent. They are all done for the MATLAB interface.

Thanks.

@jschueller and anyone interested in the development of PRIMA, make sure to read

https://github.com/orgs/libprima/discussions/39

My experiences in past years teach me three things.

  1. I do not know what will happen in a case until I have made sufficiently many tests about it.
  2. When I believe a test is stupid and unnecessary, the test will show me later that I am the stupid one.
  3. When I believe that I know numerical computation and my code well enough, some tests will show me that I am wrong.
zaikunzhang commented 1 year ago

https://github.com/jschueller/prima/blob/cport/c/prima.c

I suggest renaming prima_function_con to prima_objcon and prima_function to prima_obj, to be consistent with https://github.com/jschueller/prima/blob/cport/c/cintrf.f90.

In addition, function_con does not work so well by itself:

zaikunzhang commented 1 year ago

https://github.com/jschueller/prima/blob/cport/c/prima.c

The function passed to cobyla is called calcfc rather than calfun.

zaikunzhang commented 1 year ago

Sorry for my ignorance about CMake. Where is the place to specify compiler options? The following options must be applied so that the local arrays and automatic arrays will be allocated on the heap, rather than on the stack. Otherwise, stack overflow may occur if the problem is sufficiently large.

N.B.: In the context of derivative-free optimization, the computational cost is dominated by function evaluations (i.e., calfun or calcfc), a single of which can take minutes, hours, days, or even months. Therefore, it makes no sense to worry about the performance difference between stack and heap in the implementation of the solvers.

github-actions[bot] commented 1 year ago

@check-spelling-bot Report

:red_circle: Please review

See the :open_file_folder: files view, the :scroll:action log or :angel: SARIF report for details.

Unrecognized words (4)

cfun cobj NVHPC objcon

To accept :heavy_check_mark: these unrecognized words as correct, run the following commands ... in a clone of the [git@github.com:jschueller/prima.git](https://github.com/jschueller/prima.git) repository on the `cport` branch ([:information_source: how do I use this?]( https://github.com/check-spelling/check-spelling/wiki/Accepting-Suggestions)): ``` sh curl -s -S -L 'https://raw.githubusercontent.com/check-spelling/check-spelling/main/apply.pl' | perl - 'https://github.com/libprima/prima/actions/runs/5796830564/attempts/1' ```
If the flagged items are :exploding_head: false positives If items relate to a ... * binary file (or some other file you wouldn't want to check at all). Please add a file path to the `excludes.txt` file matching the containing file. File paths are Perl 5 Regular Expressions - you can [test]( https://www.regexplanet.com/advanced/perl/) yours before committing to verify it will match your files. `^` refers to the file's path from the root of the repository, so `^README\.md$` would exclude [README.md]( ../tree/HEAD/README.md) (on whichever branch you're using). * well-formed pattern. If you can write a [pattern](https://github.com/check-spelling/check-spelling/wiki/Configuration-Examples:-patterns) that would match it, try adding it to the `patterns.txt` file. Patterns are Perl 5 Regular Expressions - you can [test]( https://www.regexplanet.com/advanced/perl/) yours before committing to verify it will match your lines. Note that patterns can't match multiline strings.
github-actions[bot] commented 1 year ago

@check-spelling-bot Report

:red_circle: Please review

See the :open_file_folder: files view, the :scroll:action log or :angel: SARIF report for details.

Unrecognized words (1)

cmflags

To accept :heavy_check_mark: these unrecognized words as correct, run the following commands ... in a clone of the [git@github.com:jschueller/prima.git](https://github.com/jschueller/prima.git) repository on the `cport` branch ([:information_source: how do I use this?]( https://github.com/check-spelling/check-spelling/wiki/Accepting-Suggestions)): ``` sh curl -s -S -L 'https://raw.githubusercontent.com/check-spelling/check-spelling/main/apply.pl' | perl - 'https://github.com/libprima/prima/actions/runs/5797040128/attempts/1' ```
If the flagged items are :exploding_head: false positives If items relate to a ... * binary file (or some other file you wouldn't want to check at all). Please add a file path to the `excludes.txt` file matching the containing file. File paths are Perl 5 Regular Expressions - you can [test]( https://www.regexplanet.com/advanced/perl/) yours before committing to verify it will match your files. `^` refers to the file's path from the root of the repository, so `^README\.md$` would exclude [README.md]( ../tree/HEAD/README.md) (on whichever branch you're using). * well-formed pattern. If you can write a [pattern](https://github.com/check-spelling/check-spelling/wiki/Configuration-Examples:-patterns) that would match it, try adding it to the `patterns.txt` file. Patterns are Perl 5 Regular Expressions - you can [test]( https://www.regexplanet.com/advanced/perl/) yours before committing to verify it will match your lines. Note that patterns can't match multiline strings.
github-actions[bot] commented 1 year ago

@check-spelling-bot Report

:red_circle: Please review

See the :open_file_folder: files view, the :scroll:action log or :angel: SARIF report for details.

Unrecognized words (1)

cmflags

To accept :heavy_check_mark: these unrecognized words as correct, run the following commands ... in a clone of the [git@github.com:jschueller/prima.git](https://github.com/jschueller/prima.git) repository on the `cport` branch ([:information_source: how do I use this?]( https://github.com/check-spelling/check-spelling/wiki/Accepting-Suggestions)): ``` sh curl -s -S -L 'https://raw.githubusercontent.com/check-spelling/check-spelling/main/apply.pl' | perl - 'https://github.com/libprima/prima/actions/runs/5797108628/attempts/1' ```
If the flagged items are :exploding_head: false positives If items relate to a ... * binary file (or some other file you wouldn't want to check at all). Please add a file path to the `excludes.txt` file matching the containing file. File paths are Perl 5 Regular Expressions - you can [test]( https://www.regexplanet.com/advanced/perl/) yours before committing to verify it will match your files. `^` refers to the file's path from the root of the repository, so `^README\.md$` would exclude [README.md]( ../tree/HEAD/README.md) (on whichever branch you're using). * well-formed pattern. If you can write a [pattern](https://github.com/check-spelling/check-spelling/wiki/Configuration-Examples:-patterns) that would match it, try adding it to the `patterns.txt` file. Patterns are Perl 5 Regular Expressions - you can [test]( https://www.regexplanet.com/advanced/perl/) yours before committing to verify it will match your lines. Note that patterns can't match multiline strings.
jschueller commented 1 year ago

ok thanks for the compiler flags, but I didnt succeed with aflang:

[  1%] Building Fortran object fortran/CMakeFiles/primaf.dir/common/consts.F90.o
F90-F-0004-Unable to open MODULE file iso_fortran_env.mod (/home/runner/work/prima/prima/fortran/common/consts.F90: 63)
F90/x86-64 Linux Flang - 1.5 2017-05-01: compilation aborted

isnt iso_fortran_env a standard module of fortran ?

zaikunzhang commented 1 year ago

Hi @jschueller ,

  1. I have updated scripts/install_aocc . Could you try it again? iso_fortran_env is standard, but aflang cannot find it for some reason starting from AOCC 4.1, unless aflang is located in the default bin directory of AOCC. If the new script still does not work, tell me.
  2. I did some cleaning to remove some unintended large files. This rewrites some history, but it should not interfere with the changes you make. I hope this will not create problem later when we merge your PR.

Thanks.

Zaikun

jschueller commented 1 year ago

hello @zaikunzhang

  1. thanks; now it builds but the cobyla & lincoa fortran examples crash or assert (not using the C layer)
  2. ok I synced to master
zaikunzhang commented 1 year ago

hello @zaikunzhang

  1. thanks; now it builds but the cobyla & lincoa fortran examples crash or assert (not using the C layer)

  2. ok I synced to master

Thank you @jschueller . Could you direct me to the failure log? I will check it.

Thank you.

jschueller commented 1 year ago

yes; see https://github.com/jschueller/prima/actions/runs/5801361977/job/15725463847

here I run the fortran/examples as tests for the pure fortran side:


1: Test command: /home/runner/work/prima/prima/fortran/t_primaf_cobyla
1: Working Directory: /home/runner/work/prima/prima/fortran
1: Test timeout computed to be: 10000000
 1/10 Test  #1: cobyla_f .........................***Exception: SegFault  0.11 sec
...
: Test command: /home/runner/work/prima/prima/fortran/t_primaf_lincoa
5: Working Directory: /home/runner/work/prima/prima/fortran
5: Test timeout computed to be: 10000000
5: 0: Subscript out of range for array x (/home/runner/work/prima/prima/fortran/common/linalg.f90: 665)
5:     subscript=6, lower bound=1, upper bound=5, dimension=1
 5/10 Test  #5: lincoa_f .........................***Failed    0.01 sec
0: Subscript out of range for array x (/home/runner/work/prima/prima/fortran/common/linalg.f90: 665)
    subscript=6, lower bound=1, upper bound=5, dimension=1
...
The following tests FAILED:
Errors while running CTest
      1 - cobyla_f (SEGFAULT)
      5 - lincoa_f (Failed)
zaikunzhang commented 1 year ago

yes; see https://github.com/jschueller/prima/actions/runs/5801361977/job/15725463847

here I run the fortran/examples as tests for the pure fortran side:


1: Test command: /home/runner/work/prima/prima/fortran/t_primaf_cobyla

1: Working Directory: /home/runner/work/prima/prima/fortran

1: Test timeout computed to be: 10000000

 1/10 Test  #1: cobyla_f .........................***Exception: SegFault  0.11 sec

...

: Test command: /home/runner/work/prima/prima/fortran/t_primaf_lincoa

5: Working Directory: /home/runner/work/prima/prima/fortran

5: Test timeout computed to be: 10000000

5: 0: Subscript out of range for array x (/home/runner/work/prima/prima/fortran/common/linalg.f90: 665)

5:     subscript=6, lower bound=1, upper bound=5, dimension=1

 5/10 Test  #5: lincoa_f .........................***Failed    0.01 sec

0: Subscript out of range for array x (/home/runner/work/prima/prima/fortran/common/linalg.f90: 665)

    subscript=6, lower bound=1, upper bound=5, dimension=1

...

The following tests FAILED:

Errors while running CTest

    1 - cobyla_f (SEGFAULT)

    5 - lincoa_f (Failed)

This particular error is due to a known bug of aocc flang:

https://github.com/flang-compiler/flang/issues/1238

In short, the Fortran standard specifies that x(m:n) is an empty array when m>n, even if m is lager than the upper bound of the index. Aocc flang does not handle this correctly.

To circumvent this problem, you may remove "-Mbounds" from the options of aocc flang. This is not ideal, but we have no other solution before AMD fixed the bug.

Any other error on the Fortran side?

zaikunzhang commented 1 year ago

https://github.com/jschueller/prima/blob/cport/c/cintrf.f90

Hi @jschueller ,

May I ask what is the reason for packing f and constr together as fcon?

To me, this choice has the style of Powell's original Fortran 77 code, and it is exactly what PRIMA strives to avoid. I believe that each variable should have one and only one purpose, and different objects should not be packed together.

If we choose to use one variable to represent f and constr together, then this is the flow of the data:

  1. User evaluates f and constr
  2. User packs f and constr as fcon
  3. Algorithm reads fcon
  4. Algorithm decodes fcon into f and constr
  5. Algorithm solves the optimization problem and gets the final values of f and constr
  6. Algorithm packs f and constr as fcon and return it
  7. User reads fcon
  8. User decodes fcon as f and constr

I am not sure what is the advantage of this flow compared with transferring f and constr separately.

If we had many (say 5) objects like f and constr to be evaluated together, then I would agree that it might be better to pack them into a structure. But it seems not the case here.

Thank you.

jschueller commented 1 year ago

thanks, removing -Mbounds took care of the lincoa assert on the fortran side there is another problem: the cobyla example segfaults with aocc:

Program received signal SIGSEGV, Segmentation fault.
0x0000555555555d10 in calcfc_mod::calcfc_chebyquad (f=<error reading variable: Cannot access memory at address 0x0>, x=<error reading variable: Location address is not set.>, constr=...) at /home/runner/work/prima/prima/fortran/examples/cobyla/cobyla_example.f90:22
22  subroutine calcfc_chebyquad(x, f, constr)
#0  0x0000555555555d10 in calcfc_mod::calcfc_chebyquad (f=<error reading variable: Cannot access memory at address 0x0>, x=<error reading variable: Location address is not set.>, constr=...) at /home/runner/work/prima/prima/fortran/examples/cobyla/cobyla_example.f90:22
#1  0x00007ffff7f126e9 in calcfc_internal (f_internal=<error reading variable: Cannot access memory at address 0x0>, x_internal=<error reading variable: Location address is not set.>, constr_internal=...) at /home/runner/work/prima/prima/fortran/cobyla/cobylb.f90:667
#2  0x00007ffff7ed8891 in evaluate_mod::evaluatefc (calcfc=0x4155415641574155, f=0.04642817229746083, x=..., constr=...) at /home/runner/work/prima/prima/fortran/common/evaluate.f90:191
#3  0x00007ffff7f1b1b7 in initialize_cobyla_mod::initxfc (calcfc=<optimized out>, iprint=<optimized out>, maxfun=<optimized out>, ctol=<optimized out>, f0=<optimized out>, ftarget=<optimized out>, rhobeg=<optimized out>, nf=<optimized out>, info=<optimized out>, constr0=..., x0=..., chist=..., conhist=..., conmat=..., cval=..., fhist=..., fval=..., sim=..., simi=..., xhist=..., evaluated=...) at /home/runner/work/prima/prima/fortran/cobyla/initialize.f90:156
#4  0x00007ffff7f0e708 in cobylb_mod::cobylb (calcfc=<optimized out>, iprint=<optimized out>, maxfilt=<optimized out>, maxfun=<optimized out>, ctol=<optimized out>, cweight=<optimized out>, eta1=<optimized out>, eta2=<optimized out>, ftarget=<optimized out>, gamma1=<optimized out>, gamma2=<optimized out>, rhobeg=<optimized out>, rhoend=<optimized out>, f=<optimized out>, nf=<optimized out>, cstrv=<optimized out>, info=<optimized out>, amat=..., bvec=..., constr=..., x=..., chist=..., conhist=..., fhist=..., xhist=...) at /home/runner/work/prima/prima/fortran/cobyla/cobylb.f90:213
#5  0x00007ffff7f09224 in cobyla_mod::cobyla (calcfc=1096237397, m_nlcon=0, f=0.04642817229746083, cstrv=0, f0=0, nf=0, rhobeg=0, rhoend=0, ftarget=0, ctol=0, cweight=0, maxfun=0, iprint=0, eta1=0, eta2=0, gamma1=0, gamma2=0, maxhist=0, maxfilt=0, info=0, x=<error reading variable: value requires 750599937872512 bytes, which is more than max-value-size>, nlconstr=<error reading variable: Cannot access memory at address 0x0>, aineq=<error reading variable: value requires 6262704000 bytes, which is more than max-value-size>, bineq=..., aeq=<error reading variable: value requires 710976 bytes, which is more than max-value-size>, beq=..., xl=<error reading variable: value requires 17518252251111699504 bytes, which is more than max-value-size>, xu=..., nlconstr0=<error reading variable: value requires 6262704000 bytes, which is more than max-value-size>, xhist=<not allocated>, fhist=<not allocated>, chist=<not allocated>, nlchist=<not allocated>) at /home/runner/work/prima/prima/fortran/cobyla/cobyla.f90:600
#6  0x00005555555568ea in cobyla_exmp () at /home/runner/work/prima/prima/fortran/examples/cobyla/cobyla_example.f90:106

full log: https://github.com/jschueller/prima/actions/runs/5807492241/job/15742432101

as for the argument packing, that was just to try if it can avoid another cobyla segfault on the C side

zaikunzhang commented 1 year ago

thanks, removing -Mbounds took care of the lincoa assert on the fortran side there is another problem: the cobyla example segfaults with aocc:

Program received signal SIGSEGV, Segmentation fault.
0x0000555555555d10 in calcfc_mod::calcfc_chebyquad (f=<error reading variable: Cannot access memory at address 0x0>, x=<error reading variable: Location address is not set.>, constr=...) at /home/runner/work/prima/prima/fortran/examples/cobyla/cobyla_example.f90:22
22    subroutine calcfc_chebyquad(x, f, constr)
#0  0x0000555555555d10 in calcfc_mod::calcfc_chebyquad (f=<error reading variable: Cannot access memory at address 0x0>, x=<error reading variable: Location address is not set.>, constr=...) at /home/runner/work/prima/prima/fortran/examples/cobyla/cobyla_example.f90:22
#1  0x00007ffff7f126e9 in calcfc_internal (f_internal=<error reading variable: Cannot access memory at address 0x0>, x_internal=<error reading variable: Location address is not set.>, constr_internal=...) at /home/runner/work/prima/prima/fortran/cobyla/cobylb.f90:667
#2  0x00007ffff7ed8891 in evaluate_mod::evaluatefc (calcfc=0x4155415641574155, f=0.04642817229746083, x=..., constr=...) at /home/runner/work/prima/prima/fortran/common/evaluate.f90:191
#3  0x00007ffff7f1b1b7 in initialize_cobyla_mod::initxfc (calcfc=<optimized out>, iprint=<optimized out>, maxfun=<optimized out>, ctol=<optimized out>, f0=<optimized out>, ftarget=<optimized out>, rhobeg=<optimized out>, nf=<optimized out>, info=<optimized out>, constr0=..., x0=..., chist=..., conhist=..., conmat=..., cval=..., fhist=..., fval=..., sim=..., simi=..., xhist=..., evaluated=...) at /home/runner/work/prima/prima/fortran/cobyla/initialize.f90:156
#4  0x00007ffff7f0e708 in cobylb_mod::cobylb (calcfc=<optimized out>, iprint=<optimized out>, maxfilt=<optimized out>, maxfun=<optimized out>, ctol=<optimized out>, cweight=<optimized out>, eta1=<optimized out>, eta2=<optimized out>, ftarget=<optimized out>, gamma1=<optimized out>, gamma2=<optimized out>, rhobeg=<optimized out>, rhoend=<optimized out>, f=<optimized out>, nf=<optimized out>, cstrv=<optimized out>, info=<optimized out>, amat=..., bvec=..., constr=..., x=..., chist=..., conhist=..., fhist=..., xhist=...) at /home/runner/work/prima/prima/fortran/cobyla/cobylb.f90:213
#5  0x00007ffff7f09224 in cobyla_mod::cobyla (calcfc=1096237397, m_nlcon=0, f=0.04642817229746083, cstrv=0, f0=0, nf=0, rhobeg=0, rhoend=0, ftarget=0, ctol=0, cweight=0, maxfun=0, iprint=0, eta1=0, eta2=0, gamma1=0, gamma2=0, maxhist=0, maxfilt=0, info=0, x=<error reading variable: value requires 750599937872512 bytes, which is more than max-value-size>, nlconstr=<error reading variable: Cannot access memory at address 0x0>, aineq=<error reading variable: value requires 6262704000 bytes, which is more than max-value-size>, bineq=..., aeq=<error reading variable: value requires 710976 bytes, which is more than max-value-size>, beq=..., xl=<error reading variable: value requires 17518252251111699504 bytes, which is more than max-value-size>, xu=..., nlconstr0=<error reading variable: value requires 6262704000 bytes, which is more than max-value-size>, xhist=<not allocated>, fhist=<not allocated>, chist=<not allocated>, nlchist=<not allocated>) at /home/runner/work/prima/prima/fortran/cobyla/cobyla.f90:600
#6  0x00005555555568ea in cobyla_exmp () at /home/runner/work/prima/prima/fortran/examples/cobyla/cobyla_example.f90:106

as for the argument packing, that was just to try if it can avoid another cobyla segfault on the C side

The segfault seems to be the same one as https://github.com/libprima/prima/issues/41

I guess it is a bug of aocc, but not sure, as I have not got the time to boil it down to a minimal example (I would be much grateful if you happen to be able to do it).

It has been reported to aocc https://community.amd.com/t5/server-gurus-discussions/aocc-4-1-flang-quot-0-allocate-xxx-bytes-requested-not-enough/m-p/623584#M2038 , but I don’t expect anyone will respond according to my previous experiences with aocc.

That said, we don’t fully support aocc for the moment, likely due to aocc's bug.

Great to hear your explanation about the packing. So I suppose it is for debugging only and will be de-packed in the final version.

Thank you.

zaikunzhang commented 1 year ago

Sorry for my ignorance about CMake. Where is the place to specify compiler options? The following options must be applied so that the local arrays and automatic arrays will be allocated on the heap, rather than on the stack. Otherwise, stack overflow may occur if the problem is sufficiently large.

  • AOCC Flang: nothing

  • Classic Flang: nothing

  • gfortran: -fno-stack-arrays

  • ifort and ifx: -heap-arrays

  • nagfor: nothing

  • nvfortran: -Mnostack_arrays

  • Arm Flang: -fno-stack-arrays

N.B.: In the context of derivative-free optimization, the computational cost is dominated by function evaluations (i.e., calfun or calcfc), a single of which can take minutes, hours, days, or even months. Therefore, it makes no sense to worry about the performance difference between stack and heap in the implementation of the solvers.

Of course, it is always helpful to have -g included during the debugging, or gdb may not provide the full information. Thanks.

jschueller commented 1 year ago

I set the stack array flags in the root CMakeLists then the cmake workflow script sets release mode + debug info (see CMAKE_BUILD_TYPE) so that adds -g

jschueller commented 1 year ago

@zaikunzhang I disabled the cobyla test for now in the CI because of the aocc bug and possibly another one affecting all the 3 flang-based compilers (still investigating) do you think it is ready to be merged ?

github-actions[bot] commented 1 year ago

@check-spelling-bot Report

:red_circle: Please review

See the :open_file_folder: files view, the :scroll:action log or :angel: SARIF report for details.

Unrecognized words (1)

cobjcon

To accept :heavy_check_mark: these unrecognized words as correct, run the following commands ... in a clone of the [git@github.com:jschueller/prima.git](https://github.com/jschueller/prima.git) repository on the `cport` branch ([:information_source: how do I use this?]( https://github.com/check-spelling/check-spelling/wiki/Accepting-Suggestions)): ``` sh curl -s -S -L 'https://raw.githubusercontent.com/check-spelling/check-spelling/main/apply.pl' | perl - 'https://github.com/libprima/prima/actions/runs/5817806576/attempts/1' ```
If the flagged items are :exploding_head: false positives If items relate to a ... * binary file (or some other file you wouldn't want to check at all). Please add a file path to the `excludes.txt` file matching the containing file. File paths are Perl 5 Regular Expressions - you can [test]( https://www.regexplanet.com/advanced/perl/) yours before committing to verify it will match your files. `^` refers to the file's path from the root of the repository, so `^README\.md$` would exclude [README.md]( ../tree/HEAD/README.md) (on whichever branch you're using). * well-formed pattern. If you can write a [pattern](https://github.com/check-spelling/check-spelling/wiki/Configuration-Examples:-patterns) that would match it, try adding it to the `patterns.txt` file. Patterns are Perl 5 Regular Expressions - you can [test]( https://www.regexplanet.com/advanced/perl/) yours before committing to verify it will match your lines. Note that patterns can't match multiline strings.
github-actions[bot] commented 1 year ago

@check-spelling-bot Report

:red_circle: Please review

See the :open_file_folder: files view, the :scroll:action log or :angel: SARIF report for details.

Unrecognized words (1)

cobjcon

To accept :heavy_check_mark: these unrecognized words as correct, run the following commands ... in a clone of the [git@github.com:jschueller/prima.git](https://github.com/jschueller/prima.git) repository on the `cport` branch ([:information_source: how do I use this?]( https://github.com/check-spelling/check-spelling/wiki/Accepting-Suggestions)): ``` sh curl -s -S -L 'https://raw.githubusercontent.com/check-spelling/check-spelling/main/apply.pl' | perl - 'https://github.com/libprima/prima/actions/runs/5817849699/attempts/1' ```
If the flagged items are :exploding_head: false positives If items relate to a ... * binary file (or some other file you wouldn't want to check at all). Please add a file path to the `excludes.txt` file matching the containing file. File paths are Perl 5 Regular Expressions - you can [test]( https://www.regexplanet.com/advanced/perl/) yours before committing to verify it will match your files. `^` refers to the file's path from the root of the repository, so `^README\.md$` would exclude [README.md]( ../tree/HEAD/README.md) (on whichever branch you're using). * well-formed pattern. If you can write a [pattern](https://github.com/check-spelling/check-spelling/wiki/Configuration-Examples:-patterns) that would match it, try adding it to the `patterns.txt` file. Patterns are Perl 5 Regular Expressions - you can [test]( https://www.regexplanet.com/advanced/perl/) yours before committing to verify it will match your lines. Note that patterns can't match multiline strings.
zaikunzhang commented 1 year ago

Hi @jschueller ,

First of all, thank you very much for your efforts.

I made some comments. I would like to invite @ragonneau to help me review things again since I am not familiar enough with C.

Sorry to repeat this: I have a serious concern about the interfaces of COBYLA and LINCOA. 1. LINCOA should accept linear equality constraints as well. 2. COBYLA should accept linear constraints as well. 3. nlconstr should be an output of COBYLA.

For anyone interested in these points, I attache at the end of this comment the explanations made before.

I hope to emphasize two things in addition.

1. While being vital, the above-mentioned points would not complicate the interfaces essentially. 2. Once the interfaces are released, it would be very difficult to make changes to them. Wide decisions must be made now.

Thank you very much.

Zaikun

OK. I may have to give some comments from the viewpoint of numerical optimization algorithms (beyond coding). Most of them have been mentioned above, but it might be better to reiterate and clarify them.

  1. If linear constraints are not exposed, then they will have to be passed as "black-box nonlinear constraints". The obvious disadvantages are as follows. 1.1. It hurts the performance, because the algorithms will not be able to explore the linear structure. 1.2. The users will have to wrap the linear constraints into "black-box nonlinear constraints". This is tedious and error-prone.
  2. Why is it a must to return the value of the nonlinear constraint? Powell's algorithms are designed for derivative-free optimization problems. Objective/constraint functions of such problems are generally expensive to evaluate --- it is not uncommon that a single function evaluation takes minutes or hours (sometimes days or even more). In addition to the constraint violation cstrv, it is natural that the user also wants to know the value of the nonlinear constraint at the solution. If we do not return this value, then the user would have to perform an expensive function evaluation (minutes, hours, days, or more) to calculate it, which is an obvious waste because this value was already evaluated.

I believe that François @FrancoisGallard will agree with these two points. He is a practitioner that uses these algorithms in real projects. Maybe François could give some comments.

Thanks.

zaikunzhang commented 1 year ago

@zaikunzhang I disabled the cobyla test for now in the CI because of the aocc bug and possibly another one affecting all the 3 flang-based compilers (still investigating).

Do we still test cobyla when the compiler is not flang-based (i.e., gcc an ifort)? I hope so.

another one affecting all the 3 flang-based compilers

Could you direct me to the error log? On the pure fortran side, nvfortran and Classic flang do not have problems. AOCC flang is a bit left behind due to the vendor(AMD) itself and we have no much to do.

I addition, I have self-hosted runners with access to nagfor:

    runs-on: [self-hosted, nagfor]

Would you mind adding a yml that can make the tests on these runners using nagfor? nagfor is quite popular.

Thanks.

zaikunzhang commented 1 year ago

@FrancoisGallard and @jalvesz , you are also invited to give comments if interested. Thank you.

zaikunzhang commented 1 year ago

@zaikunzhang do you think the C interface is ok for you ? I was thinking maybe it could be merged.

The Python part probably still needs discussion though, that's why I removed it for now, but I can still propose it in a later PR if needed, waiting for a pure Python implementation (#37). 0001-Add-Python-binding.txt

The Python interface is of high importance. Making the solvers available in SciPy has a high priority. So yes, please propose it in a later PR.

A pure Python implementation is being worked on, but it is highly nontrivial, and sufficiently extensive tests are needed. I believe SciPy has to interface with the Fortran implementation first. Thanks.

zaikunzhang commented 1 year ago

Thank you @jschueller for working on the new version.

nlconstr_loc is better declared as nlconstr_loc(m_nlcon).

Indeed, since we enforce the automatic arrays to be allocated on the heap, the code behaves essentially the same as using an allocatable array. But it is more expressive, saves two lines, and eliminates the possibility of memory leaking.

Thanks.

jschueller commented 1 year ago

Thank you @jschueller for working on the new version.

nlconstr_loc is better declared as nlconstr_loc(m_nlcon).

Indeed, since we enforce the automatic arrays to be allocated on the heap, the code behaves essentially the same as using an allocatable array. But it is more expressive, saves two lines, and eliminates the possibility of memory leaking.

Thanks.

this doesnt work since the cobyla function wants an allocatable argument and if I specify the size it says it should have deferred shape