pmodels / mpich

Official MPICH Repository
http://www.mpich.org
Other
560 stars 279 forks source link

PATCH: increment MAX_FINALIZE_FUNC #337

Closed mpichbot closed 8 years ago

mpichbot commented 8 years ago

Originally by "Lisandro Dalcin" dalcinl@gmail.com on 2008-12-22 16:34:33 -0600



The testsuite of mpi4py fails bad with current trunk. As this
testsuite exercises almost all MPI features (P2P,COLL,RMA,I/O, etc.)
in a single run (I mean, between MPI_Init/MPI_Finalize), I guess
that's the reason this problem showed up for me (and probably never
for you).

Anyway, I'm still in doubt about this. IMHO, 32 entries could not
enough if in the near future if the usage of those finalizers
increases in the mpich2 codebase. A slightly harder way could be taken
by using a linked list... But following the KISS principle, a more
generous value would be probably  enough and surely harmless regarding
process memory consumption, right?.

--
Lisandro Dalcín
---------------
Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC)
Instituto de Desarrollo Tecnológico para la Industria Química (INTEC)
Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET)
PTLC - Güemes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594
mpichbot commented 8 years ago

Originally by Lisandro Dalcin on 2008-12-22 16:34:33 -0600


This message has 1 attachment(s)

mpichbot commented 8 years ago

Originally by Lisandro Dalcin on 2008-12-22 16:34:33 -0600


Attachment added: src_mpi_init_finalize.diff (0.5 KiB) Added by email2trac

mpichbot commented 8 years ago

Originally by Rajeev Thakur on 2008-12-23 00:54:02 -0600


Attachment added: part0001.mht (0.0 KiB) Added by email2trac

mpichbot commented 8 years ago

Originally by Rajeev Thakur on 2008-12-23 00:54:02 -0600


Attachment added: part0002.ksh (1.2 KiB) Added by email2trac

mpichbot commented 8 years ago

Originally by Rajeev Thakur on 2008-12-23 00:54:02 -0600


Attachment added: part0003.html (2.6 KiB) Added by email2trac

mpichbot commented 8 years ago

Originally by Rajeev Thakur on 2008-12-23 00:54:02 -0600



I had a cursory look at this. I don't think there are even 16 Finalize
callbacks, and the ones that are there should get added only once. So I am
not sure how that number is being exceeded in any circumstance. Need to look
into it more carefully. Attaching the original mail from mpich2-dev here.

Rajeev

> -----Original Message-----
> From: mpich2-bugs-bounces@mcs.anl.gov
> [mailto:mpich2-bugs-bounces@mcs.anl.gov] On Behalf Of mpich2
> Sent: Monday, December 22, 2008 4:35 PM
> To: undisclosed-recipients:
> Subject: [mpich2-maint] #337: PATCH: increment MAX_FINALIZE_FUNC
>
> ---------------------------------------------------+----------
> ----------
> ---------------------------------------------------+----
>  Reporter:  "Lisandro Dalcin" <dalcinl@gmail.com>  |
> Type:  bug
>    Status:  new                                    |
> Priority:  major
> Milestone:                                         |
> Component:  mpich2
> ---------------------------------------------------+----------
> ----------
> ---------------------------------------------------+----
>
>
>  {{{
>
>  The testsuite of mpi4py fails bad with current trunk. As
> this  testsuite exercises almost all MPI features
> (P2P,COLL,RMA,I/O, etc.)  in a single run (I mean, between
> MPI_Init/MPI_Finalize), I guess  that's the reason this
> problem showed up for me (and probably never  for you).
>
>  Anyway, I'm still in doubt about this. IMHO, 32 entries
> could not  enough if in the near future if the usage of those
> finalizers  increases in the mpich2 codebase. A slightly
> harder way could be taken  by using a linked list... But
> following the KISS principle, a more  generous value would be
> probably  enough and surely harmless regarding  process
> memory consumption, right?.
>
>
>  --
>  Lisandro Dalcín
>  ---------------
>  Centro Internacional de Métodos Computacionales en
> Ingeniería (CIMEC)  Instituto de Desarrollo Tecnológico para
> la Industria Química (INTEC)  Consejo Nacional de
> Investigaciones Científicas y Técnicas (CONICET)  PTLC -
> Güemes 3450, (3000) Santa Fe, Argentina
>  Tel/Fax: +54-(0)342-451.1594
>  ```
>
>
> --
> Ticket URL: <https://trac.mcs.anl.gov/projects/mpich2/ticket/337>
>
}}}
mpichbot commented 8 years ago

Originally by gropp on 2008-12-30 09:13:36 -0600


Lisandro is right - one recent change that I made to improve the ability of MPICH2 to detect memory leaks adds a finalize handler for each MPI handle type that is managed by the handle memory routines. As finalized unwinds these, it can detect when, for example, there is an unfreed MPI_Request that was allocated from the direct (preallocated) pool, which cannot be identified with valgrind or similar tools. I'll double the max handlers (that should be enough, even when ROMIO adds the handler to free the keyvals that it allocates).

mpichbot commented 8 years ago

Originally by thakur on 2009-01-01 00:57:52 -0600


Fixed by Bill in [a6b0983873a5458895f69bea67fd1692c11759b7]