Truly parallel

[nvoss12838]

Current vision is to just split by either stations or date ranges to different nodes on cluster.

Is there a more efficient way to farm this out?

[roccomalservisi]

Not sure…

That is the way that also D’Agostino in Rome was doing it. I guess if you want to update the initial solution at every step you will need to do something to keep some kind of order. But the current processing consider every measurement completely independent so it does not need to know anything from other solutions…

On Apr 23, 2018, at 5:32 PM, nvoss notifications@github.com<mailto:notifications@github.com> wrote:

Current vision is to just split by either stations or date ranges to different nodes on cluster.

Is there a more efficient way to farm this out?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/USFgeodesy/xProcess/issues/2, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AKk8RmxzZTI4GDsCaOj4DtKPzcTUE61Eks5trkhvgaJpZM4TgqEy.

[nvoss12838]

I am using the flag -nodes in my call to gd2e.py. This allows the user to set number of cores for processing each station-day (via mpi I think?). It is currently hard coded to use 4 nodes in the current station.process class. I will probably add this as a flag for the user.

[ghost]

well, when I was running on Cluster three years ago, I found not equal solution for running on parallel (each day processed on a note independently) and for running on the tower in normal mode (not parallel). I think depending on how you write your wrapper, GIPSY could use the initial coordinate from previous solution or from Rinex header or from a fast solution using code data. I think for now it's better to run on a tower to understand how new GIPSY handles the sequential processing and then try running on parallel next.

M.A.K

Makan A. Karegar,

Ph.D. candidate, School of Geosciences, University of South Florida.

4202 E. Fowler Avenue, NES 107, Tampa, Florida, USA.

ORCID: orcid.org/0000-0001-7120-0649

https://www.researchgate.net/profile/Makan_Karegar

On Tue, Apr 24, 2018 at 1:13 AM, nvoss notifications@github.com wrote:

I am using the flag -nodes in my call to gd2e.py. This allows the user to set number of cores for processing each station-day (via mpi I think?). It is currently hard coded to use 4 nodes in the current station.process class. I will probably add this as a flag for the user.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/USFgeodesy/xProcess/issues/2#issuecomment-383751557, or mute the thread https://github.com/notifications/unsubscribe-auth/ALf4TkXtymE3sXdv-l6ndZA1CZqu97Gmks5trmAdgaJpZM4TgqEy .

[roccomalservisi]

I do not think i have ever updated the solution nor the rinex solution. I had always used the position from the position from sta info file updating that if I need.

I do all my computation on cluster.

Sent from my T-Mobile 4G LTE Device

-------- Original message -------- From: MakanKaregar notifications@github.com Date: 4/24/18 4:22 AM (GMT-05:00) To: USFgeodesy/xProcess xProcess@noreply.github.com Cc: "Malservisi, Rocco" rocco@usf.edu, Comment comment@noreply.github.com Subject: Re: [USFgeodesy/xProcess] Truly parallel (#2)

well, when I was running on Cluster three years ago, I found not equal solution for running on parallel (each day processed on a note independently) and for running on the tower in normal mode (not parallel). I think depending on how you write your wrapper, GIPSY could use the initial coordinate from previous solution or from Rinex header or from a fast solution using code data. I think for now it's better to run on a tower to understand how new GIPSY handles the sequential processing and then try running on parallel next.

M.A.K

Makan A. Karegar,

Ph.D. candidate, School of Geosciences, University of South Florida.

4202 E. Fowler Avenue, NES 107, Tampa, Florida, USA.

ORCID: orcid.org/0000-0001-7120-0649

https://www.researchgate.net/profile/Makan_Karegar

On Tue, Apr 24, 2018 at 1:13 AM, nvoss notifications@github.com wrote:

I am using the flag -nodes in my call to gd2e.py. This allows the user to set number of cores for processing each station-day (via mpi I think?). It is currently hard coded to use 4 nodes in the current station.process class. I will probably add this as a flag for the user.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/USFgeodesy/xProcess/issues/2#issuecomment-383751557, or mute the thread https://github.com/notifications/unsubscribe-auth/ALf4TkXtymE3sXdv-l6ndZA1CZqu97Gmks5trmAdgaJpZM4TgqEy .

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/USFgeodesy/xProcess/issues/2#issuecomment-383846593, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AKk8Rpkev-Iag_iAZnA-gJWfZpL4lBdKks5truC4gaJpZM4TgqEy.

[nvoss12838]

One issue right now is that when I collect all the end files I do not order them by date, so days get processed randomly between the specified date ranges. So I would have to order them in order to update the initial position with the previous day.