Pipeline running time has increased by 200 s

[agioffe]

From the end of April to the beginning of June the running time of the main part of the pipeline (before anything is being written to disk) has dramatically increased from ~ 340 s up to ~ 540 s. Need to profile and see what is the reason. How could I accurately roll the AstroPack back to the end of April and then forward to the present state to measure and see what has caused it?

[EastEriq]

It's git checkout v2.1 vs. git checkout HEAD. For finer control you can checkout a specific commit SHA helping yourself figuring up with the commit messages, they are all "*.m", haha.

BUT

HEAD is more or less necessary for superunit observation, hence if you revert some machines to v2.1 you have to warn observers. If AstroPack was modular, it would be considerable to checkout different modules at different time snapshots, but as Astropack is now, I think that is hopeless.

[EranOfek]

please provide details - e.g., dataset location, and full profiling.

On Thu, Jun 6, 2024 at 9:04 AM Alexander Krassilchtchikov < @.***> wrote:

From the end of April to the beginning of June the running time of the main part of the pipeline (before anything is being written to disk) has dramatically increased from ~ 340 s up to ~ 540 s. Need to profile and see what is the reason. How could I accurately roll the AstroPack back to the end of April and then forward to the present state to measure and see what has caused it?

— Reply to this email directly, view it on GitHub https://github.com/EranOfek/AstroPack/issues/453, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJUQ4MXLBNXUIABFAYTTETZF73WNAVCNFSM6AAAAABI4AZPR2VHI2DSMVQWIX3LMV43ASLTON2WKOZSGMZTOMZYGE3DOMY . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[EranOfek]

he is testing it on his machine...

On Thu, Jun 6, 2024 at 9:38 AM EastEriq @.***> wrote:

It's git checkout v2.1 vs. git checkout HEAD. For finer control you can checkout a specific commit SHA helping yourself figuring up with the commit messages, they are all "*.m", haha.

BUT

HEAD is more or less necessary for superunit observation, hence if you revert some machines to v2.1 you have to warn observers. If AstroPack was modular, it would be considerable to checkout different modules at different time snapshots, but as Astropack is now, I think that is hopeless.

— Reply to this email directly, view it on GitHub https://github.com/EranOfek/AstroPack/issues/453#issuecomment-2151517324, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJUQ4KOKHWM4T75WC7I4DLZF77YDAVCNFSM6AAAAABI4AZPR2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNJRGUYTOMZSGQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[agioffe]

The dataset is the standard visit employed by pipeline's unit test at pipeline.DemonLAST.unitTest: 20 images starting from LAST.01.08.03_20230616.222625.384

Here are the profiling results (first HEAD, then v. 2.1):

[EranOfek]

How can we retrieve this data set? using the Installer?

On Thu, Jun 6, 2024 at 1:57 PM Alexander Krassilchtchikov < @.***> wrote:

The dataset is the standard visit employed by pipeline's unit test at pipeline.DemonLAST.unitTest: 20 images starting from LAST.01.08.03_20230616.222625.384

Here are the profiling results (first HEAD, then v. 2.1): HEAD.png (view on web) https://github.com/EranOfek/AstroPack/assets/117931420/dffb13e1-ae71-46ef-966f-5c1a981feed9 v2.1.png (view on web) https://github.com/EranOfek/AstroPack/assets/117931420/9fde7b41-819e-403c-84af-67d4dcf6b025

— Reply to this email directly, view it on GitHub https://github.com/EranOfek/AstroPack/issues/453#issuecomment-2152014948, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJUQ4JF4JPSNXP4IDI3YNTZGA6CXAVCNFSM6AAAAABI4AZPR2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNJSGAYTIOJUHA . You are receiving this because you commented.Message ID: @.***>

[agioffe]

Yes, this can be installed as 'LASTpipelineUnitTest'

[agioffe]

Changing of 1 flag suggested in https://github.com/EranOfek/AstroPack/issues/454 also cured this issue, with the processing speed of the whole pipeline going back to 380~s total, with all the disk writes and DB interactions.