search

TACC / Galaxy

Galaxy is an asynchronous parallel visualization ray tracer for performant rendering in distributed computing environments. Galaxy builds upon Intel OSPRay and Intel Embree, including ray queueing and sending logic inspired by TACC GraviT.
Apache License 2.0
19 stars 9 forks source link

Implement capability to run arbitrary scripts as part of testing #10

Open dhrogers opened 5 years ago

dhrogers commented 5 years ago

We need to expand the testing capability beyond just image comparison.

Running an arbitrary script would be a good next step.

pnav commented 5 years ago

Hi David,

Isn't this what the unit testing harness accomplishes? Now, those need to be more fully fleshed out, but the point of those are to verify functional units apart from the gold image tests.

See tests/src/framework/Application.cpp for an example.

If that's not what you have in mind, can you elaborate more on what you mean by "running an arbitrary script"?

GregAbram commented 5 years ago

We’re thinking about tests that use apps other than gxywriter. For example, we’re working on sampling…particle trace…trace-to-tube… render. The first and (soon) the last steps will use state files for setup, but it uses a separate app (sampletrace) to wrap the four steps. So I was thinking about having shell files in the tests directory that are searched for and executed (like it searches for .state files and gxywriter’s them). The shell files might refer to state files that have a different extension than .state so they don’t get caught up in the current approach. The assumption would be that the scripts would produce images for testing just like the current approach, but perhaps using several distinct steps and maybe using alternative apps to create the rendering.

On Aug 1, 2019, at 3:36 PM, Paul Navrátil notifications@github.com<mailto:notifications@github.com> wrote:

Hi David,

Isn't this what the unit testing harness accomplishes? Now, those need to be more fully fleshed out, but the point of those are to verify functional units apart from the gold image tests.

See tests/src/framework/Application.cpp for an example.

If that's not what you have in mind, can you elaborate more on what you mean by "running an arbitrary script"?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/TACC/Galaxy/issues/10?email_source=notifications&email_token=AAHZWRSHZVRO53CHE5T6RPDQCNJNVA5CNFSM4IIWE3LKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3L6XZA#issuecomment-517467108, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AAHZWRXAD5TQOR6VSDZCJCTQCNJNVANCNFSM4IIWE3LA.

This message is from an external sender. Learn more about why this matters.https://ut.service-now.com/sp?id=kb_article&number=KB0011401

pnav commented 5 years ago

got it. yep we should certainly test apps beyond gxywriter. ideally each should have its own test.

On Aug 1, 2019, at 17:17, Greg Abram notifications@github.com<mailto:notifications@github.com> wrote:

We’re thinking about tests that use apps other than gxywriter. For example, we’re working on sampling…particle trace…trace-to-tube… render. The first and (soon) the last steps will use state files for setup, but it uses a separate app (sampletrace) to wrap the four steps. So I was thinking about having shell files in the tests directory that are searched for and executed (like it searches for .state files and gxywriter’s them). The shell files might refer to state files that have a different extension than .state so they don’t get caught up in the current approach. The assumption would be that the scripts would produce images for testing just like the current approach, but perhaps using several distinct steps and maybe using alternative apps to create the rendering.

On Aug 1, 2019, at 3:36 PM, Paul Navrátil notifications@github.com<mailto:notifications@github.commailto:notifications@github.com> wrote:

Hi David,

Isn't this what the unit testing harness accomplishes? Now, those need to be more fully fleshed out, but the point of those are to verify functional units apart from the gold image tests.

See tests/src/framework/Application.cpp for an example.

If that's not what you have in mind, can you elaborate more on what you mean by "running an arbitrary script"?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/TACC/Galaxy/issues/10?email_source=notifications&email_token=AAHZWRSHZVRO53CHE5T6RPDQCNJNVA5CNFSM4IIWE3LKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3L6XZA#issuecomment-517467108, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AAHZWRXAD5TQOR6VSDZCJCTQCNJNVANCNFSM4IIWE3LA.

This message is from an external sender. Learn more about why this matters.https://ut.service-now.com/sp?id=kb_article&number=KB0011401

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHubhttps://github.com/TACC/Galaxy/issues/10?email_source=notifications&email_token=AAI74G62HCA6BU6HLNEIRELQCNOGPA5CNFSM4IIWE3LKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3MBNEA#issuecomment-517478032, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AAI74G2YMN43IIV7HRZ674DQCNOGPANCNFSM4IIWE3LA.

This message is from an external sender. Learn more about why this matters.https://ut.service-now.com/sp?id=kb_article&number=KB0011401

GregAbram commented 5 years ago

Would allow us to test the multiserver, too.

On Aug 1, 2019, at 4:19 PM, Paul Navrátil notifications@github.com<mailto:notifications@github.com> wrote:

got it. yep we should certainly test apps beyond gxywriter. ideally each should have its own test.

On Aug 1, 2019, at 17:17, Greg Abram notifications@github.com<mailto:notifications@github.commailto:notifications@github.com> wrote:

We’re thinking about tests that use apps other than gxywriter. For example, we’re working on sampling…particle trace…trace-to-tube… render. The first and (soon) the last steps will use state files for setup, but it uses a separate app (sampletrace) to wrap the four steps. So I was thinking about having shell files in the tests directory that are searched for and executed (like it searches for .state files and gxywriter’s them). The shell files might refer to state files that have a different extension than .state so they don’t get caught up in the current approach. The assumption would be that the scripts would produce images for testing just like the current approach, but perhaps using several distinct steps and maybe using alternative apps to create the rendering.

On Aug 1, 2019, at 3:36 PM, Paul Navrátil notifications@github.com<mailto:notifications@github.commailto:notifications@github.commailto:notifications@github.com> wrote:

Hi David,

Isn't this what the unit testing harness accomplishes? Now, those need to be more fully fleshed out, but the point of those are to verify functional units apart from the gold image tests.

See tests/src/framework/Application.cpp for an example.

If that's not what you have in mind, can you elaborate more on what you mean by "running an arbitrary script"?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/TACC/Galaxy/issues/10?email_source=notifications&email_token=AAHZWRSHZVRO53CHE5T6RPDQCNJNVA5CNFSM4IIWE3LKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3L6XZA#issuecomment-517467108, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AAHZWRXAD5TQOR6VSDZCJCTQCNJNVANCNFSM4IIWE3LA.

This message is from an external sender. Learn more about why this matters.https://ut.service-now.com/sp?id=kb_article&number=KB0011401

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHubhttps://github.com/TACC/Galaxy/issues/10?email_source=notifications&email_token=AAI74G62HCA6BU6HLNEIRELQCNOGPA5CNFSM4IIWE3LKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3MBNEA#issuecomment-517478032, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AAI74G2YMN43IIV7HRZ674DQCNOGPANCNFSM4IIWE3LA.

This message is from an external sender. Learn more about why this matters.https://ut.service-now.com/sp?id=kb_article&number=KB0011401

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/TACC/Galaxy/issues/10?email_source=notifications&email_token=AAHZWRUJHX3BI3WS7DB55CTQCNOO7A5CNFSM4IIWE3LKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3MBRBY#issuecomment-517478535, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AAHZWRRVH6CPTSRQWDZNQJTQCNOO7ANCNFSM4IIWE3LA.

This message is from an external sender. Learn more about why this matters.https://ut.service-now.com/sp?id=kb_article&number=KB0011401

dhrogers commented 5 years ago

I have a new application 'raysample' which can be used to test the new ray-based sampling capability. It creates images which could be compared to some 'gold' image, the way the current testing works.

pnav commented 5 years ago

Excellent! I'll find it and integrate it into the testing structure.

From: David H. Rogers notifications@github.com Sent: Friday, August 2, 2019 3:45 PM To: TACC/Galaxy Galaxy@noreply.github.com Cc: Paul Navratil pnav@tacc.utexas.edu; Assign assign@noreply.github.com Subject: Re: [TACC/Galaxy] Implement capability to run arbitrary scripts as part of testing (#10)

I have a new application 'raysample' which can be used to test the new ray-based sampling capability. It creates images which could be compared to some 'gold' image, the way the current testing works.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHubhttps://github.com/TACC/Galaxy/issues/10?email_source=notifications&email_token=AAI74G6POYXSZNEHWCCMAULQCSMG7A5CNFSM4IIWE3LKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3OZRUA#issuecomment-517839056, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AAI74GYF42W2YATCJT6L6MLQCSMG7ANCNFSM4IIWE3LA.

This message is from an external sender. Learn more about why this matters.https://ut.service-now.com/sp?id=kb_article&number=KB0011401

GregAbram commented 5 years ago

I’ve enhanced sample tracer so it takes 2 state files - one specifying the sampling, and one specifying the rendering. Here’s one of 4 renderings from it - will look way cooler when Intel fixes the sprinkles.

[cid:FA76F9FD-9F06-415C-B873-1354AB2F29AC@lanl.gov] Greg Abram gda@tacc.utexas.edumailto:gda@tacc.utexas.edu

On Aug 2, 2019, at 3:01 PM, Paul Navrátil notifications@github.com<mailto:notifications@github.com> wrote:

Excellent! I'll find it and integrate it into the testing structure.

From: David H. Rogers notifications@github.com<mailto:notifications@github.com> Sent: Friday, August 2, 2019 3:45 PM To: TACC/Galaxy Galaxy@noreply.github.com<mailto:Galaxy@noreply.github.com> Cc: Paul Navratil pnav@tacc.utexas.edu<mailto:pnav@tacc.utexas.edu>; Assign assign@noreply.github.com<mailto:assign@noreply.github.com> Subject: Re: [TACC/Galaxy] Implement capability to run arbitrary scripts as part of testing (#10)

I have a new application 'raysample' which can be used to test the new ray-based sampling capability. It creates images which could be compared to some 'gold' image, the way the current testing works.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHubhttps://github.com/TACC/Galaxy/issues/10?email_source=notifications&email_token=AAI74G6POYXSZNEHWCCMAULQCSMG7A5CNFSM4IIWE3LKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3OZRUA#issuecomment-517839056, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AAI74GYF42W2YATCJT6L6MLQCSMG7ANCNFSM4IIWE3LA.

This message is from an external sender. Learn more about why this matters.https://ut.service-now.com/sp?id=kb_article&number=KB0011401

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/TACC/Galaxy/issues/10?email_source=notifications&email_token=AAHZWRXBL3HIZ4D7NEXJIUDQCSOCTA5CNFSM4IIWE3LKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3O2Q6A#issuecomment-517843064, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AAHZWRTOV3XRFLCNTV7NMCLQCSOCTANCNFSM4IIWE3LA.

This message is from an external sender. Learn more about why this matters.https://ut.service-now.com/sp?id=kb_article&number=KB0011401