search

SJrX / SMAC

Java Implementation of Sequential Model-based Algorithm Configuration
4 stars 4 forks source link

Querying the trained model #3

Closed stephanheinemann closed 2 years ago

stephanheinemann commented 3 years ago

After having trained the model, I am looking for a method to query a given ProblemInstance. I am looking for something like

ParameterConfiguration pc = runHistory.query(ProblemInstance pi)

SMAC allows me to validate ParameterConfigurations for ProblemInstances but this is not quite what i am looking for. The closest method I could find is

public List selectChallengersWithEI(int numChallengers)

but it does not allow me to get a suitable predicted configuration for an arbitrary ProblemInstance. I was not able to find anything in the manual either. What do I need to do to obtain a suggested ParameterConfugration for a given ProblemInstance?

frank-hutter commented 3 years ago

Sorry, I think you’re confused; while it internally uses the concept of instances for variance reduction, SMAC outputs a single configuration that is good on average across instances, not a mapping from an instance to a configuration. For the latter, you’d like to, e.g., use Hydra: https://www.cs.ubc.ca/~hoos/Publ/XuEtAl10.pdf

On Sat 9. Oct 2021 at 22:15, Stephan Heinemann @.***> wrote:

After having trained the model, I am looking for a method to query a given ProblemInstance. I am looking for something like

ParameterConfiguration pc = runHistory.query(ProblemInstance pi)

SMAC allows me to validate ParameterConfigurations for ProblemInstances but this is not quite what i am looking for. The closest method I could find is

public List selectChallengersWithEI(int numChallengers)

but it does not allow me to get a suitable predicted configuration for an arbitrary ProblemInstance. I was not able to find anything in the manual either. What do I need to do to obtain a suggested ParameterConfugration for a given ProblemInstance?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/SJrX/SMAC/issues/3, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABLOK74JIJUG2OEJGROTZSLUGCPE3ANCNFSM5FVUL6SQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

stephanheinemann commented 3 years ago

Thanks Frank, Hydra might be an option but I think SMAC is still useful for me since I can associate an arbitrary problem instance to the set of training instances using domain knowledge about their features. If I have distinct problem sets, then I could use the incumbent of the associated set as the answer to my query.

Thanks again, Stephan

Sent from my iPhone

On Oct 9, 2021, at 13:27, Frank @.**@.>> wrote:

Sorry, I think you’re confused; while it internally uses the concept of instances for variance reduction, SMAC outputs a single configuration that is good on average across instances, not a mapping from an instance to a configuration. For the latter, you’d like to, e.g., use Hydra: https://www.cs.ubc.ca/~hoos/Publ/XuEtAl10.pdf

On Sat 9. Oct 2021 at 22:15, Stephan Heinemann @.***> wrote:

After having trained the model, I am looking for a method to query a given ProblemInstance. I am looking for something like

ParameterConfiguration pc = runHistory.query(ProblemInstance pi)

SMAC allows me to validate ParameterConfigurations for ProblemInstances but this is not quite what i am looking for. The closest method I could find is

public List selectChallengersWithEI(int numChallengers)

but it does not allow me to get a suitable predicted configuration for an arbitrary ProblemInstance. I was not able to find anything in the manual either. What do I need to do to obtain a suggested ParameterConfugration for a given ProblemInstance?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/SJrX/SMAC/issues/3, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABLOK74JIJUG2OEJGROTZSLUGCPE3ANCNFSM5FVUL6SQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/SJrX/SMAC/issues/3#issuecomment-939357281, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AB377TE6VATVGPKIR6HYCPDUGCQSRANCNFSM5FVUL6SQ.