TREX on Tabular Data

[ghost]

@dsbrown1331 @hiwonjoon

Can TREX be used on tabular data? Have you tested an imitation learning based method on tabular data?

[dsbrown1331]

Yes, TREX can be used on tabular data. For example, if you have a grid world and each cell has some set of features associated with it, then you can use the same approach to approximate the reward function given suboptimal demonstrations in the grid world. We experimented with this in a follow-on paper and found that a Bayesian version of T-REX works quite well on pixel-based control as well as on simple grid worlds. (https://arxiv.org/pdf/2002.09089.pdf)

Did you have a specific question about dealing with tabular data?

[ghost]

@dsbrown1331 What i have is not a grid world, take this example:

• i have +1,000,000 maps.
• each map contains some houses.
• for each single map, i have the customer's preference of the houses in that area. meaning for each map i already had the customer rank each house according to his preference, ranking which is 1st, 2nd, 3rd, etc...

The game is:

• i want to use the +1,000,000 maps and their rankings, to train an imitation learning model.

The goal is:

• to use the trained model on new and unseen maps, to predict the ranking of each house.
• Therefore, each map have rank1, rank2, rank3,......

The solution:

• The customer is ranking each house according to it's location, and also according to it's neighbors location.

The data and features:

• Each map have some houses, for each house i have these features which are basically the box and rank:
• x
• y
• width
• height
• rank

The Question: Can your imitation learning method be used to solve such task? There is a method to the madness, we humans can see the pattern easily, but can your model see the pattern?

[dsbrown1331]

That seems doable. Since it's not a sequential decision making task, then maybe T-REX isn't the best fit, but there are many other learning to rank methods and these often operate similar to T-REX. T-REX will output a predicted score for each house that could be then converted into a ranking. The inputs would be your features and then using the pairwise ranking loss you would train a classifier to predict which house is better out of a pair of houses.

[ghost]

there are many other learning to rank methods such as?

T-REX will output a predicted score for each house will the score produced for each house be independent of the other houses? or will the other houses scores/features influence how each house is scored?

then using the pairwise ranking loss can you clarify more. do you mean comparing each 2 houses? how should i apply this to all houses?

So should i use TREX, or there is a particular method that you suggest? because currently i am using decision tree, but do you recommend any other method?

[dsbrown1331]

Learning to rank: https://en.wikipedia.org/wiki/Learning_to_rank

The scores will not be independent since they will be trained relative to each other. T-REX takes a pairwise preference and trains a model such that it gives higher score to the better ranked item. If you have a total ordering you can just take all pairs and use the T-REX pairwise ranking loss. T-REX uses a Bradley-Terry pairwise ranking loss that is pretty standard so I think it's a good thing to start with.

On Mon, Aug 31, 2020 at 9:41 PM deepseek notifications@github.com wrote:

there are many other learning to rank methods such as?

T-REX will output a predicted score for each house will the score produced for each house be independent of the other houses? or will the other houses scores/features influence how each house is scored?

then using the pairwise ranking loss can you clarify more. do you mean comparing each 2 houses? how should i apply this to all houses?

So should i use TREX, or there is a particular method that you suggest?

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/hiwonjoon/ICML2019-TREX/issues/8#issuecomment-684178237, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAK724RCQNNEEOQQJSZJLU3SDRUOZANCNFSM4QMEMHQA .

[ghost]

are you saying that T-REX can be used like decision trees, inwhich i input features and it can rank

[dsbrown1331]

Training a network via a pairwise ranking loss like done in T-REX will result in a network that takes as input the features and then outputs a score. If you then sort the items based on the output scores it will give you a predicted ranking of the instances.

On Tue, Sep 1, 2020 at 12:41 PM deepseek notifications@github.com wrote:

are you saying that T-REX can be used like decision trees, inwhich i input features and it can rank

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/hiwonjoon/ICML2019-TREX/issues/8#issuecomment-685061003, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAK724WTAYAYN7MXMWZPE6DSDU55FANCNFSM4QMEMHQA .