search

Extreme-classification / ngame

11 stars 3 forks source link

Config file advice for DEXA compared to NGAME #4

Open NitinAggarwal1 opened 2 months ago

NitinAggarwal1 commented 2 months ago

I see the DEXA Specific Parameters for LF-AmazonTitles131K.json . "aux_num_vectors": 66536, "aux_num_hlp_vectors": 1000,

Q1 :Any guideline on choosing right values for aux_num_vectors and aux_num_hlp_vectors and what are your practical learnings on this number from cost / gains perspective . How that would differ May be for 10k or 2M classes .

Q2: sampling_init_cluster_size": 32 for Extreme Module in Dexa vs NGAME . Also sampling_curr_epochs": [20, 1000] ( Any reasons)

Q3: Any other advice to try DEXA and compare with RENEE ( I am looking at it to solve label semantic gap where the label text is not descriptive or discrete enough to capture multiple intents that match with that label)

kunaldahiya commented 2 months ago

Thanks for checking out DEXA and NGAME.

Q1. (a) 'aux_num_vectors' can be decided based on your budget. In general, 65K or 131K is a reasonable number for 2M classes. (b) 'aux_num_hlp_vectors' is optional and it can be used for highly popular labels (say >=500 training points).

Q2. These are minor details leading to minor differences - they probably won't have that much impact on your dataset.

Q3. Renee is better suited when label is not annotated with label text. Otherwise, it depends on NGAME's pretrained encoder to get its best performance especially on short-text-datasets.

NitinAggarwal1 commented 2 months ago

Will try and share how it goes . In our case RENEE for Product Search was able to reach P@1 at 75 vs NGAME Fusion P@1 76 . Renee was much faster, I did not use pretrained NGAME encoder , I will try that too .