search

ltl3A87 / KB-BINDER

Few-shot In-context Learning for Knowledge Base Question Answering
MIT License
56 stars 7 forks source link

Inference Time #6

Open ri-git opened 7 months ago

ri-git commented 7 months ago

Could you please provide an estimate of how much time is required for the inference of one question and how long will it take to run inference on the entire grailqa dev set?

yhshu commented 6 months ago

Hello, I also want to ask about the estimated running time for each question. I have a running log that takes ~7 minutes for one question, I'm not sure if there's some setting or cache I haven't noticed.

2024-01-07 13:59:48 INFO     fb_roles loaded
2024-01-07 14:00:09 INFO     name_to_id_dict loaded
2024-01-07 14:00:33 INFO     ==========
2024-01-07 14:00:33 INFO     data[id]: 246000102
2024-01-07 14:00:33 INFO     data[question]: what are the common symptoms of fip and some disease caused by old age?
2024-01-07 14:00:33 INFO     data[exp]: (AND base.pethealth.symptom (AND (JOIN (R base.pethealth.pet_disease_or_medical_condition.symptoms) m.06_1k9) (JOIN (R base.pethealth.pet_disease_or_medical_condition.symptoms) (JOIN (R base.pethealth.pet_disease_risk_factor.pet_diseases_with_this_risk_factor) m.01hbgs))))
2024-01-07 14:00:33 INFO     top_score: -1.3732653608351366
2024-01-07 14:00:33 INFO     top related questions: ['the bipropellant rocket engine with kerosene and gas-generator cycle is designed by who?']
2024-01-07 14:00:35 INFO     HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
2024-01-07 14:00:35 INFO     gene_exps: ['(OR (AND veterinary_medicine.disease_common_symptoms (JOIN veterinary_medicine.disease_common_symptoms.disease fip)) (AND medical_conditions.disease_caused_by_old_age (JOIN medical_conditions.disease_caused_by_old_age.disease some_disease)))', '(OR (AND veterinary_medicine.disease.symptoms (JOIN veterinary_medicine.disease.symptoms (R veterinary_medicine.disease.name "fip"))) (AND medical_disease.symptoms (JOIN medical_disease.causes (R medical_disease.name "old age"))))', '(OR (AND veterinary_medicine.disease_common_symptoms (JOIN veterinary_medicine.disease_common_symptoms.disease fip)) (AND medical.disease_caused_by_age (JOIN medical.disease_caused_by_age.age old)))', '(OR (AND veterinary_medicine.disease_common_symptoms (JOIN veterinary_medicine.disease_common_symptoms.disease fip)) (AND medical.disease_caused_by (JOIN medical.disease_caused_by.cause old_age)))', '(OR (AND veterinary_medicine.disease_common_symptoms (JOIN veterinary_medicine.disease_common_symptoms.disease (R veterinary_medicine.disease "fip"))) (AND medical.disease_caused_by_age (R medical.disease "old age"))))', '(OR (AND veterinary.mammal_disease.symptoms (JOIN veterinary.mammal_disease.symptoms (R veterinary.mammal_disease.name "fip"))) (AND medical.disease.symptoms (JOIN medical.disease.symptoms (R medical.disease.caused_by "old age")))) (AND veterinary.mammal_disease.name "fip" (R veterinary.mammal_disease.caused_by "old age")))', '(OR (AND medical_condition.symptoms (JOIN medical_condition.symptoms.condition FIP)) (AND medical_condition.causes (JOIN medical_condition.causes.condition (AND (JOIN (R medical_condition.causes.condition) old age) disease))))']
2024-01-07 14:00:35 INFO     gene_exp: (OR (AND veterinary_medicine.disease_common_symptoms (JOIN veterinary_medicine.disease_common_symptoms.disease fip)) (AND medical_conditions.disease_caused_by_old_age (JOIN medical_conditions.disease_caused_by_old_age.disease some_disease)))
2024-01-07 14:00:35 INFO     all_iters: [('m.03nnmkj',), ('m.02pv7y8',), ('m.0g_0p_',), ('m.05yx0z',), ('m.071hyv',), ('m.074n2y',), ('m.04g1ktv',), ('m.0gttrfn',), ('m.04dvxh',), ('m.03c5gry',), ('m.051wrn1',), ('m.0j67b4h',), ('m.03nnmkj',), ('m.02pv7y8',), ('m.0g_0p_',)]
2024-01-07 14:00:35 INFO     mid_iters: ('m.03nnmkj',)
2024-01-07 14:00:42 INFO     top3_ques rela: []
2024-01-07 14:00:42 INFO     top3_ques rela: []
2024-01-07 14:00:42 INFO     mid_iters: ('m.02pv7y8',)
2024-01-07 14:00:48 INFO     top3_ques rela: []
2024-01-07 14:00:49 INFO     top3_ques rela: []
2024-01-07 14:00:49 INFO     mid_iters: ('m.0g_0p_',)
2024-01-07 14:00:55 INFO     top3_ques rela: []
2024-01-07 14:00:55 INFO     top3_ques rela: []
2024-01-07 14:00:55 INFO     mid_iters: ('m.05yx0z',)
2024-01-07 14:01:02 INFO     top3_ques rela: []
2024-01-07 14:01:02 INFO     top3_ques rela: []
2024-01-07 14:01:02 INFO     mid_iters: ('m.071hyv',)
2024-01-07 14:01:09 INFO     top3_ques rela: []
2024-01-07 14:01:09 INFO     top3_ques rela: []
2024-01-07 14:01:09 INFO     mid_iters: ('m.074n2y',)
2024-01-07 14:01:16 INFO     top3_ques rela: []
2024-01-07 14:01:16 INFO     top3_ques rela: []
2024-01-07 14:01:16 INFO     mid_iters: ('m.04g1ktv',)
2024-01-07 14:01:22 INFO     top3_ques rela: []
2024-01-07 14:01:23 INFO     top3_ques rela: []
2024-01-07 14:01:23 INFO     mid_iters: ('m.0gttrfn',)
2024-01-07 14:01:29 INFO     top3_ques rela: []
2024-01-07 14:01:29 INFO     top3_ques rela: []
2024-01-07 14:01:29 INFO     mid_iters: ('m.04dvxh',)
2024-01-07 14:01:36 INFO     top3_ques rela: []
2024-01-07 14:01:36 INFO     top3_ques rela: []
2024-01-07 14:01:36 INFO     mid_iters: ('m.03c5gry',)
2024-01-07 14:01:43 INFO     top3_ques rela: []
2024-01-07 14:01:43 INFO     top3_ques rela: []
2024-01-07 14:01:43 INFO     mid_iters: ('m.051wrn1',)
2024-01-07 14:01:50 INFO     top3_ques rela: []
2024-01-07 14:01:50 INFO     top3_ques rela: []
2024-01-07 14:01:50 INFO     mid_iters: ('m.0j67b4h',)
2024-01-07 14:01:56 INFO     top3_ques rela: []
2024-01-07 14:01:56 INFO     top3_ques rela: []
2024-01-07 14:01:56 INFO     mid_iters: ('m.03nnmkj',)
2024-01-07 14:01:57 INFO     top3_ques rela: []
2024-01-07 14:01:57 INFO     top3_ques rela: []
2024-01-07 14:01:57 INFO     mid_iters: ('m.02pv7y8',)
2024-01-07 14:01:57 INFO     top3_ques rela: []
2024-01-07 14:01:57 INFO     top3_ques rela: []
2024-01-07 14:01:57 INFO     mid_iters: ('m.0g_0p_',)
2024-01-07 14:01:57 INFO     top3_ques rela: []
2024-01-07 14:01:57 INFO     top3_ques rela: []
2024-01-07 14:01:57 INFO     predicted_answer: None
2024-01-07 14:01:57 INFO     label: ['m.023s6n', 'm.0brgy', 'm.0f3kl', 'm.0cjf0', 'm.01j6t0', 'm.0lcdk']
2024-01-07 14:01:57 INFO     ================================================================
2024-01-07 14:01:57 INFO     consistent candidates number: 1
2024-01-07 14:01:57 INFO     em_score: 0.0
2024-01-07 14:01:57 INFO     correct: 0
2024-01-07 14:01:57 INFO     total: 1
2024-01-07 14:01:57 INFO     no_ans: 1
2024-01-07 14:01:57 INFO      
2024-01-07 14:01:57 INFO     ================================================================
2024-01-07 14:01:57 INFO     gene_exp: (OR (AND veterinary_medicine.disease.symptoms (JOIN veterinary_medicine.disease.symptoms (R veterinary_medicine.disease.name "fip"))) (AND medical_disease.symptoms (JOIN medical_disease.causes (R medical_disease.name "old age"))))
2024-01-07 14:01:57 INFO     all_iters: [()]
2024-01-07 14:01:57 INFO     mid_iters: ()
2024-01-07 14:01:57 INFO     top3_ques rela: ['medicine.disease.symptoms', 'medicine.symptom.symptom_of', 'medicine.disease_cause.diseases', 'medicine.symptom.includes_symptoms', 'medicine.symptom.parent_symptom', 'medicine.disease.causes', 'medicine.diagnostic_test_signs.disease', 'medicine.disease.risk_factors']
2024-01-07 14:01:57 INFO     top3_ques rela: ['medicine.disease_cause.diseases', 'medicine.disease.symptoms', 'medicine.disease.causes', 'medicine.diagnostic_test_signs.disease', 'medicine.disease.includes_diseases', 'medicine.disease.parent_disease', 'medicine.disease.affiliated_diseases', 'medicine.vector_of_disease.disease']
2024-01-07 14:01:57 INFO     top3_ques rela: ['medicine.disease_cause.diseases', 'medicine.disease.causes', 'biology.plant_disease_cause.plant_disease_triangle', 'biology.plant_disease_triangle.plant_disease_cause', 'location.country.fips10_4', 'medicine.diagnostic_test_signs.disease', 'medicine.symptom.side_effect_of', 'medicine.symptom.includes_symptoms']
2024-01-07 14:01:57 INFO     top3_ques rela: ['medicine.disease_cause.diseases', 'medicine.disease.symptoms', 'location.country.fips10_4', 'medicine.diagnostic_test_signs.disease', 'medicine.disease.causes', 'medicine.disease.risk_factors', 'biology.plant_disease_cause.plant_disease_triangle', 'biology.plant_disease_triangle.plant_disease_cause']
2024-01-07 14:01:58 INFO     predicted_answer: None
2024-01-07 14:01:58 INFO     label: ['m.023s6n', 'm.0brgy', 'm.0f3kl', 'm.0cjf0', 'm.01j6t0', 'm.0lcdk']
2024-01-07 14:01:58 INFO     ================================================================
2024-01-07 14:01:58 INFO     consistent candidates number: 2
2024-01-07 14:01:58 INFO     em_score: 0.0
2024-01-07 14:01:58 INFO     correct: 0
2024-01-07 14:01:58 INFO     total: 1
2024-01-07 14:01:58 INFO     no_ans: 1
2024-01-07 14:01:58 INFO      
2024-01-07 14:01:58 INFO     ================================================================
2024-01-07 14:01:58 INFO     gene_exp: (OR (AND veterinary_medicine.disease_common_symptoms (JOIN veterinary_medicine.disease_common_symptoms.disease fip)) (AND medical.disease_caused_by_age (JOIN medical.disease_caused_by_age.age old)))
2024-01-07 14:03:14 INFO     all_iters: [('m.03nnmkj', 'm.0xy3jy'), ('m.03nnmkj', 'm.0lf7l0'), ('m.03nnmkj', 'm.0nn96kf'), ('m.03nnmkj', 'm.0nm93j'), ('m.03nnmkj', 'm.0gct2bf'), ('m.03nnmkj', 'm.0_656_t'), ('m.03nnmkj', 'm.0119mz_'), ('m.03nnmkj', 'm.0wcsyxd'), ('m.03nnmkj', 'm.010t_zq_'), ('m.03nnmkj', 'm.0387nv1'), ('m.03nnmkj', 'm.035d5jd'), ('m.03nnmkj', 'm.01j6xtn'), ('m.03nnmkj', 'm.034xx7w'), ('m.03nnmkj', 'm.0g93yd1'), ('m.03nnmkj', 'm.03_bxlm'), ('m.02pv7y8', 'm.0xy3jy'), ('m.02pv7y8', 'm.0lf7l0'), ('m.02pv7y8', 'm.0nn96kf'), ('m.02pv7y8', 'm.0nm93j'), ('m.02pv7y8', 'm.0gct2bf'), ('m.02pv7y8', 'm.0_656_t'), ('m.02pv7y8', 'm.0119mz_'), ('m.02pv7y8', 'm.0wcsyxd'), ('m.02pv7y8', 'm.010t_zq_'), ('m.02pv7y8', 'm.0387nv1'), ('m.02pv7y8', 'm.035d5jd'), ('m.02pv7y8', 'm.01j6xtn'), ('m.02pv7y8', 'm.034xx7w'), ('m.02pv7y8', 'm.0g93yd1'), ('m.02pv7y8', 'm.03_bxlm'), ('m.0g_0p_', 'm.0xy3jy'), ('m.0g_0p_', 'm.0lf7l0'), ('m.0g_0p_', 'm.0nn96kf'), ('m.0g_0p_', 'm.0nm93j'), ('m.0g_0p_', 'm.0gct2bf'), ('m.0g_0p_', 'm.0_656_t'), ('m.0g_0p_', 'm.0119mz_'), ('m.0g_0p_', 'm.0wcsyxd'), ('m.0g_0p_', 'm.010t_zq_'), ('m.0g_0p_', 'm.0387nv1'), ('m.0g_0p_', 'm.035d5jd'), ('m.0g_0p_', 'm.01j6xtn'), ('m.0g_0p_', 'm.034xx7w'), ('m.0g_0p_', 'm.0g93yd1'), ('m.0g_0p_', 'm.03_bxlm'), ('m.05yx0z', 'm.0xy3jy'), ('m.05yx0z', 'm.0lf7l0'), ('m.05yx0z', 'm.0nn96kf'), ('m.05yx0z', 'm.0nm93j'), ('m.05yx0z', 'm.0gct2bf'), ('m.05yx0z', 'm.0_656_t'), ('m.05yx0z', 'm.0119mz_'), ('m.05yx0z', 'm.0wcsyxd'), ('m.05yx0z', 'm.010t_zq_'), ('m.05yx0z', 'm.0387nv1'), ('m.05yx0z', 'm.035d5jd'), ('m.05yx0z', 'm.01j6xtn'), ('m.05yx0z', 'm.034xx7w'), ('m.05yx0z', 'm.0g93yd1'), ('m.05yx0z', 'm.03_bxlm'), ('m.071hyv', 'm.0xy3jy'), ('m.071hyv', 'm.0lf7l0'), ('m.071hyv', 'm.0nn96kf'), ('m.071hyv', 'm.0nm93j'), ('m.071hyv', 'm.0gct2bf'), ('m.071hyv', 'm.0_656_t'), ('m.071hyv', 'm.0119mz_'), ('m.071hyv', 'm.0wcsyxd'), ('m.071hyv', 'm.010t_zq_'), ('m.071hyv', 'm.0387nv1'), ('m.071hyv', 'm.035d5jd'), ('m.071hyv', 'm.01j6xtn'), ('m.071hyv', 'm.034xx7w'), ('m.071hyv', 'm.0g93yd1'), ('m.071hyv', 'm.03_bxlm'), ('m.074n2y', 'm.0xy3jy'), ('m.074n2y', 'm.0lf7l0'), ('m.074n2y', 'm.0nn96kf'), ('m.074n2y', 'm.0nm93j'), ('m.074n2y', 'm.0gct2bf'), ('m.074n2y', 'm.0_656_t'), ('m.074n2y', 'm.0119mz_'), ('m.074n2y', 'm.0wcsyxd'), ('m.074n2y', 'm.010t_zq_'), ('m.074n2y', 'm.0387nv1'), ('m.074n2y', 'm.035d5jd'), ('m.074n2y', 'm.01j6xtn'), ('m.074n2y', 'm.034xx7w'), ('m.074n2y', 'm.0g93yd1'), ('m.074n2y', 'm.03_bxlm'), ('m.04g1ktv', 'm.0xy3jy'), ('m.04g1ktv', 'm.0lf7l0'), ('m.04g1ktv', 'm.0nn96kf'), ('m.04g1ktv', 'm.0nm93j'), ('m.04g1ktv', 'm.0gct2bf'), ('m.04g1ktv', 'm.0_656_t'), ('m.04g1ktv', 'm.0119mz_'), ('m.04g1ktv', 'm.0wcsyxd'), ('m.04g1ktv', 'm.010t_zq_'), ('m.04g1ktv', 'm.0387nv1'), ('m.04g1ktv', 'm.035d5jd'), ('m.04g1ktv', 'm.01j6xtn'), ('m.04g1ktv', 'm.034xx7w'), ('m.04g1ktv', 'm.0g93yd1'), ('m.04g1ktv', 'm.03_bxlm'), ('m.0gttrfn', 'm.0xy3jy'), ('m.0gttrfn', 'm.0lf7l0'), ('m.0gttrfn', 'm.0nn96kf'), ('m.0gttrfn', 'm.0nm93j'), ('m.0gttrfn', 'm.0gct2bf'), ('m.0gttrfn', 'm.0_656_t'), ('m.0gttrfn', 'm.0119mz_'), ('m.0gttrfn', 'm.0wcsyxd'), ('m.0gttrfn', 'm.010t_zq_'), ('m.0gttrfn', 'm.0387nv1'), ('m.0gttrfn', 'm.035d5jd'), ('m.0gttrfn', 'm.01j6xtn'), ('m.0gttrfn', 'm.034xx7w'), ('m.0gttrfn', 'm.0g93yd1'), ('m.0gttrfn', 'm.03_bxlm'), ('m.04dvxh', 'm.0xy3jy'), ('m.04dvxh', 'm.0lf7l0'), ('m.04dvxh', 'm.0nn96kf'), ('m.04dvxh', 'm.0nm93j'), ('m.04dvxh', 'm.0gct2bf'), ('m.04dvxh', 'm.0_656_t'), ('m.04dvxh', 'm.0119mz_'), ('m.04dvxh', 'm.0wcsyxd'), ('m.04dvxh', 'm.010t_zq_'), ('m.04dvxh', 'm.0387nv1'), ('m.04dvxh', 'm.035d5jd'), ('m.04dvxh', 'm.01j6xtn'), ('m.04dvxh', 'm.034xx7w'), ('m.04dvxh', 'm.0g93yd1'), ('m.04dvxh', 'm.03_bxlm'), ('m.03c5gry', 'm.0xy3jy'), ('m.03c5gry', 'm.0lf7l0'), ('m.03c5gry', 'm.0nn96kf'), ('m.03c5gry', 'm.0nm93j'), ('m.03c5gry', 'm.0gct2bf'), ('m.03c5gry', 'm.0_656_t'), ('m.03c5gry', 'm.0119mz_'), ('m.03c5gry', 'm.0wcsyxd'), ('m.03c5gry', 'm.010t_zq_'), ('m.03c5gry', 'm.0387nv1'), ('m.03c5gry', 'm.035d5jd'), ('m.03c5gry', 'm.01j6xtn'), ('m.03c5gry', 'm.034xx7w'), ('m.03c5gry', 'm.0g93yd1'), ('m.03c5gry', 'm.03_bxlm'), ('m.051wrn1', 'm.0xy3jy'), ('m.051wrn1', 'm.0lf7l0'), ('m.051wrn1', 'm.0nn96kf'), ('m.051wrn1', 'm.0nm93j'), ('m.051wrn1', 'm.0gct2bf'), ('m.051wrn1', 'm.0_656_t'), ('m.051wrn1', 'm.0119mz_'), ('m.051wrn1', 'm.0wcsyxd'), ('m.051wrn1', 'm.010t_zq_'), ('m.051wrn1', 'm.0387nv1'), ('m.051wrn1', 'm.035d5jd'), ('m.051wrn1', 'm.01j6xtn'), ('m.051wrn1', 'm.034xx7w'), ('m.051wrn1', 'm.0g93yd1'), ('m.051wrn1', 'm.03_bxlm'), ('m.0j67b4h', 'm.0xy3jy'), ('m.0j67b4h', 'm.0lf7l0'), ('m.0j67b4h', 'm.0nn96kf'), ('m.0j67b4h', 'm.0nm93j'), ('m.0j67b4h', 'm.0gct2bf'), ('m.0j67b4h', 'm.0_656_t'), ('m.0j67b4h', 'm.0119mz_'), ('m.0j67b4h', 'm.0wcsyxd'), ('m.0j67b4h', 'm.010t_zq_'), ('m.0j67b4h', 'm.0387nv1'), ('m.0j67b4h', 'm.035d5jd'), ('m.0j67b4h', 'm.01j6xtn'), ('m.0j67b4h', 'm.034xx7w'), ('m.0j67b4h', 'm.0g93yd1'), ('m.0j67b4h', 'm.03_bxlm'), ('m.03nnmkj', 'm.0xy3jy'), ('m.03nnmkj', 'm.0lf7l0'), ('m.03nnmkj', 'm.0nn96kf'), ('m.03nnmkj', 'm.0nm93j'), ('m.03nnmkj', 'm.0gct2bf'), ('m.03nnmkj', 'm.0_656_t'), ('m.03nnmkj', 'm.0119mz_'), ('m.03nnmkj', 'm.0wcsyxd'), ('m.03nnmkj', 'm.010t_zq_'), ('m.03nnmkj', 'm.0387nv1'), ('m.03nnmkj', 'm.035d5jd'), ('m.03nnmkj', 'm.01j6xtn'), ('m.03nnmkj', 'm.034xx7w'), ('m.03nnmkj', 'm.0g93yd1'), ('m.03nnmkj', 'm.03_bxlm'), ('m.02pv7y8', 'm.0xy3jy'), ('m.02pv7y8', 'm.0lf7l0'), ('m.02pv7y8', 'm.0nn96kf'), ('m.02pv7y8', 'm.0nm93j'), ('m.02pv7y8', 'm.0gct2bf'), ('m.02pv7y8', 'm.0_656_t'), ('m.02pv7y8', 'm.0119mz_'), ('m.02pv7y8', 'm.0wcsyxd'), ('m.02pv7y8', 'm.010t_zq_'), ('m.02pv7y8', 'm.0387nv1'), ('m.02pv7y8', 'm.035d5jd'), ('m.02pv7y8', 'm.01j6xtn'), ('m.02pv7y8', 'm.034xx7w'), ('m.02pv7y8', 'm.0g93yd1'), ('m.02pv7y8', 'm.03_bxlm'), ('m.0g_0p_', 'm.0xy3jy'), ('m.0g_0p_', 'm.0lf7l0'), ('m.0g_0p_', 'm.0nn96kf'), ('m.0g_0p_', 'm.0nm93j'), ('m.0g_0p_', 'm.0gct2bf'), ('m.0g_0p_', 'm.0_656_t'), ('m.0g_0p_', 'm.0119mz_'), ('m.0g_0p_', 'm.0wcsyxd'), ('m.0g_0p_', 'm.010t_zq_'), ('m.0g_0p_', 'm.0387nv1'), ('m.0g_0p_', 'm.035d5jd'), ('m.0g_0p_', 'm.01j6xtn'), ('m.0g_0p_', 'm.034xx7w'), ('m.0g_0p_', 'm.0g93yd1'), ('m.0g_0p_', 'm.03_bxlm')]
2024-01-07 14:03:14 INFO     mid_iters: ('m.03nnmkj', 'm.0xy3jy')
2024-01-07 14:03:33 INFO     top3_ques rela: []
2024-01-07 14:03:33 INFO     top3_ques rela: []
2024-01-07 14:03:33 INFO     mid_iters: ('m.03nnmkj', 'm.0lf7l0')
2024-01-07 14:03:54 INFO     top3_ques rela: []
2024-01-07 14:03:54 INFO     top3_ques rela: []
2024-01-07 14:03:54 INFO     mid_iters: ('m.03nnmkj', 'm.0nn96kf')
2024-01-07 14:04:07 INFO     top3_ques rela: []
2024-01-07 14:04:07 INFO     top3_ques rela: []
2024-01-07 14:04:07 INFO     mid_iters: ('m.03nnmkj', 'm.0nm93j')
2024-01-07 14:04:25 INFO     top3_ques rela: []
2024-01-07 14:04:25 INFO     top3_ques rela: []
2024-01-07 14:04:25 INFO     mid_iters: ('m.03nnmkj', 'm.0gct2bf')
2024-01-07 14:04:35 INFO     top3_ques rela: []
2024-01-07 14:04:35 INFO     top3_ques rela: []
2024-01-07 14:04:35 INFO     mid_iters: ('m.03nnmkj', 'm.0_656_t')
2024-01-07 14:04:45 INFO     top3_ques rela: []
2024-01-07 14:04:45 INFO     top3_ques rela: []
2024-01-07 14:04:45 INFO     mid_iters: ('m.03nnmkj', 'm.0119mz_')
2024-01-07 14:04:57 INFO     top3_ques rela: []
2024-01-07 14:04:57 INFO     top3_ques rela: []
2024-01-07 14:04:57 INFO     mid_iters: ('m.03nnmkj', 'm.0wcsyxd')
2024-01-07 14:05:10 INFO     top3_ques rela: []
2024-01-07 14:05:10 INFO     top3_ques rela: []
2024-01-07 14:05:10 INFO     mid_iters: ('m.03nnmkj', 'm.010t_zq_')
2024-01-07 14:05:25 INFO     top3_ques rela: []
2024-01-07 14:05:25 INFO     top3_ques rela: []
2024-01-07 14:05:25 INFO     mid_iters: ('m.03nnmkj', 'm.0387nv1')
2024-01-07 14:05:50 INFO     top3_ques rela: []
2024-01-07 14:05:50 INFO     top3_ques rela: ['location.country.fips10_4']
2024-01-07 14:05:50 INFO     mid_iters: ('m.03nnmkj', 'm.035d5jd')
2024-01-07 14:06:02 INFO     top3_ques rela: []
2024-01-07 14:06:02 INFO     top3_ques rela: []
2024-01-07 14:06:02 INFO     mid_iters: ('m.03nnmkj', 'm.01j6xtn')
2024-01-07 14:06:17 INFO     top3_ques rela: []
2024-01-07 14:06:17 INFO     top3_ques rela: []
2024-01-07 14:06:17 INFO     mid_iters: ('m.03nnmkj', 'm.034xx7w')
2024-01-07 14:06:30 INFO     top3_ques rela: []
2024-01-07 14:06:30 INFO     top3_ques rela: []
2024-01-07 14:06:30 INFO     mid_iters: ('m.03nnmkj', 'm.0g93yd1')
2024-01-07 14:06:45 INFO     top3_ques rela: []
2024-01-07 14:06:45 INFO     top3_ques rela: []
2024-01-07 14:06:45 INFO     mid_iters: ('m.03nnmkj', 'm.03_bxlm')
2024-01-07 14:06:52 INFO     top3_ques rela: []
2024-01-07 14:06:52 INFO     top3_ques rela: []
2024-01-07 14:06:52 INFO     mid_iters: ('m.02pv7y8', 'm.0xy3jy')
2024-01-07 14:06:52 INFO     top3_ques rela: []
2024-01-07 14:06:52 INFO     top3_ques rela: []
2024-01-07 14:06:52 INFO     mid_iters: ('m.02pv7y8', 'm.0lf7l0')
2024-01-07 14:06:52 INFO     top3_ques rela: []
2024-01-07 14:06:52 INFO     top3_ques rela: []
2024-01-07 14:06:52 INFO     mid_iters: ('m.02pv7y8', 'm.0nn96kf')
2024-01-07 14:06:52 INFO     top3_ques rela: []
2024-01-07 14:06:52 INFO     top3_ques rela: []
2024-01-07 14:06:52 INFO     mid_iters: ('m.02pv7y8', 'm.0nm93j')
2024-01-07 14:06:52 INFO     top3_ques rela: []
2024-01-07 14:06:52 INFO     top3_ques rela: []
2024-01-07 14:06:52 INFO     mid_iters: ('m.02pv7y8', 'm.0gct2bf')
2024-01-07 14:06:53 INFO     top3_ques rela: []
2024-01-07 14:06:53 INFO     top3_ques rela: []
2024-01-07 14:06:53 INFO     mid_iters: ('m.02pv7y8', 'm.0_656_t')
2024-01-07 14:06:53 INFO     top3_ques rela: []
2024-01-07 14:06:53 INFO     top3_ques rela: []
2024-01-07 14:06:53 INFO     mid_iters: ('m.02pv7y8', 'm.0119mz_')
2024-01-07 14:06:53 INFO     top3_ques rela: []
2024-01-07 14:06:53 INFO     top3_ques rela: []
2024-01-07 14:06:53 INFO     mid_iters: ('m.02pv7y8', 'm.0wcsyxd')
2024-01-07 14:06:53 INFO     top3_ques rela: []
2024-01-07 14:06:53 INFO     top3_ques rela: []
2024-01-07 14:06:53 INFO     mid_iters: ('m.02pv7y8', 'm.010t_zq_')
2024-01-07 14:06:53 INFO     top3_ques rela: []
2024-01-07 14:06:53 INFO     top3_ques rela: []
2024-01-07 14:06:53 INFO     mid_iters: ('m.02pv7y8', 'm.0387nv1')
2024-01-07 14:06:53 INFO     top3_ques rela: []
2024-01-07 14:06:54 INFO     top3_ques rela: ['location.country.fips10_4']
2024-01-07 14:06:54 INFO     mid_iters: ('m.02pv7y8', 'm.035d5jd')
2024-01-07 14:06:54 INFO     top3_ques rela: []
2024-01-07 14:06:54 INFO     top3_ques rela: []
2024-01-07 14:06:54 INFO     mid_iters: ('m.02pv7y8', 'm.01j6xtn')
2024-01-07 14:06:54 INFO     top3_ques rela: []
2024-01-07 14:06:54 INFO     top3_ques rela: []
2024-01-07 14:06:54 INFO     mid_iters: ('m.02pv7y8', 'm.034xx7w')
2024-01-07 14:06:54 INFO     top3_ques rela: []
2024-01-07 14:06:54 INFO     top3_ques rela: []
2024-01-07 14:06:54 INFO     mid_iters: ('m.02pv7y8', 'm.0g93yd1')
2024-01-07 14:06:54 INFO     top3_ques rela: []
2024-01-07 14:06:54 INFO     top3_ques rela: []
2024-01-07 14:06:54 INFO     mid_iters: ('m.02pv7y8', 'm.03_bxlm')
2024-01-07 14:06:54 INFO     top3_ques rela: []
2024-01-07 14:06:55 INFO     top3_ques rela: []
2024-01-07 14:06:55 INFO     mid_iters: ('m.0g_0p_', 'm.0xy3jy')
2024-01-07 14:06:55 INFO     top3_ques rela: []
2024-01-07 14:06:55 INFO     top3_ques rela: []
2024-01-07 14:06:55 INFO     mid_iters: ('m.0g_0p_', 'm.0lf7l0')
2024-01-07 14:06:55 INFO     top3_ques rela: []
2024-01-07 14:06:55 INFO     top3_ques rela: []
2024-01-07 14:06:55 INFO     mid_iters: ('m.0g_0p_', 'm.0nn96kf')
2024-01-07 14:06:55 INFO     top3_ques rela: []
2024-01-07 14:06:55 INFO     top3_ques rela: []
2024-01-07 14:06:55 INFO     mid_iters: ('m.0g_0p_', 'm.0nm93j')
2024-01-07 14:06:55 INFO     top3_ques rela: []
2024-01-07 14:06:55 INFO     top3_ques rela: []
2024-01-07 14:06:55 INFO     mid_iters: ('m.0g_0p_', 'm.0gct2bf')
2024-01-07 14:06:55 INFO     top3_ques rela: []
2024-01-07 14:06:55 INFO     top3_ques rela: []
2024-01-07 14:06:55 INFO     mid_iters: ('m.0g_0p_', 'm.0_656_t')
2024-01-07 14:06:56 INFO     top3_ques rela: []
2024-01-07 14:06:56 INFO     top3_ques rela: []
2024-01-07 14:06:56 INFO     mid_iters: ('m.0g_0p_', 'm.0119mz_')
2024-01-07 14:06:56 INFO     top3_ques rela: []
2024-01-07 14:06:56 INFO     top3_ques rela: []
2024-01-07 14:06:56 INFO     mid_iters: ('m.0g_0p_', 'm.0wcsyxd')
2024-01-07 14:06:56 INFO     top3_ques rela: []
2024-01-07 14:06:56 INFO     top3_ques rela: []
2024-01-07 14:06:56 INFO     mid_iters: ('m.0g_0p_', 'm.010t_zq_')
2024-01-07 14:06:56 INFO     top3_ques rela: []
2024-01-07 14:06:56 INFO     top3_ques rela: []
2024-01-07 14:06:56 INFO     mid_iters: ('m.0g_0p_', 'm.0387nv1')
2024-01-07 14:06:56 INFO     top3_ques rela: []
2024-01-07 14:06:56 INFO     top3_ques rela: ['location.country.fips10_4']
2024-01-07 14:06:56 INFO     mid_iters: ('m.0g_0p_', 'm.035d5jd')
2024-01-07 14:06:56 INFO     top3_ques rela: []
2024-01-07 14:06:57 INFO     top3_ques rela: []
2024-01-07 14:06:57 INFO     mid_iters: ('m.0g_0p_', 'm.01j6xtn')
2024-01-07 14:06:57 INFO     top3_ques rela: []
2024-01-07 14:06:57 INFO     top3_ques rela: []
2024-01-07 14:06:57 INFO     mid_iters: ('m.0g_0p_', 'm.034xx7w')
2024-01-07 14:06:57 INFO     top3_ques rela: []
2024-01-07 14:06:57 INFO     top3_ques rela: []
2024-01-07 14:06:57 INFO     mid_iters: ('m.0g_0p_', 'm.0g93yd1')
2024-01-07 14:06:57 INFO     top3_ques rela: []
2024-01-07 14:06:57 INFO     top3_ques rela: []
2024-01-07 14:06:57 INFO     mid_iters: ('m.0g_0p_', 'm.03_bxlm')
2024-01-07 14:06:57 INFO     top3_ques rela: []
2024-01-07 14:06:57 INFO     top3_ques rela: []
2024-01-07 14:06:57 INFO     mid_iters: ('m.05yx0z', 'm.0xy3jy')
2024-01-07 14:06:57 INFO     top3_ques rela: []
2024-01-07 14:06:58 INFO     top3_ques rela: []
2024-01-07 14:06:58 INFO     mid_iters: ('m.05yx0z', 'm.0lf7l0')
2024-01-07 14:06:58 INFO     top3_ques rela: []
2024-01-07 14:06:58 INFO     top3_ques rela: []
2024-01-07 14:06:58 INFO     mid_iters: ('m.05yx0z', 'm.0nn96kf')
2024-01-07 14:06:58 INFO     top3_ques rela: []
2024-01-07 14:06:58 INFO     top3_ques rela: []
2024-01-07 14:06:58 INFO     mid_iters: ('m.05yx0z', 'm.0nm93j')
2024-01-07 14:06:58 INFO     top3_ques rela: []
2024-01-07 14:06:58 INFO     top3_ques rela: []
2024-01-07 14:06:58 INFO     mid_iters: ('m.05yx0z', 'm.0gct2bf')
2024-01-07 14:06:58 INFO     top3_ques rela: []
2024-01-07 14:06:58 INFO     top3_ques rela: []
2024-01-07 14:06:58 INFO     mid_iters: ('m.05yx0z', 'm.0_656_t')
2024-01-07 14:06:58 INFO     top3_ques rela: []
2024-01-07 14:06:58 INFO     top3_ques rela: []
2024-01-07 14:06:58 INFO     mid_iters: ('m.05yx0z', 'm.0119mz_')
2024-01-07 14:06:59 INFO     top3_ques rela: []
2024-01-07 14:06:59 INFO     top3_ques rela: []
2024-01-07 14:06:59 INFO     mid_iters: ('m.05yx0z', 'm.0wcsyxd')
2024-01-07 14:06:59 INFO     top3_ques rela: []
2024-01-07 14:06:59 INFO     top3_ques rela: []
2024-01-07 14:06:59 INFO     mid_iters: ('m.05yx0z', 'm.010t_zq_')
2024-01-07 14:06:59 INFO     top3_ques rela: []
2024-01-07 14:06:59 INFO     top3_ques rela: []
2024-01-07 14:06:59 INFO     mid_iters: ('m.05yx0z', 'm.0387nv1')
2024-01-07 14:06:59 INFO     top3_ques rela: []
2024-01-07 14:06:59 INFO     top3_ques rela: ['location.country.fips10_4']
2024-01-07 14:06:59 INFO     mid_iters: ('m.05yx0z', 'm.035d5jd')
2024-01-07 14:06:59 INFO     top3_ques rela: []
2024-01-07 14:06:59 INFO     top3_ques rela: []
2024-01-07 14:06:59 INFO     mid_iters: ('m.05yx0z', 'm.01j6xtn')
2024-01-07 14:07:00 INFO     top3_ques rela: []
2024-01-07 14:07:00 INFO     top3_ques rela: []
2024-01-07 14:07:00 INFO     mid_iters: ('m.05yx0z', 'm.034xx7w')
2024-01-07 14:07:00 INFO     top3_ques rela: []
2024-01-07 14:07:00 INFO     top3_ques rela: []
2024-01-07 14:07:00 INFO     mid_iters: ('m.05yx0z', 'm.0g93yd1')
2024-01-07 14:07:00 INFO     top3_ques rela: []
2024-01-07 14:07:00 INFO     top3_ques rela: []
2024-01-07 14:07:00 INFO     mid_iters: ('m.05yx0z', 'm.03_bxlm')
2024-01-07 14:07:00 INFO     top3_ques rela: []
2024-01-07 14:07:00 INFO     top3_ques rela: []
2024-01-07 14:07:00 INFO     mid_iters: ('m.071hyv', 'm.0xy3jy')
2024-01-07 14:07:00 INFO     top3_ques rela: []
2024-01-07 14:07:00 INFO     top3_ques rela: []
2024-01-07 14:07:00 INFO     mid_iters: ('m.071hyv', 'm.0lf7l0')
2024-01-07 14:07:00 INFO     top3_ques rela: []
2024-01-07 14:07:01 INFO     top3_ques rela: []
2024-01-07 14:07:01 INFO     mid_iters: ('m.071hyv', 'm.0nn96kf')
2024-01-07 14:07:01 INFO     top3_ques rela: []
2024-01-07 14:07:01 INFO     top3_ques rela: []
2024-01-07 14:07:01 INFO     mid_iters: ('m.071hyv', 'm.0nm93j')
2024-01-07 14:07:01 INFO     top3_ques rela: []
2024-01-07 14:07:01 INFO     top3_ques rela: []
2024-01-07 14:07:01 INFO     mid_iters: ('m.071hyv', 'm.0gct2bf')
2024-01-07 14:07:01 INFO     top3_ques rela: []
2024-01-07 14:07:01 INFO     top3_ques rela: []
2024-01-07 14:07:01 INFO     mid_iters: ('m.071hyv', 'm.0_656_t')
2024-01-07 14:07:01 INFO     top3_ques rela: []
2024-01-07 14:07:01 INFO     top3_ques rela: []
2024-01-07 14:07:01 INFO     mid_iters: ('m.071hyv', 'm.0119mz_')
2024-01-07 14:07:01 INFO     top3_ques rela: []
2024-01-07 14:07:01 INFO     top3_ques rela: []
2024-01-07 14:07:01 INFO     mid_iters: ('m.071hyv', 'm.0wcsyxd')
2024-01-07 14:07:02 INFO     top3_ques rela: []
2024-01-07 14:07:02 INFO     top3_ques rela: []
2024-01-07 14:07:02 INFO     mid_iters: ('m.071hyv', 'm.010t_zq_')
2024-01-07 14:07:02 INFO     top3_ques rela: []
2024-01-07 14:07:02 INFO     top3_ques rela: []
2024-01-07 14:07:02 INFO     mid_iters: ('m.071hyv', 'm.0387nv1')
2024-01-07 14:07:02 INFO     top3_ques rela: []
2024-01-07 14:07:02 INFO     top3_ques rela: ['location.country.fips10_4']
2024-01-07 14:07:02 INFO     mid_iters: ('m.071hyv', 'm.035d5jd')
2024-01-07 14:07:02 INFO     top3_ques rela: []
2024-01-07 14:07:02 INFO     top3_ques rela: []
2024-01-07 14:07:02 INFO     mid_iters: ('m.071hyv', 'm.01j6xtn')
2024-01-07 14:07:02 INFO     top3_ques rela: []
2024-01-07 14:07:02 INFO     top3_ques rela: []
2024-01-07 14:07:02 INFO     mid_iters: ('m.071hyv', 'm.034xx7w')
2024-01-07 14:07:03 INFO     top3_ques rela: []
2024-01-07 14:07:03 INFO     top3_ques rela: []
2024-01-07 14:07:03 INFO     mid_iters: ('m.071hyv', 'm.0g93yd1')
2024-01-07 14:07:03 INFO     top3_ques rela: []
2024-01-07 14:07:03 INFO     top3_ques rela: []
2024-01-07 14:07:03 INFO     mid_iters: ('m.071hyv', 'm.03_bxlm')
2024-01-07 14:07:03 INFO     top3_ques rela: []
2024-01-07 14:07:03 INFO     top3_ques rela: []
2024-01-07 14:07:03 INFO     mid_iters: ('m.074n2y', 'm.0xy3jy')
2024-01-07 14:07:03 INFO     top3_ques rela: []
2024-01-07 14:07:03 INFO     top3_ques rela: []
2024-01-07 14:07:03 INFO     mid_iters: ('m.074n2y', 'm.0lf7l0')
2024-01-07 14:07:03 INFO     top3_ques rela: []
2024-01-07 14:07:03 INFO     top3_ques rela: []
2024-01-07 14:07:03 INFO     mid_iters: ('m.074n2y', 'm.0nn96kf')
2024-01-07 14:07:03 INFO     top3_ques rela: []
2024-01-07 14:07:04 INFO     top3_ques rela: []
2024-01-07 14:07:04 INFO     mid_iters: ('m.074n2y', 'm.0nm93j')
2024-01-07 14:07:04 INFO     top3_ques rela: []
2024-01-07 14:07:04 INFO     top3_ques rela: []
2024-01-07 14:07:04 INFO     mid_iters: ('m.074n2y', 'm.0gct2bf')
2024-01-07 14:07:04 INFO     top3_ques rela: []
2024-01-07 14:07:04 INFO     top3_ques rela: []
2024-01-07 14:07:04 INFO     mid_iters: ('m.074n2y', 'm.0_656_t')
2024-01-07 14:07:04 INFO     top3_ques rela: []
2024-01-07 14:07:04 INFO     top3_ques rela: []
2024-01-07 14:07:04 INFO     mid_iters: ('m.074n2y', 'm.0119mz_')
2024-01-07 14:07:04 INFO     top3_ques rela: []
2024-01-07 14:07:04 INFO     top3_ques rela: []
2024-01-07 14:07:04 INFO     mid_iters: ('m.074n2y', 'm.0wcsyxd')
2024-01-07 14:07:04 INFO     top3_ques rela: []
2024-01-07 14:07:05 INFO     top3_ques rela: []
2024-01-07 14:07:05 INFO     mid_iters: ('m.074n2y', 'm.010t_zq_')
2024-01-07 14:07:05 INFO     top3_ques rela: []
2024-01-07 14:07:05 INFO     top3_ques rela: []
2024-01-07 14:07:05 INFO     mid_iters: ('m.074n2y', 'm.0387nv1')
2024-01-07 14:07:05 INFO     top3_ques rela: []
2024-01-07 14:07:05 INFO     top3_ques rela: ['location.country.fips10_4']
2024-01-07 14:07:05 INFO     mid_iters: ('m.074n2y', 'm.035d5jd')
2024-01-07 14:07:05 INFO     top3_ques rela: []
2024-01-07 14:07:05 INFO     top3_ques rela: []
2024-01-07 14:07:05 INFO     mid_iters: ('m.074n2y', 'm.01j6xtn')
2024-01-07 14:07:05 INFO     top3_ques rela: []
2024-01-07 14:07:05 INFO     top3_ques rela: []
2024-01-07 14:07:05 INFO     mid_iters: ('m.074n2y', 'm.034xx7w')
2024-01-07 14:07:05 INFO     top3_ques rela: []
2024-01-07 14:07:05 INFO     top3_ques rela: []
2024-01-07 14:07:05 INFO     mid_iters: ('m.074n2y', 'm.0g93yd1')
2024-01-07 14:07:06 INFO     top3_ques rela: []
2024-01-07 14:07:06 INFO     top3_ques rela: []
2024-01-07 14:07:06 INFO     mid_iters: ('m.074n2y', 'm.03_bxlm')
2024-01-07 14:07:06 INFO     top3_ques rela: []
2024-01-07 14:07:06 INFO     top3_ques rela: []
2024-01-07 14:07:06 INFO     mid_iters: ('m.04g1ktv', 'm.0xy3jy')
2024-01-07 14:07:06 INFO     top3_ques rela: []
2024-01-07 14:07:06 INFO     top3_ques rela: []
2024-01-07 14:07:06 INFO     mid_iters: ('m.04g1ktv', 'm.0lf7l0')
2024-01-07 14:07:06 INFO     top3_ques rela: []
2024-01-07 14:07:06 INFO     top3_ques rela: []
2024-01-07 14:07:06 INFO     mid_iters: ('m.04g1ktv', 'm.0nn96kf')
2024-01-07 14:07:06 INFO     top3_ques rela: []
2024-01-07 14:07:06 INFO     top3_ques rela: []
2024-01-07 14:07:06 INFO     mid_iters: ('m.04g1ktv', 'm.0nm93j')
2024-01-07 14:07:07 INFO     top3_ques rela: []
2024-01-07 14:07:07 INFO     top3_ques rela: []
2024-01-07 14:07:07 INFO     mid_iters: ('m.04g1ktv', 'm.0gct2bf')
2024-01-07 14:07:07 INFO     top3_ques rela: []
2024-01-07 14:07:07 INFO     top3_ques rela: []
2024-01-07 14:07:07 INFO     mid_iters: ('m.04g1ktv', 'm.0_656_t')
2024-01-07 14:07:07 INFO     top3_ques rela: []
2024-01-07 14:07:07 INFO     top3_ques rela: []
2024-01-07 14:07:07 INFO     mid_iters: ('m.04g1ktv', 'm.0119mz_')
2024-01-07 14:07:07 INFO     top3_ques rela: []
2024-01-07 14:07:07 INFO     top3_ques rela: []
2024-01-07 14:07:07 INFO     mid_iters: ('m.04g1ktv', 'm.0wcsyxd')
2024-01-07 14:07:07 INFO     top3_ques rela: []
2024-01-07 14:07:07 INFO     top3_ques rela: []
2024-01-07 14:07:07 INFO     mid_iters: ('m.04g1ktv', 'm.010t_zq_')
2024-01-07 14:07:07 INFO     top3_ques rela: []
2024-01-07 14:07:08 INFO     top3_ques rela: []
2024-01-07 14:07:08 INFO     mid_iters: ('m.04g1ktv', 'm.0387nv1')
2024-01-07 14:07:08 INFO     top3_ques rela: []
2024-01-07 14:07:08 INFO     top3_ques rela: ['location.country.fips10_4']
2024-01-07 14:07:08 INFO     mid_iters: ('m.04g1ktv', 'm.035d5jd')
2024-01-07 14:07:08 INFO     top3_ques rela: []
2024-01-07 14:07:08 INFO     top3_ques rela: []
2024-01-07 14:07:08 INFO     mid_iters: ('m.04g1ktv', 'm.01j6xtn')
2024-01-07 14:07:08 INFO     top3_ques rela: []
2024-01-07 14:07:08 INFO     top3_ques rela: []
2024-01-07 14:07:08 INFO     mid_iters: ('m.04g1ktv', 'm.034xx7w')
2024-01-07 14:07:08 INFO     top3_ques rela: []
2024-01-07 14:07:08 INFO     top3_ques rela: []
2024-01-07 14:07:08 INFO     mid_iters: ('m.04g1ktv', 'm.0g93yd1')
2024-01-07 14:07:08 INFO     top3_ques rela: []
2024-01-07 14:07:08 INFO     top3_ques rela: []
2024-01-07 14:07:08 INFO     mid_iters: ('m.04g1ktv', 'm.03_bxlm')
2024-01-07 14:07:09 INFO     top3_ques rela: []
2024-01-07 14:07:09 INFO     top3_ques rela: []
2024-01-07 14:07:09 INFO     mid_iters: ('m.0gttrfn', 'm.0xy3jy')
2024-01-07 14:07:09 INFO     top3_ques rela: []
2024-01-07 14:07:09 INFO     top3_ques rela: []
2024-01-07 14:07:09 INFO     mid_iters: ('m.0gttrfn', 'm.0lf7l0')
2024-01-07 14:07:09 INFO     top3_ques rela: []
2024-01-07 14:07:09 INFO     top3_ques rela: []
2024-01-07 14:07:09 INFO     mid_iters: ('m.0gttrfn', 'm.0nn96kf')
2024-01-07 14:07:09 INFO     top3_ques rela: []
2024-01-07 14:07:09 INFO     top3_ques rela: []
2024-01-07 14:07:09 INFO     mid_iters: ('m.0gttrfn', 'm.0nm93j')
2024-01-07 14:07:09 INFO     top3_ques rela: []
2024-01-07 14:07:09 INFO     top3_ques rela: []
2024-01-07 14:07:09 INFO     mid_iters: ('m.0gttrfn', 'm.0gct2bf')
2024-01-07 14:07:10 INFO     top3_ques rela: []
2024-01-07 14:07:10 INFO     top3_ques rela: []
2024-01-07 14:07:10 INFO     mid_iters: ('m.0gttrfn', 'm.0_656_t')
2024-01-07 14:07:10 INFO     top3_ques rela: []
2024-01-07 14:07:10 INFO     top3_ques rela: []
2024-01-07 14:07:10 INFO     mid_iters: ('m.0gttrfn', 'm.0119mz_')
2024-01-07 14:07:10 INFO     top3_ques rela: []
2024-01-07 14:07:10 INFO     top3_ques rela: []
2024-01-07 14:07:10 INFO     mid_iters: ('m.0gttrfn', 'm.0wcsyxd')
2024-01-07 14:07:10 INFO     top3_ques rela: []
2024-01-07 14:07:10 INFO     top3_ques rela: []
2024-01-07 14:07:10 INFO     mid_iters: ('m.0gttrfn', 'm.010t_zq_')
2024-01-07 14:07:10 INFO     top3_ques rela: []
2024-01-07 14:07:10 INFO     top3_ques rela: []
2024-01-07 14:07:10 INFO     mid_iters: ('m.0gttrfn', 'm.0387nv1')
2024-01-07 14:07:10 INFO     top3_ques rela: []
2024-01-07 14:07:11 INFO     top3_ques rela: ['location.country.fips10_4']
2024-01-07 14:07:11 INFO     mid_iters: ('m.0gttrfn', 'm.035d5jd')
2024-01-07 14:07:11 INFO     top3_ques rela: []
2024-01-07 14:07:11 INFO     top3_ques rela: []
2024-01-07 14:07:11 INFO     mid_iters: ('m.0gttrfn', 'm.01j6xtn')
2024-01-07 14:07:11 INFO     top3_ques rela: []
2024-01-07 14:07:11 INFO     top3_ques rela: []
2024-01-07 14:07:11 INFO     mid_iters: ('m.0gttrfn', 'm.034xx7w')
2024-01-07 14:07:11 INFO     top3_ques rela: []
2024-01-07 14:07:11 INFO     top3_ques rela: []
2024-01-07 14:07:11 INFO     mid_iters: ('m.0gttrfn', 'm.0g93yd1')
2024-01-07 14:07:11 INFO     top3_ques rela: []
2024-01-07 14:07:11 INFO     top3_ques rela: []
2024-01-07 14:07:11 INFO     mid_iters: ('m.0gttrfn', 'm.03_bxlm')
2024-01-07 14:07:11 INFO     top3_ques rela: []
2024-01-07 14:07:12 INFO     top3_ques rela: []
2024-01-07 14:07:12 INFO     mid_iters: ('m.04dvxh', 'm.0xy3jy')
2024-01-07 14:07:12 INFO     top3_ques rela: []
2024-01-07 14:07:12 INFO     top3_ques rela: []
2024-01-07 14:07:12 INFO     mid_iters: ('m.04dvxh', 'm.0lf7l0')
2024-01-07 14:07:12 INFO     top3_ques rela: []
2024-01-07 14:07:12 INFO     top3_ques rela: []
2024-01-07 14:07:12 INFO     mid_iters: ('m.04dvxh', 'm.0nn96kf')
2024-01-07 14:07:12 INFO     top3_ques rela: []
2024-01-07 14:07:12 INFO     top3_ques rela: []
2024-01-07 14:07:12 INFO     mid_iters: ('m.04dvxh', 'm.0nm93j')
2024-01-07 14:07:12 INFO     top3_ques rela: []
2024-01-07 14:07:12 INFO     top3_ques rela: []
2024-01-07 14:07:12 INFO     mid_iters: ('m.04dvxh', 'm.0gct2bf')
2024-01-07 14:07:12 INFO     top3_ques rela: []
2024-01-07 14:07:12 INFO     top3_ques rela: []
2024-01-07 14:07:12 INFO     mid_iters: ('m.04dvxh', 'm.0_656_t')
2024-01-07 14:07:13 INFO     top3_ques rela: []
2024-01-07 14:07:13 INFO     top3_ques rela: []
2024-01-07 14:07:13 INFO     mid_iters: ('m.04dvxh', 'm.0119mz_')
2024-01-07 14:07:13 INFO     top3_ques rela: []
2024-01-07 14:07:13 INFO     top3_ques rela: []
2024-01-07 14:07:13 INFO     mid_iters: ('m.04dvxh', 'm.0wcsyxd')
2024-01-07 14:07:13 INFO     top3_ques rela: []
2024-01-07 14:07:13 INFO     top3_ques rela: []
2024-01-07 14:07:13 INFO     mid_iters: ('m.04dvxh', 'm.010t_zq_')
2024-01-07 14:07:13 INFO     top3_ques rela: []
2024-01-07 14:07:13 INFO     top3_ques rela: []
2024-01-07 14:07:13 INFO     mid_iters: ('m.04dvxh', 'm.0387nv1')
2024-01-07 14:07:13 INFO     top3_ques rela: []
2024-01-07 14:07:13 INFO     top3_ques rela: ['location.country.fips10_4']
2024-01-07 14:07:13 INFO     mid_iters: ('m.04dvxh', 'm.035d5jd')
2024-01-07 14:07:13 INFO     top3_ques rela: []
2024-01-07 14:07:14 INFO     top3_ques rela: []
2024-01-07 14:07:14 INFO     mid_iters: ('m.04dvxh', 'm.01j6xtn')
2024-01-07 14:07:14 INFO     top3_ques rela: []
2024-01-07 14:07:14 INFO     top3_ques rela: []
2024-01-07 14:07:14 INFO     mid_iters: ('m.04dvxh', 'm.034xx7w')
2024-01-07 14:07:14 INFO     top3_ques rela: []
2024-01-07 14:07:14 INFO     top3_ques rela: []
2024-01-07 14:07:14 INFO     mid_iters: ('m.04dvxh', 'm.0g93yd1')
2024-01-07 14:07:14 INFO     top3_ques rela: []
2024-01-07 14:07:14 INFO     top3_ques rela: []
2024-01-07 14:07:14 INFO     mid_iters: ('m.04dvxh', 'm.03_bxlm')
2024-01-07 14:07:14 INFO     top3_ques rela: []
2024-01-07 14:07:14 INFO     top3_ques rela: []
2024-01-07 14:07:14 INFO     mid_iters: ('m.03c5gry', 'm.0xy3jy')
2024-01-07 14:07:14 INFO     top3_ques rela: []
2024-01-07 14:07:15 INFO     top3_ques rela: []
2024-01-07 14:07:15 INFO     mid_iters: ('m.03c5gry', 'm.0lf7l0')
2024-01-07 14:07:15 INFO     top3_ques rela: []
2024-01-07 14:07:15 INFO     top3_ques rela: []
2024-01-07 14:07:15 INFO     mid_iters: ('m.03c5gry', 'm.0nn96kf')
2024-01-07 14:07:15 INFO     top3_ques rela: []
2024-01-07 14:07:15 INFO     top3_ques rela: []
2024-01-07 14:07:15 INFO     mid_iters: ('m.03c5gry', 'm.0nm93j')
2024-01-07 14:07:15 INFO     top3_ques rela: []
2024-01-07 14:07:15 INFO     top3_ques rela: []
2024-01-07 14:07:15 INFO     mid_iters: ('m.03c5gry', 'm.0gct2bf')
2024-01-07 14:07:15 INFO     top3_ques rela: []
2024-01-07 14:07:15 INFO     top3_ques rela: []
2024-01-07 14:07:15 INFO     mid_iters: ('m.03c5gry', 'm.0_656_t')
2024-01-07 14:07:15 INFO     top3_ques rela: []
2024-01-07 14:07:15 INFO     top3_ques rela: []
2024-01-07 14:07:15 INFO     mid_iters: ('m.03c5gry', 'm.0119mz_')
2024-01-07 14:07:16 INFO     top3_ques rela: []
2024-01-07 14:07:16 INFO     top3_ques rela: []
2024-01-07 14:07:16 INFO     mid_iters: ('m.03c5gry', 'm.0wcsyxd')
2024-01-07 14:07:16 INFO     top3_ques rela: []
2024-01-07 14:07:16 INFO     top3_ques rela: []
2024-01-07 14:07:16 INFO     mid_iters: ('m.03c5gry', 'm.010t_zq_')
2024-01-07 14:07:16 INFO     top3_ques rela: []
2024-01-07 14:07:16 INFO     top3_ques rela: []
2024-01-07 14:07:16 INFO     mid_iters: ('m.03c5gry', 'm.0387nv1')
2024-01-07 14:07:16 INFO     top3_ques rela: []
2024-01-07 14:07:16 INFO     top3_ques rela: ['location.country.fips10_4']
2024-01-07 14:07:16 INFO     mid_iters: ('m.03c5gry', 'm.035d5jd')
2024-01-07 14:07:16 INFO     top3_ques rela: []
2024-01-07 14:07:16 INFO     top3_ques rela: []
2024-01-07 14:07:16 INFO     mid_iters: ('m.03c5gry', 'm.01j6xtn')
2024-01-07 14:07:16 INFO     top3_ques rela: []
2024-01-07 14:07:17 INFO     top3_ques rela: []
2024-01-07 14:07:17 INFO     mid_iters: ('m.03c5gry', 'm.034xx7w')
2024-01-07 14:07:17 INFO     top3_ques rela: []
2024-01-07 14:07:17 INFO     top3_ques rela: []
2024-01-07 14:07:17 INFO     mid_iters: ('m.03c5gry', 'm.0g93yd1')
2024-01-07 14:07:17 INFO     top3_ques rela: []
2024-01-07 14:07:17 INFO     top3_ques rela: []
2024-01-07 14:07:17 INFO     mid_iters: ('m.03c5gry', 'm.03_bxlm')
2024-01-07 14:07:17 INFO     top3_ques rela: []
2024-01-07 14:07:17 INFO     top3_ques rela: []
2024-01-07 14:07:17 INFO     mid_iters: ('m.051wrn1', 'm.0xy3jy')
2024-01-07 14:07:17 INFO     top3_ques rela: []
2024-01-07 14:07:17 INFO     top3_ques rela: []
2024-01-07 14:07:17 INFO     mid_iters: ('m.051wrn1', 'm.0lf7l0')
2024-01-07 14:07:17 INFO     top3_ques rela: []
2024-01-07 14:07:17 INFO     top3_ques rela: []
2024-01-07 14:07:17 INFO     mid_iters: ('m.051wrn1', 'm.0nn96kf')
2024-01-07 14:07:18 INFO     top3_ques rela: []
2024-01-07 14:07:18 INFO     top3_ques rela: []
2024-01-07 14:07:18 INFO     mid_iters: ('m.051wrn1', 'm.0nm93j')
2024-01-07 14:07:18 INFO     top3_ques rela: []
2024-01-07 14:07:18 INFO     top3_ques rela: []
2024-01-07 14:07:18 INFO     mid_iters: ('m.051wrn1', 'm.0gct2bf')
2024-01-07 14:07:18 INFO     top3_ques rela: []
2024-01-07 14:07:18 INFO     top3_ques rela: []
2024-01-07 14:07:18 INFO     mid_iters: ('m.051wrn1', 'm.0_656_t')
2024-01-07 14:07:18 INFO     top3_ques rela: []
2024-01-07 14:07:18 INFO     top3_ques rela: []
2024-01-07 14:07:18 INFO     mid_iters: ('m.051wrn1', 'm.0119mz_')
2024-01-07 14:07:18 INFO     top3_ques rela: []
2024-01-07 14:07:18 INFO     top3_ques rela: []
2024-01-07 14:07:18 INFO     mid_iters: ('m.051wrn1', 'm.0wcsyxd')
2024-01-07 14:07:19 INFO     top3_ques rela: []
2024-01-07 14:07:19 INFO     top3_ques rela: []
2024-01-07 14:07:19 INFO     mid_iters: ('m.051wrn1', 'm.010t_zq_')
2024-01-07 14:07:19 INFO     top3_ques rela: []
2024-01-07 14:07:19 INFO     top3_ques rela: []
2024-01-07 14:07:19 INFO     mid_iters: ('m.051wrn1', 'm.0387nv1')
2024-01-07 14:07:19 INFO     top3_ques rela: []
2024-01-07 14:07:19 INFO     top3_ques rela: ['location.country.fips10_4']
2024-01-07 14:07:19 INFO     mid_iters: ('m.051wrn1', 'm.035d5jd')
2024-01-07 14:07:19 INFO     top3_ques rela: []
2024-01-07 14:07:19 INFO     top3_ques rela: []
2024-01-07 14:07:19 INFO     mid_iters: ('m.051wrn1', 'm.01j6xtn')
2024-01-07 14:07:19 INFO     top3_ques rela: []
2024-01-07 14:07:19 INFO     top3_ques rela: []
2024-01-07 14:07:19 INFO     mid_iters: ('m.051wrn1', 'm.034xx7w')
2024-01-07 14:07:19 INFO     top3_ques rela: []
2024-01-07 14:07:20 INFO     top3_ques rela: []
2024-01-07 14:07:20 INFO     mid_iters: ('m.051wrn1', 'm.0g93yd1')
2024-01-07 14:07:20 INFO     top3_ques rela: []
2024-01-07 14:07:20 INFO     top3_ques rela: []
2024-01-07 14:07:20 INFO     mid_iters: ('m.051wrn1', 'm.03_bxlm')
2024-01-07 14:07:20 INFO     top3_ques rela: []
2024-01-07 14:07:20 INFO     top3_ques rela: []
2024-01-07 14:07:20 INFO     mid_iters: ('m.0j67b4h', 'm.0xy3jy')
2024-01-07 14:07:20 INFO     top3_ques rela: []
2024-01-07 14:07:20 INFO     top3_ques rela: []
2024-01-07 14:07:20 INFO     mid_iters: ('m.0j67b4h', 'm.0lf7l0')
2024-01-07 14:07:20 INFO     top3_ques rela: []
2024-01-07 14:07:20 INFO     top3_ques rela: []
2024-01-07 14:07:20 INFO     mid_iters: ('m.0j67b4h', 'm.0nn96kf')
2024-01-07 14:07:20 INFO     top3_ques rela: []
2024-01-07 14:07:21 INFO     top3_ques rela: []
2024-01-07 14:07:21 INFO     mid_iters: ('m.0j67b4h', 'm.0nm93j')
2024-01-07 14:07:21 INFO     top3_ques rela: []
2024-01-07 14:07:21 INFO     top3_ques rela: []
2024-01-07 14:07:21 INFO     mid_iters: ('m.0j67b4h', 'm.0gct2bf')
2024-01-07 14:07:21 INFO     top3_ques rela: []
2024-01-07 14:07:21 INFO     top3_ques rela: []
2024-01-07 14:07:21 INFO     mid_iters: ('m.0j67b4h', 'm.0_656_t')
2024-01-07 14:07:21 INFO     top3_ques rela: []
2024-01-07 14:07:21 INFO     top3_ques rela: []
2024-01-07 14:07:21 INFO     mid_iters: ('m.0j67b4h', 'm.0119mz_')
2024-01-07 14:07:21 INFO     top3_ques rela: []
2024-01-07 14:07:21 INFO     top3_ques rela: []
2024-01-07 14:07:21 INFO     mid_iters: ('m.0j67b4h', 'm.0wcsyxd')
2024-01-07 14:07:21 INFO     top3_ques rela: []
2024-01-07 14:07:21 INFO     top3_ques rela: []
2024-01-07 14:07:21 INFO     mid_iters: ('m.0j67b4h', 'm.010t_zq_')
2024-01-07 14:07:22 INFO     top3_ques rela: []
2024-01-07 14:07:22 INFO     top3_ques rela: []
2024-01-07 14:07:22 INFO     mid_iters: ('m.0j67b4h', 'm.0387nv1')
2024-01-07 14:07:22 INFO     top3_ques rela: []
2024-01-07 14:07:22 INFO     top3_ques rela: ['location.country.fips10_4']
2024-01-07 14:07:22 INFO     mid_iters: ('m.0j67b4h', 'm.035d5jd')
2024-01-07 14:07:22 INFO     top3_ques rela: []
2024-01-07 14:07:22 INFO     top3_ques rela: []
2024-01-07 14:07:22 INFO     mid_iters: ('m.0j67b4h', 'm.01j6xtn')
2024-01-07 14:07:22 INFO     top3_ques rela: []
2024-01-07 14:07:22 INFO     top3_ques rela: []
2024-01-07 14:07:22 INFO     mid_iters: ('m.0j67b4h', 'm.034xx7w')
2024-01-07 14:07:22 INFO     top3_ques rela: []
2024-01-07 14:07:22 INFO     top3_ques rela: []
2024-01-07 14:07:22 INFO     mid_iters: ('m.0j67b4h', 'm.0g93yd1')
2024-01-07 14:07:22 INFO     top3_ques rela: []
2024-01-07 14:07:23 INFO     top3_ques rela: []
2024-01-07 14:07:23 INFO     mid_iters: ('m.0j67b4h', 'm.03_bxlm')
2024-01-07 14:07:23 INFO     top3_ques rela: []
2024-01-07 14:07:23 INFO     top3_ques rela: []
2024-01-07 14:07:23 INFO     mid_iters: ('m.03nnmkj', 'm.0xy3jy')
2024-01-07 14:07:23 INFO     top3_ques rela: []
2024-01-07 14:07:23 INFO     top3_ques rela: []
2024-01-07 14:07:23 INFO     mid_iters: ('m.03nnmkj', 'm.0lf7l0')
2024-01-07 14:07:23 INFO     top3_ques rela: []
2024-01-07 14:07:23 INFO     top3_ques rela: []
2024-01-07 14:07:23 INFO     mid_iters: ('m.03nnmkj', 'm.0nn96kf')
2024-01-07 14:07:23 INFO     top3_ques rela: []
2024-01-07 14:07:23 INFO     top3_ques rela: []
2024-01-07 14:07:23 INFO     mid_iters: ('m.03nnmkj', 'm.0nm93j')
2024-01-07 14:07:23 INFO     top3_ques rela: []
2024-01-07 14:07:23 INFO     top3_ques rela: []
2024-01-07 14:07:23 INFO     mid_iters: ('m.03nnmkj', 'm.0gct2bf')
2024-01-07 14:07:24 INFO     top3_ques rela: []
2024-01-07 14:07:24 INFO     top3_ques rela: []
2024-01-07 14:07:24 INFO     mid_iters: ('m.03nnmkj', 'm.0_656_t')
2024-01-07 14:07:24 INFO     top3_ques rela: []
2024-01-07 14:07:24 INFO     top3_ques rela: []
2024-01-07 14:07:24 INFO     mid_iters: ('m.03nnmkj', 'm.0119mz_')
2024-01-07 14:07:24 INFO     top3_ques rela: []
2024-01-07 14:07:24 INFO     top3_ques rela: []
2024-01-07 14:07:24 INFO     mid_iters: ('m.03nnmkj', 'm.0wcsyxd')
2024-01-07 14:07:24 INFO     top3_ques rela: []
2024-01-07 14:07:24 INFO     top3_ques rela: []
2024-01-07 14:07:24 INFO     mid_iters: ('m.03nnmkj', 'm.010t_zq_')
2024-01-07 14:07:24 INFO     top3_ques rela: []
2024-01-07 14:07:24 INFO     top3_ques rela: []
2024-01-07 14:07:24 INFO     mid_iters: ('m.03nnmkj', 'm.0387nv1')
2024-01-07 14:07:25 INFO     top3_ques rela: []
2024-01-07 14:07:25 INFO     top3_ques rela: ['location.country.fips10_4']
2024-01-07 14:07:25 INFO     mid_iters: ('m.03nnmkj', 'm.035d5jd')
2024-01-07 14:07:25 INFO     top3_ques rela: []
2024-01-07 14:07:25 INFO     top3_ques rela: []
2024-01-07 14:07:25 INFO     mid_iters: ('m.03nnmkj', 'm.01j6xtn')
2024-01-07 14:07:25 INFO     top3_ques rela: []
2024-01-07 14:07:25 INFO     top3_ques rela: []
2024-01-07 14:07:25 INFO     mid_iters: ('m.03nnmkj', 'm.034xx7w')
2024-01-07 14:07:25 INFO     top3_ques rela: []
2024-01-07 14:07:25 INFO     top3_ques rela: []
2024-01-07 14:07:25 INFO     mid_iters: ('m.03nnmkj', 'm.0g93yd1')
2024-01-07 14:07:25 INFO     top3_ques rela: []
2024-01-07 14:07:25 INFO     top3_ques rela: []
2024-01-07 14:07:25 INFO     mid_iters: ('m.03nnmkj', 'm.03_bxlm')
2024-01-07 14:07:25 INFO     top3_ques rela: []
2024-01-07 14:07:26 INFO     top3_ques rela: []
2024-01-07 14:07:26 INFO     mid_iters: ('m.02pv7y8', 'm.0xy3jy')
2024-01-07 14:07:26 INFO     top3_ques rela: []
2024-01-07 14:07:26 INFO     top3_ques rela: []
2024-01-07 14:07:26 INFO     mid_iters: ('m.02pv7y8', 'm.0lf7l0')
2024-01-07 14:07:26 INFO     top3_ques rela: []
2024-01-07 14:07:26 INFO     top3_ques rela: []
2024-01-07 14:07:26 INFO     mid_iters: ('m.02pv7y8', 'm.0nn96kf')
2024-01-07 14:07:26 INFO     top3_ques rela: []
2024-01-07 14:07:26 INFO     top3_ques rela: []
2024-01-07 14:07:26 INFO     mid_iters: ('m.02pv7y8', 'm.0nm93j')
2024-01-07 14:07:26 INFO     top3_ques rela: []
2024-01-07 14:07:26 INFO     top3_ques rela: []
2024-01-07 14:07:26 INFO     mid_iters: ('m.02pv7y8', 'm.0gct2bf')
2024-01-07 14:07:26 INFO     top3_ques rela: []
2024-01-07 14:07:27 INFO     top3_ques rela: []
2024-01-07 14:07:27 INFO     mid_iters: ('m.02pv7y8', 'm.0_656_t')
2024-01-07 14:07:27 INFO     top3_ques rela: []
2024-01-07 14:07:27 INFO     top3_ques rela: []
2024-01-07 14:07:27 INFO     mid_iters: ('m.02pv7y8', 'm.0119mz_')
2024-01-07 14:07:27 INFO     top3_ques rela: []
2024-01-07 14:07:27 INFO     top3_ques rela: []
2024-01-07 14:07:27 INFO     mid_iters: ('m.02pv7y8', 'm.0wcsyxd')
2024-01-07 14:07:27 INFO     top3_ques rela: []
2024-01-07 14:07:27 INFO     top3_ques rela: []
2024-01-07 14:07:27 INFO     mid_iters: ('m.02pv7y8', 'm.010t_zq_')
2024-01-07 14:07:27 INFO     top3_ques rela: []
2024-01-07 14:07:27 INFO     top3_ques rela: []
2024-01-07 14:07:27 INFO     mid_iters: ('m.02pv7y8', 'm.0387nv1')
2024-01-07 14:07:27 INFO     top3_ques rela: []
2024-01-07 14:07:27 INFO     top3_ques rela: ['location.country.fips10_4']
2024-01-07 14:07:27 INFO     mid_iters: ('m.02pv7y8', 'm.035d5jd')
2024-01-07 14:07:28 INFO     top3_ques rela: []
2024-01-07 14:07:28 INFO     top3_ques rela: []
2024-01-07 14:07:28 INFO     mid_iters: ('m.02pv7y8', 'm.01j6xtn')
2024-01-07 14:07:28 INFO     top3_ques rela: []
2024-01-07 14:07:28 INFO     top3_ques rela: []
2024-01-07 14:07:28 INFO     mid_iters: ('m.02pv7y8', 'm.034xx7w')
2024-01-07 14:07:28 INFO     top3_ques rela: []
2024-01-07 14:07:28 INFO     top3_ques rela: []
2024-01-07 14:07:28 INFO     mid_iters: ('m.02pv7y8', 'm.0g93yd1')
2024-01-07 14:07:28 INFO     top3_ques rela: []
2024-01-07 14:07:28 INFO     top3_ques rela: []
2024-01-07 14:07:28 INFO     mid_iters: ('m.02pv7y8', 'm.03_bxlm')
2024-01-07 14:07:28 INFO     top3_ques rela: []
2024-01-07 14:07:28 INFO     top3_ques rela: []
2024-01-07 14:07:28 INFO     mid_iters: ('m.0g_0p_', 'm.0xy3jy')
2024-01-07 14:07:28 INFO     top3_ques rela: []
2024-01-07 14:07:29 INFO     top3_ques rela: []
2024-01-07 14:07:29 INFO     mid_iters: ('m.0g_0p_', 'm.0lf7l0')
2024-01-07 14:07:29 INFO     top3_ques rela: []
2024-01-07 14:07:29 INFO     top3_ques rela: []
2024-01-07 14:07:29 INFO     mid_iters: ('m.0g_0p_', 'm.0nn96kf')
2024-01-07 14:07:29 INFO     top3_ques rela: []
2024-01-07 14:07:29 INFO     top3_ques rela: []
2024-01-07 14:07:29 INFO     mid_iters: ('m.0g_0p_', 'm.0nm93j')
2024-01-07 14:07:29 INFO     top3_ques rela: []
2024-01-07 14:07:29 INFO     top3_ques rela: []
2024-01-07 14:07:29 INFO     mid_iters: ('m.0g_0p_', 'm.0gct2bf')
2024-01-07 14:07:29 INFO     top3_ques rela: []
2024-01-07 14:07:29 INFO     top3_ques rela: []
2024-01-07 14:07:29 INFO     mid_iters: ('m.0g_0p_', 'm.0_656_t')
2024-01-07 14:07:29 INFO     top3_ques rela: []
2024-01-07 14:07:30 INFO     top3_ques rela: []
2024-01-07 14:07:30 INFO     mid_iters: ('m.0g_0p_', 'm.0119mz_')
2024-01-07 14:07:30 INFO     top3_ques rela: []
2024-01-07 14:07:30 INFO     top3_ques rela: []
2024-01-07 14:07:30 INFO     mid_iters: ('m.0g_0p_', 'm.0wcsyxd')
2024-01-07 14:07:30 INFO     top3_ques rela: []
2024-01-07 14:07:30 INFO     top3_ques rela: []
2024-01-07 14:07:30 INFO     mid_iters: ('m.0g_0p_', 'm.010t_zq_')
2024-01-07 14:07:30 INFO     top3_ques rela: []
2024-01-07 14:07:30 INFO     top3_ques rela: []
2024-01-07 14:07:30 INFO     mid_iters: ('m.0g_0p_', 'm.0387nv1')
2024-01-07 14:07:30 INFO     top3_ques rela: []
2024-01-07 14:07:30 INFO     top3_ques rela: ['location.country.fips10_4']
2024-01-07 14:07:30 INFO     mid_iters: ('m.0g_0p_', 'm.035d5jd')
2024-01-07 14:07:30 INFO     top3_ques rela: []
2024-01-07 14:07:30 INFO     top3_ques rela: []
2024-01-07 14:07:30 INFO     mid_iters: ('m.0g_0p_', 'm.01j6xtn')
2024-01-07 14:07:31 INFO     top3_ques rela: []
2024-01-07 14:07:31 INFO     top3_ques rela: []
2024-01-07 14:07:31 INFO     mid_iters: ('m.0g_0p_', 'm.034xx7w')
2024-01-07 14:07:31 INFO     top3_ques rela: []
2024-01-07 14:07:31 INFO     top3_ques rela: []
2024-01-07 14:07:31 INFO     mid_iters: ('m.0g_0p_', 'm.0g93yd1')
2024-01-07 14:07:31 INFO     top3_ques rela: []
2024-01-07 14:07:31 INFO     top3_ques rela: []
2024-01-07 14:07:31 INFO     mid_iters: ('m.0g_0p_', 'm.03_bxlm')
2024-01-07 14:07:31 INFO     top3_ques rela: []
2024-01-07 14:07:31 INFO     top3_ques rela: []
2024-01-07 14:07:31 INFO     predicted_answer: None
2024-01-07 14:07:31 INFO     label: ['m.023s6n', 'm.0brgy', 'm.0f3kl', 'm.0cjf0', 'm.01j6t0', 'm.0lcdk']
2024-01-07 14:07:31 INFO     ================================================================
2024-01-07 14:07:31 INFO     consistent candidates number: 3
2024-01-07 14:07:31 INFO     em_score: 0.0
2024-01-07 14:07:31 INFO     correct: 0
2024-01-07 14:07:31 INFO     total: 1
2024-01-07 14:07:31 INFO     no_ans: 1
2024-01-07 14:07:31 INFO      
2024-01-07 14:07:31 INFO     ================================================================
2024-01-07 14:07:31 INFO     gene_exp: (OR (AND veterinary_medicine.disease_common_symptoms (JOIN veterinary_medicine.disease_common_symptoms.disease fip)) (AND medical.disease_caused_by (JOIN medical.disease_caused_by.cause old_age)))
2024-01-07 14:07:31 INFO     all_iters: [('m.03nnmkj',), ('m.02pv7y8',), ('m.0g_0p_',), ('m.05yx0z',), ('m.071hyv',), ('m.074n2y',), ('m.04g1ktv',), ('m.0gttrfn',), ('m.04dvxh',), ('m.03c5gry',), ('m.051wrn1',), ('m.0j67b4h',), ('m.03nnmkj',), ('m.02pv7y8',), ('m.0g_0p_',)]
2024-01-07 14:07:31 INFO     mid_iters: ('m.03nnmkj',)
2024-01-07 14:07:31 INFO     top3_ques rela: []
2024-01-07 14:07:32 INFO     top3_ques rela: []
2024-01-07 14:07:32 INFO     mid_iters: ('m.02pv7y8',)
2024-01-07 14:07:32 INFO     top3_ques rela: []
2024-01-07 14:07:32 INFO     top3_ques rela: []
2024-01-07 14:07:32 INFO     mid_iters: ('m.0g_0p_',)
2024-01-07 14:07:32 INFO     top3_ques rela: []
2024-01-07 14:07:32 INFO     top3_ques rela: []
2024-01-07 14:07:32 INFO     mid_iters: ('m.05yx0z',)
2024-01-07 14:07:32 INFO     top3_ques rela: []
2024-01-07 14:07:32 INFO     top3_ques rela: []
2024-01-07 14:07:32 INFO     mid_iters: ('m.071hyv',)
2024-01-07 14:07:32 INFO     top3_ques rela: []
2024-01-07 14:07:32 INFO     top3_ques rela: []
2024-01-07 14:07:32 INFO     mid_iters: ('m.074n2y',)
2024-01-07 14:07:32 INFO     top3_ques rela: []
2024-01-07 14:07:32 INFO     top3_ques rela: []
2024-01-07 14:07:32 INFO     mid_iters: ('m.04g1ktv',)
2024-01-07 14:07:33 INFO     top3_ques rela: []
2024-01-07 14:07:33 INFO     top3_ques rela: []
2024-01-07 14:07:33 INFO     mid_iters: ('m.0gttrfn',)
2024-01-07 14:07:33 INFO     top3_ques rela: []
2024-01-07 14:07:33 INFO     top3_ques rela: []
2024-01-07 14:07:33 INFO     mid_iters: ('m.04dvxh',)
2024-01-07 14:07:33 INFO     top3_ques rela: []
2024-01-07 14:07:33 INFO     top3_ques rela: []
2024-01-07 14:07:33 INFO     mid_iters: ('m.03c5gry',)
2024-01-07 14:07:33 INFO     top3_ques rela: []
2024-01-07 14:07:33 INFO     top3_ques rela: []
2024-01-07 14:07:33 INFO     mid_iters: ('m.051wrn1',)
2024-01-07 14:07:33 INFO     top3_ques rela: []
2024-01-07 14:07:33 INFO     top3_ques rela: []
2024-01-07 14:07:33 INFO     mid_iters: ('m.0j67b4h',)
2024-01-07 14:07:33 INFO     top3_ques rela: []
2024-01-07 14:07:34 INFO     top3_ques rela: []
2024-01-07 14:07:34 INFO     mid_iters: ('m.03nnmkj',)
2024-01-07 14:07:34 INFO     top3_ques rela: []
2024-01-07 14:07:34 INFO     top3_ques rela: []
2024-01-07 14:07:34 INFO     mid_iters: ('m.02pv7y8',)
2024-01-07 14:07:34 INFO     top3_ques rela: []
2024-01-07 14:07:34 INFO     top3_ques rela: []
2024-01-07 14:07:34 INFO     mid_iters: ('m.0g_0p_',)
2024-01-07 14:07:34 INFO     top3_ques rela: []
2024-01-07 14:07:34 INFO     top3_ques rela: []
2024-01-07 14:07:34 INFO     predicted_answer: None
2024-01-07 14:07:34 INFO     label: ['m.023s6n', 'm.0brgy', 'm.0f3kl', 'm.0cjf0', 'm.01j6t0', 'm.0lcdk']
2024-01-07 14:07:34 INFO     ================================================================
2024-01-07 14:07:34 INFO     consistent candidates number: 4
2024-01-07 14:07:34 INFO     em_score: 0.0
2024-01-07 14:07:34 INFO     correct: 0
2024-01-07 14:07:34 INFO     total: 1
2024-01-07 14:07:34 INFO     no_ans: 1
2024-01-07 14:07:34 INFO      
2024-01-07 14:07:34 INFO     ================================================================
2024-01-07 14:07:34 INFO     gene_exp: (OR (AND veterinary_medicine.disease_common_symptoms (JOIN veterinary_medicine.disease_common_symptoms.disease (R veterinary_medicine.disease "fip"))) (AND medical.disease_caused_by_age (R medical.disease "old age"))))
2024-01-07 14:07:34 INFO     all_iters: [()]
2024-01-07 14:07:34 INFO     mid_iters: ()
2024-01-07 14:07:34 INFO     top3_ques rela: ['medicine.disease.symptoms', 'medicine.disease_cause.diseases', 'medicine.disease.causes', 'medicine.diagnostic_test_signs.disease', 'medicine.disease.risk_factors', 'medicine.disease.includes_diseases', 'medicine.disease.parent_disease', 'medicine.diagnostic_test_signs.sign']
2024-01-07 14:07:34 INFO     top3_ques rela: ['medicine.disease_cause.diseases', 'medicine.disease.symptoms', 'medicine.disease.causes', 'medicine.diagnostic_test_signs.disease', 'medicine.disease.risk_factors', 'medicine.disease.includes_diseases', 'medicine.disease.parent_disease', 'medicine.disease.affiliated_diseases']
2024-01-07 14:07:34 INFO     top3_ques rela: ['medicine.disease_cause.diseases', 'medicine.disease.symptoms', 'medicine.disease.causes', 'medicine.diagnostic_test_signs.disease', 'location.country.fips10_4', 'biology.plant_disease_cause.plant_disease_triangle', 'biology.plant_disease_triangle.plant_disease_cause', 'medicine.disease.risk_factors']
2024-01-07 14:07:34 INFO     predicted_answer: None
2024-01-07 14:07:34 INFO     label: ['m.023s6n', 'm.0brgy', 'm.0f3kl', 'm.0cjf0', 'm.01j6t0', 'm.0lcdk']
2024-01-07 14:07:34 INFO     ================================================================
2024-01-07 14:07:34 INFO     consistent candidates number: 5
2024-01-07 14:07:34 INFO     em_score: 0.0
2024-01-07 14:07:34 INFO     correct: 0
2024-01-07 14:07:34 INFO     total: 1
2024-01-07 14:07:34 INFO     no_ans: 1
2024-01-07 14:07:34 INFO      
2024-01-07 14:07:34 INFO     ================================================================
2024-01-07 14:07:34 INFO     gene_exp: (OR (AND veterinary.mammal_disease.symptoms (JOIN veterinary.mammal_disease.symptoms (R veterinary.mammal_disease.name "fip"))) (AND medical.disease.symptoms (JOIN medical.disease.symptoms (R medical.disease.caused_by "old age")))) (AND veterinary.mammal_disease.name "fip" (R veterinary.mammal_disease.caused_by "old age")))
2024-01-07 14:07:34 INFO     all_iters: [()]
2024-01-07 14:07:34 INFO     mid_iters: ()
2024-01-07 14:07:34 INFO     top3_ques rela: ['medicine.disease.symptoms', 'medicine.symptom.symptom_of', 'medicine.disease_cause.diseases', 'medicine.symptom.includes_symptoms', 'medicine.symptom.parent_symptom', 'biology.plant_disease_cause.plant_disease_triangle']
2024-01-07 14:07:35 INFO     top3_ques rela: ['medicine.disease_cause.diseases', 'medicine.disease.symptoms', 'biology.plant_disease_cause.plant_disease_triangle', 'biology.plant_disease_triangle.plant_disease_cause', 'medicine.disease.causes']
2024-01-07 14:07:35 INFO     top3_ques rela: ['medicine.disease.symptoms', 'medicine.diagnostic_test_signs.disease', 'medicine.disease.risk_factors', 'medicine.diagnostic_test_signs.sign', 'location.country.fips10_4', 'medicine.condition_prevention_factors.conditions_this_may_prevent', 'medicine.symptom.symptom_of', 'medicine.disease.causes']
2024-01-07 14:07:35 INFO     top3_ques rela: ['medicine.disease_cause.diseases', 'medicine.disease.causes', 'biology.plant_disease_cause.plant_disease_triangle', 'biology.plant_disease_triangle.plant_disease_cause', 'location.country.fips10_4', 'medicine.medical_treatment.used_to_treat', 'medicine.diagnostic_test_signs.disease', 'medicine.symptom.includes_symptoms']
2024-01-07 14:07:35 INFO     top3_ques rela: ['medicine.disease_cause.diseases', 'biology.plant_disease_cause.plant_disease_triangle', 'biology.plant_disease_triangle.plant_disease_cause', 'medicine.disease.causes', 'medicine.disease.symptoms']
2024-01-07 14:07:36 INFO     predicted_answer: None
2024-01-07 14:07:36 INFO     label: ['m.023s6n', 'm.0brgy', 'm.0f3kl', 'm.0cjf0', 'm.01j6t0', 'm.0lcdk']
2024-01-07 14:07:36 INFO     ================================================================
2024-01-07 14:07:36 INFO     consistent candidates number: 6
2024-01-07 14:07:36 INFO     em_score: 0.0
2024-01-07 14:07:36 INFO     correct: 0
2024-01-07 14:07:36 INFO     total: 1
2024-01-07 14:07:36 INFO     no_ans: 1
2024-01-07 14:07:36 INFO      
2024-01-07 14:07:36 INFO     ================================================================
divinerIU commented 3 months ago

hello，I want to ask some question about running time，my code is not moving,and this is my log, I want to know if you meet same problem.

2024-03-19 22:26:39 INFO     selected_quest_compose: ['what is the championship game for european beach volleyball championships in sports league?', 'what type of airport is zirku airport?', 'what are the different female gender pedigree breeds in borzois?', "black knight's quotation comes from what source?", 'what are the camera formats of the olympus stylus line of digital cameras?', 'what location is the tv show joy of the lord is your strength filmed in?', 'which tournament champions were defeated by mike bryan?', 'edges were edited by who?', 'who manufactured a rocket whose country of origin is ukraine?', 'what organization does james buchanan barnes belong to?', 'glacier is the closest city to which ski area?', 'jul. the 30th, 2012 was the end date of which broadcast content?', 'what patents does michael d. mcgehee invent?', '1999 rodney strong sonoma county cabernet sauvignon is what color of wine?', 'what wines do fisher vineyards produce?', 'ryan barber contributed on which live album?', 'the three shades is an edition for what artworks?', 'sodium chloride/sodium lactate/potassium chloride/calcium chloride belongs to what drug formulation?', 'trapped in dimension x is in which comic book story?', 'what helicobacter infections treatments has sludge syndrome side effects?']
2024-03-19 22:26:39 INFO     selected_quest_compare: ['the rd-250pm designed which rocket engine designer?', 'tamil nadu forest department is the agency of which government?', "the return of hopeful monsters cited joel b. dacks's work in what journal article?", 'paul curcio produced which musical release?', 'particle nutrino has what fundamental interaction?', 'of the wine in county which has the greatest amount of alcohol?', 'what religion does ja a. jahannes believe in?', 'what comic book series is in the same genre as beware of the blob!?', 'what is the subatomic particle with the greatest electrical charge?', 'artist and dj ras cue plays what type of music?', 'what is the name of the tropical cyclone category has the smallest maximum wind speed (gust)?', 'who developed cascading style sheets?', 'name the programming language designed by stroustrup that is used in enterprise dynamics.', 'what rocket engine cycles do have rs-25', 'what digital camera uses the same viewfinder as the canon eos and weighs less than 1074.0 ?', "which digital camera's effective resolution is 2007-04?", 'universiteit leiden houses what telescope?', 'what is a recipe that takes less than 225.0?', 'which one of the olympic games events is a contest, and which one of the olympic games has the smallest number of events?', 'what time zone has an offset from utc more than or equal to 8.0?']
2024-03-19 22:26:39 INFO     selected_quest: ['what is the championship game for european beach volleyball championships in sports league?', 'what type of airport is zirku airport?', 'what are the different female gender pedigree breeds in borzois?', "black knight's quotation comes from what source?", 'what are the camera formats of the olympus stylus line of digital cameras?', 'what location is the tv show joy of the lord is your strength filmed in?', 'the rd-250pm designed which rocket engine designer?', 'tamil nadu forest department is the agency of which government?', "the return of hopeful monsters cited joel b. dacks's work in what journal article?", 'paul curcio produced which musical release?', 'particle nutrino has what fundamental interaction?', 'of the wine in county which has the greatest amount of alcohol?']
2024-03-19 22:33:39 INFO     ==========
2024-03-19 22:33:39 INFO     data[id]: 3202959008000
2024-03-19 22:33:39 INFO     data[question]: what is the role of opera designer gig who designed the telephone / the medium?
2024-03-19 22:33:39 INFO     data[exp]: (AND opera.opera_designer_role (JOIN (R opera.opera_designer_gig.design_role) (JOIN (R opera.opera_production.designers) m.0pm2fgf)))