Open sujeetvkulkarni opened 2 months ago
Here is what I have so far:
$ cat tests/examples/pages/list_init.json
{
"table_id":"protein"
}
$ http POST :8082/pages/list_init/ < tests/examples/pages/list_init.json
HTTP/1.1 200 OK
Connection: close
Content-Length: 3698
Content-Type: application/json
Date: Fri, 20 Sep 2024 18:09:00 GMT
Server: gunicorn
{
"columns": [
{
"id": "general",
"label": "General",
"options": [
{
"id": "uniprot_canonical_ac",
"label": "UniProtKB Accession",
"order": 1
},
{
"id": "mass",
"label": "Chemical Mass (Da)",
"order": 2
},
{
"id": "protein_name",
"label": "UniProtKB Name",
"order": 3
},
{
"id": "gene_name",
"label": "Gene Name",
"order": 4
},
{
"id": "organism",
"label": "Organism",
"order": 5
},
{
"id": "refseq_name",
"label": "RefSeq Name",
"order": 6
},
{
"id": "refseq_ac",
"label": "RefSeq Accession",
"order": 7
},
{
"id": "tax_id",
"label": "Taxonomy ID",
"order": 8
},
{
"id": "missing_score",
"label": "missing_score",
"order": 9
},
{
"id": "hit_score",
"label": "Hit Score",
"order": 10
}
],
"order": 1,
"tooltip": ""
},
{
"id": "names",
"label": "Names",
"options": [
{
"id": "gene_names_uniprotkb",
"label": "gene_names_uniprotkb",
"order": 11
},
{
"id": "gene_names_refseq",
"label": "gene_names_refseq",
"order": 12
},
{
"id": "protein_names_uniprotkb",
"label": "protein_names_uniprotkb",
"order": 1
},
{
"id": "protein_names_refseq",
"label": "protein_names_refseq",
"order": 1
}
],
"order": 11,
"tooltip": ""
},
{
"id": "diseases",
"label": "Diseases",
"options": [
{
"id": "disease",
"label": "Disease",
"order": 1
}
],
"order": 1,
"tooltip": ""
},
{
"id": "pathways",
"label": "Pathways",
"options": [
{
"id": "pathway",
"label": "Pathway",
"order": 1
}
],
"order": 1,
"tooltip": ""
},
{
"id": "publications",
"label": "Publications",
"options": [
{
"id": "publication_count",
"label": "publication_count",
"order": 1
}
],
"order": 1,
"tooltip": ""
},
{
"id": "function",
"label": "Function",
"options": [
{
"id": "function",
"label": "function",
"order": 1
}
],
"order": 1,
"tooltip": ""
},
{
"id": "glycosylation",
"label": "Glycosylation",
"options": [
{
"id": "predicted_glycosites",
"label": "predicted_glycosites",
"order": 1
},
{
"id": "predicted_n_glycosites",
"label": "predicted_n_glycosites",
"order": 1
},
{
"id": "predicted_o_glycosites",
"label": "predicted_o_glycosites",
"order": 1
},
{
"id": "reported_n_glycosites",
"label": "reported_n_glycosites",
"order": 1
},
{
"id": "reported_o_glycosites",
"label": "reported_o_glycosites",
"order": 1
},
{
"id": "reported_n_glycosites_with_glycan",
"label": "reported_n_glycosites_with_glycan",
"order": 1
},
{
"id": "reported_o_glycosites_with_glycan",
"label": "reported_o_glycosites_with_glycan",
"order": 1
},
{
"id": "total_reported_n_glycosites",
"label": "total_reported_n_glycosites",
"order": 1
},
{
"id": "total_reported_o_glycosites",
"label": "total_reported_o_glycosites",
"order": 1
},
{
"id": "total_n_glycosites",
"label": "total_n_glycosites",
"order": 1
},
{
"id": "total_o_glycosites",
"label": "total_o_glycosites",
"order": 1
},
{
"id": "reported_fully_resolved_glycans",
"label": "reported_fully_resolved_glycans",
"order": 1
}
],
"order": 1,
"tooltip": ""
},
{
"id": "phosphorylation",
"label": "Phosphorylation",
"options": [
{
"id": "reported_phosphosites",
"label": "reported_phosphosites",
"order": 1
}
],
"order": 1,
"tooltip": ""
},
{
"id": "mutagensis",
"label": "Mutagensis",
"options": [
{
"id": "reported_mutagensis",
"label": "reported_mutagensis",
"order": 1
}
],
"order": 1,
"tooltip": ""
},
{
"id": "glycation",
"label": "Glycation",
"options": [
{
"id": "reported_glycation",
"label": "reported_glycation",
"order": 1
}
],
"order": 1,
"tooltip": ""
},
{
"id": "snv",
"label": "SNV",
"options": [
{
"id": "reported_snv",
"label": "reported_snv",
"order": 1
}
],
"order": 1,
"tooltip": ""
},
{
"id": "interactions",
"label": "Interactions",
"options": [
{
"id": "reported_interactions",
"label": "reported_interactions",
"order": 1
}
],
"order": 1,
"tooltip": ""
},
{
"id": "biomarkers",
"label": "Biomarkers",
"options": [
{
"id": "biomarker_type",
"label": "biomarker_type",
"order": 1
}
],
"order": 1,
"tooltip": ""
}
]
}
@rykahsay it would also need default and immutable property per column (I added it today above after realizing it was missing). In some cases it would be true or in some case it would be false (or maybe absent). For protein going by existing columns : uniprot_canonical_ac, mass, protein_name, gene_name, hit_score, organism, refseq_name, refseq_ac. These need to have default property set to true. Also, column like uniprot_canonical_ac and hit_score needs to have immutable property set to true - user cant remove them.
In case if immutable is true default will always be true. So maybe in such case we dont need to specify default property.
@ReneRanzinger suggestions for json for the list_init api. This way we can have same column in multiple categories - which might be needed.
{
"columns":[
{
"id":"column_1_id",
"property_name":"property in list json",
"label":"Column_1",
"default":true,
"immutable":true,
"tooltip":"",
"categories":[
{
"id":"category_1",
"order":1
},
{
"id":"category_2",
"order":1
}
]
}
],
"categories":[
{
"id":"category_1",
"label":"Category 1",
"order":1,
"tooltip":""
},
{
"id":"category_2",
"label":"Category 2",
"order":2,
"tooltip":""
}
]
}
I have tried to implement, response is now as shown below.
@ubhuiyan @katewarner ... this API response depends CSV files "generated/misc/list_init_categories.csv" and "generated/misc/list_init_fields.csv" I have initiated that you will start maintain. I will make a separate ticket to update those files based on the spreadsheet Sujeeet shared.
$ head generated/misc/list_init_fields.csv
record_type,field_id,field_label,field_order,is_default,is_immutable,category_id_list
protein,uniprot_canonical_ac,UniProtKB Accession,1,true,true,General
protein,mass,Chemical Mass (Da),2,true,false,General
protein,protein_name,UniProtKB Name,3,true,false,General
protein,gene_name,Gene Name,4,true,false,General
protein,organism,Organism,5,true,false,General
protein,refseq_name,RefSeq Name,6,true,false,General
protein,refseq_ac,RefSeq Accession,7,true,false,General
protein,tax_id,Taxonomy ID,8,false,false,General
protein,missing_score,missing_score,9,false,false,General
$ head generated/misc/list_init_categories.csv
record_type,cat_id,cat_label,cat_order
protein,General,General,1
protein,Biomarkers,Biomarkers,1
protein,Diseases,Diseases,1
protein,Function,Function,1
protein,Glycation,Glycation,1
protein,Glycosylation,Glycosylation,1
protein,Interactions,Interactions,1
protein,Mutagenesis,Mutagenesis,1
protein,Mutagensis,Mutagensis,1
$ http POST :8082/pages/list_init/ < tests/examples/pages/list_init.json
HTTP/1.1 200 OK
Connection: close
Content-Length: 8695
Content-Type: application/json
Date: Fri, 27 Sep 2024 17:10:57 GMT
Server: gunicorn
{
"categories": [
{
"id": "General",
"label": "General",
"order": "1",
"tooltip": ""
},
{
"id": "Names",
"label": "Names",
"order": "1",
"tooltip": ""
},
{
"id": "Diseases",
"label": "Diseases",
"order": "1",
"tooltip": ""
},
{
"id": "Pathways",
"label": "Pathways",
"order": "1",
"tooltip": ""
},
{
"id": "Publications",
"label": "Publications",
"order": "1",
"tooltip": ""
},
{
"id": "Function",
"label": "Function",
"order": "1",
"tooltip": ""
},
{
"id": "Glycosylation",
"label": "Glycosylation",
"order": "1",
"tooltip": ""
},
{
"id": "Phosphorylation",
"label": "Phosphorylation",
"order": "1",
"tooltip": ""
},
{
"id": "Mutagensis",
"label": "Mutagensis",
"order": "1",
"tooltip": ""
},
{
"id": "Glycation",
"label": "Glycation",
"order": "1",
"tooltip": ""
},
{
"id": "SNV",
"label": "SNV",
"order": "1",
"tooltip": ""
},
{
"id": "Interactions",
"label": "Interactions",
"order": "1",
"tooltip": ""
},
{
"id": "Biomarkers",
"label": "Biomarkers",
"order": "1",
"tooltip": ""
}
],
"columns": [
{
"categories": [
{
"id": "General",
"order": "1"
}
],
"default": true,
"id": "uniprot_canonical_ac",
"immutable": true,
"label": "UniProtKB Accession",
"order": 1,
"property_name": "uniprot_canonical_ac",
"tooltip": ""
},
{
"categories": [
{
"id": "General",
"order": "1"
}
],
"default": true,
"id": "mass",
"immutable": false,
"label": "Chemical Mass (Da)",
"order": 2,
"property_name": "mass",
"tooltip": ""
},
{
"categories": [
{
"id": "General",
"order": "1"
}
],
"default": true,
"id": "protein_name",
"immutable": false,
"label": "UniProtKB Name",
"order": 3,
"property_name": "protein_name",
"tooltip": ""
},
{
"categories": [
{
"id": "General",
"order": "1"
}
],
"default": true,
"id": "gene_name",
"immutable": false,
"label": "Gene Name",
"order": 4,
"property_name": "gene_name",
"tooltip": ""
},
{
"categories": [
{
"id": "General",
"order": "1"
}
],
"default": true,
"id": "organism",
"immutable": false,
"label": "Organism",
"order": 5,
"property_name": "organism",
"tooltip": ""
},
{
"categories": [
{
"id": "General",
"order": "1"
}
],
"default": true,
"id": "refseq_name",
"immutable": false,
"label": "RefSeq Name",
"order": 6,
"property_name": "refseq_name",
"tooltip": ""
},
{
"categories": [
{
"id": "General",
"order": "1"
}
],
"default": true,
"id": "refseq_ac",
"immutable": false,
"label": "RefSeq Accession",
"order": 7,
"property_name": "refseq_ac",
"tooltip": ""
},
{
"categories": [
{
"id": "General",
"order": "1"
}
],
"default": false,
"id": "tax_id",
"immutable": false,
"label": "Taxonomy ID",
"order": 8,
"property_name": "tax_id",
"tooltip": ""
},
{
"categories": [
{
"id": "General",
"order": "1"
}
],
"default": false,
"id": "missing_score",
"immutable": false,
"label": "missing_score",
"order": 9,
"property_name": "missing_score",
"tooltip": ""
},
{
"categories": [
{
"id": "General",
"order": "1"
}
],
"default": true,
"id": "hit_score",
"immutable": true,
"label": "Hit Score",
"order": 10,
"property_name": "hit_score",
"tooltip": ""
},
{
"categories": [
{
"id": "Names",
"order": "1"
}
],
"default": false,
"id": "gene_names_uniprotkb",
"immutable": false,
"label": "gene_names_uniprotkb",
"order": 11,
"property_name": "gene_names_uniprotkb",
"tooltip": ""
},
{
"categories": [
{
"id": "Names",
"order": "1"
}
],
"default": false,
"id": "gene_names_refseq",
"immutable": false,
"label": "gene_names_refseq",
"order": 12,
"property_name": "gene_names_refseq",
"tooltip": ""
},
{
"categories": [
{
"id": "Names",
"order": "1"
}
],
"default": false,
"id": "protein_names_uniprotkb",
"immutable": false,
"label": "protein_names_uniprotkb",
"order": 1,
"property_name": "protein_names_uniprotkb",
"tooltip": ""
},
{
"categories": [
{
"id": "Names",
"order": "1"
}
],
"default": false,
"id": "protein_names_refseq",
"immutable": false,
"label": "protein_names_refseq",
"order": 1,
"property_name": "protein_names_refseq",
"tooltip": ""
},
{
"categories": [
{
"id": "Diseases",
"order": "1"
}
],
"default": false,
"id": "disease",
"immutable": false,
"label": "Disease",
"order": 1,
"property_name": "disease",
"tooltip": ""
},
{
"categories": [
{
"id": "Pathways",
"order": "1"
}
],
"default": false,
"id": "pathway",
"immutable": false,
"label": "Pathway",
"order": 1,
"property_name": "pathway",
"tooltip": ""
},
{
"categories": [
{
"id": "Publications",
"order": "1"
}
],
"default": false,
"id": "publication_count",
"immutable": false,
"label": "publication_count",
"order": 1,
"property_name": "publication_count",
"tooltip": ""
},
{
"categories": [
{
"id": "Function",
"order": "1"
}
],
"default": false,
"id": "function",
"immutable": false,
"label": "function",
"order": 1,
"property_name": "function",
"tooltip": ""
},
{
"categories": [
{
"id": "Glycosylation",
"order": "1"
}
],
"default": false,
"id": "predicted_glycosites",
"immutable": false,
"label": "predicted_glycosites",
"order": 1,
"property_name": "predicted_glycosites",
"tooltip": ""
},
{
"categories": [
{
"id": "Glycosylation",
"order": "1"
}
],
"default": false,
"id": "predicted_n_glycosites",
"immutable": false,
"label": "predicted_n_glycosites",
"order": 1,
"property_name": "predicted_n_glycosites",
"tooltip": ""
},
{
"categories": [
{
"id": "Glycosylation",
"order": "1"
}
],
"default": false,
"id": "predicted_o_glycosites",
"immutable": false,
"label": "predicted_o_glycosites",
"order": 1,
"property_name": "predicted_o_glycosites",
"tooltip": ""
},
{
"categories": [
{
"id": "Glycosylation",
"order": "1"
}
],
"default": false,
"id": "reported_n_glycosites",
"immutable": false,
"label": "reported_n_glycosites",
"order": 1,
"property_name": "reported_n_glycosites",
"tooltip": ""
},
{
"categories": [
{
"id": "Glycosylation",
"order": "1"
}
],
"default": false,
"id": "reported_o_glycosites",
"immutable": false,
"label": "reported_o_glycosites",
"order": 1,
"property_name": "reported_o_glycosites",
"tooltip": ""
},
{
"categories": [
{
"id": "Glycosylation",
"order": "1"
}
],
"default": false,
"id": "reported_n_glycosites_with_glycan",
"immutable": false,
"label": "reported_n_glycosites_with_glycan",
"order": 1,
"property_name": "reported_n_glycosites_with_glycan",
"tooltip": ""
},
{
"categories": [
{
"id": "Glycosylation",
"order": "1"
}
],
"default": false,
"id": "reported_o_glycosites_with_glycan",
"immutable": false,
"label": "reported_o_glycosites_with_glycan",
"order": 1,
"property_name": "reported_o_glycosites_with_glycan",
"tooltip": ""
},
{
"categories": [
{
"id": "Glycosylation",
"order": "1"
}
],
"default": false,
"id": "total_reported_n_glycosites",
"immutable": false,
"label": "total_reported_n_glycosites",
"order": 1,
"property_name": "total_reported_n_glycosites",
"tooltip": ""
},
{
"categories": [
{
"id": "Glycosylation",
"order": "1"
}
],
"default": false,
"id": "total_reported_o_glycosites",
"immutable": false,
"label": "total_reported_o_glycosites",
"order": 1,
"property_name": "total_reported_o_glycosites",
"tooltip": ""
},
{
"categories": [
{
"id": "Glycosylation",
"order": "1"
}
],
"default": false,
"id": "total_n_glycosites",
"immutable": false,
"label": "total_n_glycosites",
"order": 1,
"property_name": "total_n_glycosites",
"tooltip": ""
},
{
"categories": [
{
"id": "Glycosylation",
"order": "1"
}
],
"default": false,
"id": "total_o_glycosites",
"immutable": false,
"label": "total_o_glycosites",
"order": 1,
"property_name": "total_o_glycosites",
"tooltip": ""
},
{
"categories": [
{
"id": "Glycosylation",
"order": "1"
}
],
"default": false,
"id": "reported_fully_resolved_glycans",
"immutable": false,
"label": "reported_fully_resolved_glycans",
"order": 1,
"property_name": "reported_fully_resolved_glycans",
"tooltip": ""
},
{
"categories": [
{
"id": "Phosphorylation",
"order": "1"
}
],
"default": false,
"id": "reported_phosphosites",
"immutable": false,
"label": "reported_phosphosites",
"order": 1,
"property_name": "reported_phosphosites",
"tooltip": ""
},
{
"categories": [
{
"id": "Mutagensis",
"order": "1"
}
],
"default": false,
"id": "reported_mutagensis",
"immutable": false,
"label": "reported_mutagensis",
"order": 1,
"property_name": "reported_mutagensis",
"tooltip": ""
},
{
"categories": [
{
"id": "Glycation",
"order": "1"
}
],
"default": false,
"id": "reported_glycation",
"immutable": false,
"label": "reported_glycation",
"order": 1,
"property_name": "reported_glycation",
"tooltip": ""
},
{
"categories": [
{
"id": "SNV",
"order": "1"
}
],
"default": false,
"id": "reported_snv",
"immutable": false,
"label": "reported_snv",
"order": 1,
"property_name": "reported_snv",
"tooltip": ""
},
{
"categories": [
{
"id": "Interactions",
"order": "1"
}
],
"default": false,
"id": "reported_interactions",
"immutable": false,
"label": "reported_interactions",
"order": 1,
"property_name": "reported_interactions",
"tooltip": ""
},
{
"categories": [
{
"id": "Biomarkers",
"order": "1"
}
],
"default": false,
"id": "biomarker_type",
"immutable": false,
"label": "biomarker_type",
"order": 1,
"property_name": "biomarker_type",
"tooltip": ""
}
]
}
@rykahsay list_init api needs to support GET method too.
It is already doing that, I don't know why it is not working for you:
$ http GET :8082/pages/list_init/ < tests/examples/pages/list_init.json
HTTP/1.1 200 OK
Connection: close
Content-Length: 10262
Content-Type: application/json
Date: Wed, 23 Oct 2024 18:33:01 GMT
Server: gunicorn
{
"categories": [
{
"id": "General",
"label": "General",
"order": 1,
"tooltip": ""
},
{
"id": "Identifier",
"label": "Identifier",
"order": 1,
@rykahsay Below is list_init api response for "glycan" list page. Need to merge properties if id is same and label should be same in this case (id: "number_monosaccharides"), categories:[] should have all the categories in which column should be present. Also, default have different values in each case.
@sujeetvkulkarni I have already fixed this. It appears you are working with an older version of the file. This is what I see:
[sbhuiyan28@glygen-vm-dev tmp]$ awk -F, '{print $1, $2, $3, $7}' /data/projects/glygen/generated/misc/list_init_fields.csv | grep number_monosaccharides
glycan number_monosaccharides Monosaccharide Count General | Structure-Based
@sujeetvkulkarni I've also made sure the defaults match for record_type
that has the same field_id
. Here's an example:
[sbhuiyan28@glygen-vm-dev tmp]$ awk -F, '{print $1, $2, $5, $6}' /data/projects/glygen/generated/misc/list_init_fields.csv | grep uniprot_canonical_ac
protein uniprot_canonical_ac true true
site uniprot_canonical_ac true true
I think the file you are using is an older one. These changes were implemented after our call on Wednesday.
@rykahsay Can you please update the init api to use latest changes from the file above.
I have updated it now, please check. Only issue I see is (maybe it can be ignored) the field order are not starting from "1" within a given category as shown below). Ideally, the field order should be local (not global) within a category (orders 28 and 29 are global as you can see below)
$ cat generated/misc/list_init_fields.csv | grep ^protein | grep Annotation
protein,total_n_glycosites,Total N-Glycosites,28,false,false,Glycosylation | Annotation | Protein Modification,
protein,total_o_glycosites,Total O-Glycosites,29,false,false,Glycosylation | Annotation | Protein Modification,
@rykahsay @ubhuiyan Order field inside column object (not related to category) needs to be set properly for where default/immutable is true, other columns can have random order value inside column object. User will decide order for such newly added columns (apart from default). This order value needs to be decided based on existing list pages - glycan, protein, site etc or file if order value is specified. Order should 1 based index in this case. User will not be able change order for columns where immutable is set to true.
Order for specific column inside a category needs to be unique, this is how column will be displayed to the user inside a category to select.
@sujeetvkulkarni Hi Sujeet, I suggest we discuss this further during the next developer meeting if that's alright. I'd like to pull up the csv's and have you take a look and offer any additional suggestions before we provide you with the modified files.
@ubhuiyan for your reference I have added a order column in this file. This order column refers order property in column object. In case of site list there I see that all fields are marked manadorty which is wrong, in my opinion only UniProtKB Accession, HitScore, Start Pos, End Pos should be mandatory. We can definitely discuss this in dev meeting.
@sujeetvkulkarni I understand. I simply copied and pasted whatever was put in those excel files. I don't think I can make any final calls on what labels should be present and in what order. The developer meeting might be a better time to discuss especially if Rene will be present.
Notes: InChI Key in glycan init is object - should be string.
@rykahsay inchi_key is string type but in case it is empty, object ({}) is getting returned it should be empty string("").
https://api.tst.glygen.org/glycan/list?query= {"id":"a7e4068d39c1f9c017cb6f3c069e6ea5","offset":1,"sort":"hit_score","limit":20,"order":"desc","filters":[],"columns":["glytoucan_ac","mass","pubchem_id","glycoprotein_count","chebi_id","hit_score","mass_pme","motifs","keywords","inchi_key","image_url"]}
Tried to fix it, please check:
Need to implement a list init api which takes a table id as input and returns list of columns supported by the table.
The response needs to contain id, name, classification/category.
Would like to have: List of ids representing columns will be sent in list api so that backend can reduce the response size of list api.
Current plan is backend will send all column values as response to list api call and frontend will only display user selected ones.
schema for list inti api: https://api.glygen.org/pages/list_init?query={"table_id":"specific_table_id"}
response of list inti api:
With above response backend column name and frontend column will be tightly coupled.
Need to decide: Does backend need to send more/modify information in list_init_api/list_api response related to each column so that columns can be created dynamically? Looks like below additional information need to be send to make it dynamic: