Switching to new core shows no results

[jchartrand]

@sfarnel

Results are coming back from the query, just not showing in the UI. maybe some difference between new core results and old core results.

Will need to track down where it's hanging up. Shouldn't take too too long I wouldn't think.

Sharon, should I give this priority? I can alternatively just leave the old core in place.

[jchartrand]

I think it might be that there are new universities in the new data, and I don't have mappings for those in my universities file (along with geocoordinates). Shall I add the missing universities?

We should probably at some point think about putting this mapping into a file on the server that people can edit when new universities are added to the data. The can-link app would then just load up that file via xhr.

[sfarnel]

Thanks @jchartrand I agree; let's make this file dynamic and editable. @danydvd can you help make this happen? Thanks!

[sfarnel]

@danydvd I see that Tunis University is still showing up in results; can we fix this?

[jchartrand]

This is probably because we are still using the original ’test’ core. Once we’ve fixed the university list, I can then update to the latest core, and I’m guessing Danoosh has removed Tunis from the data in that latest core.

On Sep 1, 2020, at 6:22 PM, Sharon Farnel notifications@github.com wrote:

@danydvd https://github.com/danydvd I see that Tunis University is still showing up in results; can we fix this?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jchartrand/can-link/issues/41#issuecomment-685164557, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEFSXICPCESJXH3NPFG4YDSDVXYVANCNFSM4QSEGTCA.

[jchartrand]

For better or worse, this is how I structured the university list. Happy to use a different structure as needed, but this could simply be saved to the server as-is. It just has to be at a public url.

 {
    "http://canlink.library.ualberta.ca/institution/University_of_Alberta":
        {
            "name":"University of Alberta",
            "coordinates": [53.5213, -113.5213]
        },
    "http://canlink.library.ualberta.ca/institution/Memorial_University_of_Newfoundland":
        {
            "name":"Memorial University of Newfoundland",
            "coordinates": [47.5720, -52.7327]
        },
    "http://canlink.library.ualberta.ca/institution/Queen's_University":
        {
            "name":"Queen's University",
            "coordinates": [44.2250, -76.4951]
        },
    "http://canlink.library.ualberta.ca/institution/Tunis_University":
        {
            "name":"Tunis University",
            "coordinates": [36.8190, 10.1658]
        },
    "http://canlink.library.ualberta.ca/institution/Université_de_Montréal":
        {
            "name":"Université de Montréal",
            "coordinates": [45.5042, -73.6181]
        },
    "http://canlink.library.ualberta.ca/institution/University_of_Ottawa":
        {
            "name":"University of Ottawa",
            "coordinates": [45.4222, -75.6824]
        },
    "http://canlink.library.ualberta.ca/institution/University_of_Regina":
        {
            "name":"University of Regina",
            "coordinates": [50.4093, -104.5861 ]
        },
    "http://canlink.library.ualberta.ca/institution/Dalhousie_University":
        {
            "name":"Dalhousie University",
            "coordinates": [44.6355, -63.5883]
        },
    "http://canlink.library.ualberta.ca/institution/Carleton_University":
        {
            "name":"Carleton University",
            "coordinates": [45.3831,-75.6976 ]
        },
    "http://canlink.library.ualberta.ca/institution/University_of_Victoria":
        {
            "name":"University of Victoria",
            "coordinates": [48.4633, -123.3118]
        },
    "http://canlink.library.ualberta.ca/institution/University_of_Waterloo":
        {
            "name":"University of Waterloo",
            "coordinates": [43.4680, -80.5373 ]
        },
    "http://canlink.library.ualberta.ca/institution/Royal_Roads_University":
        {
            "name":"Royal Roads University",
            "coordinates": [48.433998264, -123.470331452 ]
        },
    "http://canlink.library.ualberta.ca/institution/University_of_Manitoba":
        {
            "name":"University of Manitoba",
            "coordinates": [49.80603,-97.13972 ]
        },
    "http://canlink.library.ualberta.ca/institution/University_of_Ontario_Institute_of_Technology":
        {
            "name":"University of Ontario Institute of Technology",
            "coordinates": [43.89863,-78.86212]
        },
    "http://canlink.library.ualberta.ca/institution/Brock_University":
        {
            "name":"Brock University",
            "coordinates": [43.1190,-79.2490]
        },
    "http://canlink.library.ualberta.ca/institution/Concordia_University":
        {
            "name":"Concordia University",
            "coordinates": [45.4974,-73.5771]
        },
    "http://canlink.library.ualberta.ca/institution/University_of_Lethbridge":
        {
            "name":"University of Lethbridge",
            "coordinates": [49.6667,-112.8583 ]
        },
    "http://canlink.library.ualberta.ca/institution/University_of_Saskatchewan":
        {
            "name":"University of Saskatchewan",
            "coordinates": [52.1245611684,-106.626344161],

        },
    "http://canlink.library.ualberta.ca/institution/Laurentian_University":
        {
            "name":"Laurentian University",
            "coordinates": [46.4596,-80.9690]
        },
    "http://canlink.library.ualberta.ca/institution/University_of_Guelph":
        {
            "name":"University of Guelph",
            "coordinates": [ 43.5266,-80.2208]
        },
    "http://canlink.library.ualberta.ca/institution/Athabasca_University":
        {
            "name":"Athabasca University",
            "coordinates": [54.7201,-113.3020]
        },
    "http://canlink.library.ualberta.ca/institution/Morioka_University":
        {
            "name":"Morioka University",
            "coordinates": [39.7000, 141.1500]
        },
    "http://canlink.library.ualberta.ca/institution/York_University":
        {
            "name":"York University",
            "coordinates": [43.7705, -79.5022]
        },
    "http://canlink.library.ualberta.ca/institution/OCAD_University":
        {
            "name":"OCAD University",
            "coordinates": [43.651830726, -79.3880484478]
        },
    "http://canlink.library.ualberta.ca/institution/International_Design_School":
        {
            "name":"International Design School",
            "coordinates": [-6.213898,106.834788]
        },
    "http://canlink.library.ualberta.ca/institution/McGill_University":
        {
            "name":"McGill University",
            "coordinates": [45.5041,-73.5747]
        },
    "http://canlink.library.ualberta.ca/institution/Memorial_University_of_Newfoundland_Faculty_of_Medicine":
        {
            "name":"Memorial University of Newfoundland Faculty of Medicine",
            "coordinates": [47.5701510527, -52.7382937135]
        },
    "http://canlink.library.ualberta.ca/institution/University_of_Toronto":
        {
            "name":"University of Toronto",
            "coordinates": [43.656997372, -79.390331772]
        }
}

[sfarnel]

Thanks James. @danydvd can you have a look and see if any are missing?

[jchartrand]

We'll still want to move the list to the server, but for now I've added UBC to the list in my local copy, which has fixed the problem.

[jchartrand]

Status: we will move the university mapping list to the server at some point.

[danydvd]

@jchartrand I was using this script to pull coordinate from dbpedia for universities (for google maps). I can modifiy this to generate the format that you have created.

Basically, this would query the triplestore for all institutions and then a subsequent query to dbpedia. @sfarnel @jchartrand we can run this periodically to generate new mapping.

[jchartrand]

That is very cool. Sounds good to me.

On Sep 2, 2020, at 12:44 PM, Danoosh Davoodi notifications@github.com wrote:

@jchartrand https://github.com/jchartrand I was using this https://github.com/ualbertalib/metadata/blob/master/scripts/canlink-data/code/scripts/pull_from_dbpedia.py script to pull coordinate from dbpedia for universities (for google maps). I can modifiy this to generate the format that you have created.

Basically, this would query the triplestore for all institutions and then a subsequent query to dbpedia. @sfarnel https://github.com/sfarnel @jchartrand https://github.com/jchartrand we can run this periodically to generate new mapping.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jchartrand/can-link/issues/41#issuecomment-685859613, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEFSXLTKTCJE7N6QQZUHJLSDZY5HANCNFSM4QSEGTCA.

[sfarnel]

Thanks @danydvd Sounds good!

[sfarnel]

Thanks @jchartrand Yes, the logos on the landing page should only include institutions we have records for

[danydvd]

@jchartrand @sfarnel This is the output of the script. If this looks OK, I can move the json file to the react folder.

{'http://canlink.library.ualberta.ca/institution/University_of_British_Columbia': {'name': 'University of British Columbia', 'items': '47612', 'coordinates': [49.2611, -123.253]}, 'http://canlink.library.ualberta.ca/institution/University_of_Toronto': {'name': 'University of Toronto', 'items': '23497', 'coordinates': [43.6617, -79.395]}, 'http://canlink.library.ualberta.ca/institution/McGill_University': {'name': 'McGill University', 'items': '15602', 'coordinates': [13.1667, -59.5833]}, 'http://canlink.library.ualberta.ca/institution/Memorial_University_of_Newfoundland': {'name': 'Memorial University of Newfoundland', 'items': '4901', 'coordinates': [47.572, -52.7327]}, "http://canlink.library.ualberta.ca/institution/Queen's_University": {'name': "Queen's University", 'items': '4333', 'coordinates': [44.225, -76.4951]}, 'http://canlink.library.ualberta.ca/institution/Université_de_Montréal': {'name': 'Université de Montréal', 'items': '194', 'coordinates': [45.5047, -73.6128]}, 'http://canlink.library.ualberta.ca/institution/University_of_Ottawa': {'name': 'University of Ottawa', 'items': '27', 'coordinates': [45.4222, -75.6824]}, 'http://canlink.library.ualberta.ca/institution/University_of_Regina': {'name': 'University of Regina', 'items': '21', 'coordinates': [50.4156, -104.588]}, 'http://canlink.library.ualberta.ca/institution/Dalhousie_University': {'name': 'Dalhousie University', 'items': '19', 'coordinates': [44.6369, -63.5917]}, 'http://canlink.library.ualberta.ca/institution/Carleton_University': {'name': 'Carleton University', 'items': '14', 'coordinates': [45.3831, -75.6976]}, 'http://canlink.library.ualberta.ca/institution/University_of_Victoria': {'name': 'University of Victoria', 'items': '14', 'coordinates': [48.4633, -123.312]}, 'http://canlink.library.ualberta.ca/institution/University_of_Waterloo': {'name': 'University of Waterloo', 'items': '13', 'coordinates': [43.4689, -80.54]}, 'http://canlink.library.ualberta.ca/institution/Royal_Roads_University': {'name': 'Royal Roads University', 'items': '11', 'coordinates': [48.4344, -123.473]}, 'http://canlink.library.ualberta.ca/institution/University_of_Manitoba': {'name': 'University of Manitoba', 'items': '9', 'coordinates': [49.8094, -97.1328]}, 'http://canlink.library.ualberta.ca/institution/University_of_Ontario_Institute_of_Technology': {'name': 'University of Ontario Institute of Technology', 'items': '9', 'coordinates': [43.9448, -78.8917]}, 'http://canlink.library.ualberta.ca/institution/Brock_University': {'name': 'Brock University', 'items': '8', 'coordinates': [43.119, -79.249]}, 'http://canlink.library.ualberta.ca/institution/Concordia_University': {'name': 'Concordia University', 'items': '8', 'coordinates': [45.497, -73.578]}, 'http://canlink.library.ualberta.ca/institution/University_of_Saskatchewan': {'name': 'University of Saskatchewan', 'items': '8', 'coordinates': [52.1297, -106.633]}, 'http://canlink.library.ualberta.ca/institution/Laurentian_University': {'name': 'Laurentian University', 'items': '7', 'coordinates': [46.466, -80.9705]}, 'http://canlink.library.ualberta.ca/institution/University_of_Guelph': {'name': 'University of Guelph', 'items': '6', 'coordinates': [43.5333, -80.2236]}, 'http://canlink.library.ualberta.ca/institution/Athabasca_University': {'name': 'Athabasca University', 'items': '4', 'coordinates': [54.7224, -113.303]}, 'http://canlink.library.ualberta.ca/institution/York_University': {'name': 'York University', 'items': '3', 'coordinates': [43.7731, -79.5036]}, 'http://canlink.library.ualberta.ca/institution/OCAD_University': {'name': 'OCAD University', 'items': '2', 'coordinates': [43.6531, -79.3912]}}

[sfarnel]

Thanks @danydvd looks like University of Alberta is missing. other than that looks good tho

[jchartrand]

yes, looks good to me too.

[danydvd]

@sfarnel Yes I will add that manually for now as we don't have that data in the new index.

[sfarnel]

Thanks @danydvd

[danydvd]

The JSON file institutions.json has been uploaded to the server /var/www/canlink.ca/html. For now U of A has been added manually (with 0 items) until the new data is processed and indexed.

[sfarnel]

Thanks Danoosh

[sfarnel]

Done. Closing issue

[sfarnel]

Dropdown on site doesn't seem to reflect the list above

[jchartrand]

@sfarnel I have been working today on incorporating the remote list. It has been a bit more complicated because I need to download the list (easy enough) but then put it somewhere that it can be used by 5 different React components (each of which uses the list for either finding geo coords or looking up the display label). In other words, I effectively cache the list so that they don't all have to reload the list on each render. And so, I've created what's called a React context to share it across components. I'm hopeful it is now working, but have just run into a different problem which I will write a separate comment about in a second, for Danoosh.

[jchartrand]

@danydvd Danoosh,

The universities list looks good, and I can access it directly at:

http://206.167.181.124/institutions.json

but get the cross origin error when trying to request it from my local development machine.

Can you open it up to all origins?

[jchartrand]

@danydvd

Thinking about the cross origin thing, I guess you'd have to move the institutions file into its own directory so that you could enable CORS on only that one file rather than on the whole can-link web app?

It might be a good idea to move it into its own directory anyhow, because it could potentially accidentally get overwritten (where it is now) if someone blew away the whole react app, intending to rebuild it. They might not realize that the institutions.json file isn't part of the react app.

[danydvd]

@jchartrand I just open the CORS for GET and POST. Can you please check if it is working. If yes then I will then I will do your suggestion.

[jchartrand]

@danydvd https://github.com/danydvd Hi Danoosh, I am trying to test it now. The CORS seems to be working but I’m getting a different error, so am not totally yet sure. I will let you know as soon as I figure it out. Hopefully just a few minutes...

On Sep 3, 2020, at 4:43 PM, Danoosh Davoodi notifications@github.com wrote:

@jchartrand https://github.com/jchartrand I just open the CORS for GET and POST. Can you please check if it is working. If yes then I will then I will do your suggestion.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jchartrand/can-link/issues/41#issuecomment-686754997, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEFSXIDKBDC7SW3W4NMYILSD75X7ANCNFSM4QSEGTCA.

[jchartrand]

@danydvd

Turns out the problem is that javascript doesn't like the single quotes (although it didn't give me this error immediately, but rather some other less meaningful error - I had to try a couple things to first get this error):

"Unhandled Rejection (SyntaxError): JSON Parse error: Single quotes (') are not allowed in JSON"

And so, I am pretty sure the CORS change you made worked in so far as I can now pull down the data, but the single quotes have to be replaced with double quotes.

Who knew?

[danydvd]

@jchartrand I will reprocess the file and will use double quotes. That very picky on javascript's side!

[danydvd]

@jchartrand The file is replaced.

[jchartrand]

@danydvd The json is coming through fine now, so the CORS fix worked!

I will try it out within the dropdowns, etc. and let you know soon if those are all good too.

[jchartrand]

I will have to return to this in the morning, but it should be okay. There is a minor problem with the timing of the calls - when the search page opens, it fires off the solr search and gets the results back before the universities file has loaded. I just need to better coordinate the Promises from each, but that's complicated slightly by the React Context.

[jchartrand]

Okay, I've got it all working properly. All is pulling from the server-side universities file, and the call is cached and shared across all react components through a Context to reduce server-side calls from browser.

[sfarnel]

Thanks @jchartrand One final question for you and @danydvd : since the list is not dynamic in the sense that it doesn't change based on the subset of data returned from a search, can we order the list alphabetically?

[jchartrand]

I think that’s a question for @danydvd - I’m not sure how his script pulls the universities from the triple store and whether they could be alphabetized somewhere in the script.

On Sep 4, 2020, at 9:47 AM, Sharon Farnel notifications@github.com wrote:

Thanks @jchartrand https://github.com/jchartrand One final question for you and @danydvd https://github.com/danydvd : since the list is not dynamic in the sense that it doesn't change based on the subset of data returned from a search, can we order the list alphabetically?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jchartrand/can-link/issues/41#issuecomment-687156254, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEFSXK7UTW74KZ5OB35HWLSEDVWRANCNFSM4QSEGTCA.

[danydvd]

@jchartrand I think this can be done on my end. I will try and reprocess.

[danydvd]

@jchartrand The sort file has been uploaded (replaced the old one)

[jchartrand]

Excellent - thanks @danydvd

The file is dynamically pulled every time the app is loaded, so the list should immediately appear alphabetized.

[sfarnel]

Believe this is done. Closing