frontend based on JSON data from REST API

[drdhaval2785]

@funderburkjim and @gasyoun

Please find a screencast of the the frontend developed on top of the JSON API mentioned at #290 . frontend1.mp4.zip

This is the frontend which has tabs for each dictionary for given headword.

Merry christmas.

[funderburkjim]

Thanks for the Christmas presents! Very nice idea. Agree Flask a good modern tool on which to base python web applications.

Can you provide

• flask install instructions, including flask and/or other python components
• How you are serving flask, and where the 'home' url resides.
• The python programs used in your two demos.

Would love to have this locally to play with and try to understand.

Are you using the 'standard local installation' for the dictionaries? Or have you tweaked those somehow?

[drdhaval2785]

It uses standard local installation.

I am trying to note the steps. I have not tasted on any other machine. May miss a step or two.

step 1 - Install flask

pip install flask
pip install flask-restplus
pip install flask-cors
pip install indic-transliteration

Step 2 - Update cologne-apidev

cd /path/to/csl-apidev
git pull origin master

This should pull api_trial.py and a folder named frontend. Ignore the wsgi file. It is not yet working.

Step 3 - Start server

cd /path/to/csl-apidev
python api_trial.py

Step 4 - Browser

Open browser and type 127.0.0.1:5000/ It should serve the python APIs.

[drdhaval2785]

Html, js and css files for frontend are inside /path/to/csl-apidev/frontend/

[funderburkjim]

Will try this soon. Thanks for docs.

[gasyoun]

tabs for each dictionary

Mesmerizing! One interface - finally.

Very nice idea.

Not only idea, but realization - and it seems quick!

[funderburkjim]

Local installation instructions were good and complete. Now installed locally.

[drdhaval2785]

Great. Let me know your feedback after local testing. Happy new year.

[funderburkjim]

Try api_trial1.py .

Main difference is that it doesn't use the sqlite files directly, but rather uses getword_xml.php. This getword_xml module is currently not used elsewhere (e.g. not used in simple-search), so we can experiment with it freely.

getword_xml already returned json, and was able to rather simply add an html field to the returned data. Added features to it so that getword_xml could process the different api forms:

• dict,key, input, output already part of the logic (same as the getword api parameters)
• lnum added this
• regex added this
  • For regex search, made use of sqlite wildcard searches; getwordXMLClass changes our glob characters * and ? to sqlite's wildcard characters % and _, and dal.php returns search results with sqlite LIKE search.

Of course, frontend/one.html works as before, api_trial1.py starts a server with same local host url as api_trial.py. I like the one.html/one.js code. Gets a lot of functionality with a small amount of code. The use of await and fetch was new to me in one.js. Provides a nice way to do ajax calls without jquery.ajax().

One plus on this variation to api_trial is that we don't have to rewrite the xml to html conversion logic in python (or in JS). The html probably needs to be tweaked a bit, but it will probably be easiest currently to do the tweaks to already existing php code.
Note that the html returned currently doesn't contain css; although it does contain some inline styles. This is one tweak.

One other issue pertains to what records should be returned from a 'key' search. Try one.html for agre in MW, and compare to Basic display of agre.

[funderburkjim]

Made an .htaccess + php version of api_trial.

Try: https://www.sanskrit-lexicon.uni-koeln.de/api/dicts/ae/hw/dog/slp1/slp1.

This url makes sense due to an .htaccess file (see next comment)

You'll see a very ugly page, representing a json object with lots of strings containing xml/html code. It looks better with 'view source'. The development console available with flask is much prettier for playing with the api urls.

There is also a version of frontend/one_php.html that generates api urls like above, and displays.

https://www.sanskrit-lexicon.uni-koeln.de/scans/csl-apidev/frontend/one_php.html

[funderburkjim]

.htaccess for Cologne

# rewrite rule for Cologne
# ref: https://stackoverflow.com/questions/8291712/using-clean-urls-in-restful-api
RewriteEngine On
# assume this file in root directory at Cologne
RewriteRule api/(.*$) scans/csl-apidev/api_trial.php

.htaccess for xampp

# rewrite rule for XAMPP
# ref: https://stackoverflow.com/questions/8291712/using-clean-urls-in-restful-api
# assume this file in cologne directory, and csl-apidev is subfolder
RewriteEngine On
RewriteRule api/(.*$) csl-apidev/api_trial.php

[funderburkjim]

a 'disp' parameter

We could add a 'disp' parameter to the api. For example /api/dicts/ae/hw/dog/disp/basic/slp1/deva would result in a basic display of AE /api/dicts/mw/hw/krishna/disp/simple/simple/deva would get simple search of mw for krishna

/api/dicts/mw/hw/rA_a/one/slp1/deva would evoke a php version of https://www.sanskrit-lexicon.uni-koeln.de/scans/csl-apidev/frontend/one_php.html

etc.

Will wait until the dust settles a bit before further coding.

Others can play around with the two urls above to get an idea where some sticky points are. Maybe we'll soon get flask running at Cologne.

I may try to make a php display somewhat like the debug console in the flask version.

[drdhaval2785]

I would keep three separate HTMLs for basic, list, advance versions, rather than incorporate it in API call.

[funderburkjim]

Maybe develop rewrite rules for basic, etc.?

I've introduces php in two ways:

• via api_trial1.py Flask calls php
• via api_trial.php, frontend/one_php.html all php.
• the rewrite rule currently goes to the php.

What's your opinion of the php approach? Pluses and minuses?

[funderburkjim]

Time for sleep. Happy new year!

[drdhaval2785]

Reason behind ugly HTMLs as result

if (isset($_GET['callback'])) {
 header('content-type: application/json; charset=utf-8');
}
header("Access-Control-Allow-Origin: *");

We need to display the result as JSON for all cases and not only when a 'callback' payload exists. Then the browser will display it as proper JSON, and not try to parse as HTML.

[drdhaval2785]

What's your opinion of the php approach? Pluses and minuses?

Pluses

1. No dependency on any third party libraries like flask, flask-restplus, flask-cors.
2. Uses the existing code for HTML rendering, thereby handling dictionary specific quirks without hassle.

Minuses

1. One needs to maintain / update rewrite rules and php script whenever the API structure is changed.
2. Calls to database queries is slower.
3. We are still stuck with hard-coded HTMLs. We get results via API, but the same old result which was designed for specific layouts / designs in mind. This may not work well if we want our output to be mobile responsive.

[funderburkjim]

1 update rewrite rules

Disagree. The current rewrite rule is very general. It remains the same regardless of api change.

1 update php script when API changes

Definitely true. api_trial.php parses the url. Similarly, api_trial.py would have to change with changes in API. However, the Flask code takes provides a level of parsing automation, which likely would make the code update easier.

2. Calls to database queries is slower.

I doubt that php calls to sqlite are slower than python calls.
dal.php (data abstraction layer) is the php interface with sqlite.
Even in an all-python system, there should be some analogue of dal.php. That is, api_trial.py should not interface with sqlite directly, but via some middleware code.

There are python projects dealing precisely with this DAL issue; sqlalchemy. may be the best known database middleware.

3 still stuck with hard-coded HTML

Agree.

[funderburkjim]

we should review

• 99 JSON for integrating with new frontend

• 117 RESTful APIs + Clean URLs for cologne

• Suggestion for UI improvements

[funderburkjim]

3 still stuck with hard-coded HTML

I thought there was some discussion about 'web components' a couple of years ago, maybe by @vvasuki. Is the web component software sufficiently mature that we should consider writing display code in that form?

A web component is a collection of javascript, html, and css that is self-contained; it is self-contained in the sense that its insertion into a web page is comparatively easy, and doesn't interfere with other parts of the page.

[gasyoun]

One other issue pertains to what records should be returned from a 'key' search.

Jim, did you mean than can it be used as input for simple?

https://www.sanskrit-lexicon.uni-koeln.de/api/dicts/ae/hw/dog/slp1/slp1 looks ugly in browser mode, but is what I was waiting for so long. I can finally give a direct link and it has no ? in it.

So we can exclude /api/dicts/ and /hw/ from the URL, right? Or even have https://www.sanskrit-lexicon.uni-koeln.de/ae/dog supposing if nothing is given, slp1/slp1 is used by default, right?

https://www.sanskrit-lexicon.uni-koeln.de/api/dicts/mw/hw/krishna/disp/simple/simple/deva returned only "", so did not worked for now, but I like the /api/dicts/mw/hw/krishna/disp/simple/simple/deva would get simple search of mw for krishna idea for sure.

Can we have an ALL mode, where all entries from all dictinaries are on single page, without tabs split, please, as well?

Maybe we'll soon get flask running at Cologne.

Yeah, let's for it and not for PHP generation as before.

We are still stuck with hard-coded HTMLs.

The major minuse. Altough mobile responsive is not an aim, it might be a wanted side-effect.

Is the web component software sufficiently mature that we should consider writing display code in that form?

Have no expertise. Looke at https://blog.bitsrc.io/7-tools-for-developing-web-components-in-2019-1d5b7360654d Guess it will not be a disaster.

[funderburkjim]

Please see #294 for a solution for simple search.