sanskrit-lexicon / COLOGNE

Development of http://www.sanskrit-lexicon.uni-koeln.de/
18 stars 3 forks source link

HTML in sqlite #291

Open drdhaval2785 opened 4 years ago

drdhaval2785 commented 4 years ago

@funderburkjim Currently the sqlite files store key, lnum and data. data is the text as per dict.xml file.

When the user searches something from this, the PHP programs generate pretty printed HTMLs on the fly.

When we want to move towards API based structure, it is not a good idea to do so much computation on the fly.

May I request you to modify the sqlite generator program to store key, lnum, data and HTML. That way we can directly retrieve the appropriate HTML for each entry directly from database without computation.

Currently API gives 'headword', 'key2', 'lnum' and 'data' as fields. I would want it to give 'headword', 'key2', 'lnum', 'data' and 'html' fields. Thereafter any UI can directly post the html data with proper CSS without much ado. @gasyoun What is your take?

gasyoun commented 4 years ago

'html' fields

Adding 'html' field - I'm totally for it. API is the backend priority №1.

funderburkjim commented 4 years ago

modify the sqlite generator

In the first version of apidev, there was a xxxhtml.sqlite file, which had an almost-completed html file for each record. The main incompleted part related to the user-chosen output choices (Devanagari, IAST; accents or not)

I'm a bit squeamish about adapting the current system in this way. It would more than double the space requirements (html contains many tags, as well as tooltip texts for abbreviations). It takes more time to generate all the html pages.

@drdhaval2785 Did this thought cross your mind when you were thinking of extending your Flask displays to have html forms? If so, would use of existing php api (in csl-apidev) provide what you need?

A more flexible longer-term solution might to provide a Javascript library to convert from xml to html and a corresponding CSS library to provide styling. The Javascript library would essentially do what php (basicdisplay.php, basicadjust.php) does now.

It might be possible to somehow make use of advances in the software technology of web-component, but there still seems to be awkwardness in the way CSS works with web components.

drdhaval2785 commented 4 years ago

I had thought about using current php APIs. Then it seemed awkward to call an external service from the python API. Another issue with php APIs are that they require some HTML as input and gives out divs etc suited to cologne stylesheet. Therefore abandoned the idea.

I undertstand that even of we store html in sqlite, it would be for one particular input and output transliteration. Therefore, for redt of the combination, some or the other preprocessing or postprocessing.

I like your idea regarding use of a javascript library to do xml to html processing based on user specified parameters. Let us go ahead in that direction.

funderburkjim commented 4 years ago

When we want to move towards API based structure, it is not a good idea to do so much computation on the fly.

What are the downsides?

funderburkjim commented 4 years ago

Another issue with php APIs are that they require some HTML as input

This doesn't sound right - not sure what you're thinking of here.

gasyoun commented 4 years ago

I like your idea regarding use of a javascript library to do xml to html processing based on user specified parameters.

JS in the backend or frontend? Too much JS is bad, as there is no Javasciptist around, it's usually SEO unfriendly and heavily varies the display on different platforms, including mobile.

funderburkjim commented 4 years ago

it's usually SEO unfriendly and heavily varies the display on different platforms, including mobile

Need references that discuss some of these downsides to JS, so we can avoid the downsides if we go the JS route.

gasyoun commented 4 years ago

downsides to JS

Would need to know the name of the library. Other than that I must say that we have no JS-expert and it could be a burden, if we ever decide to customize it.

funderburkjim commented 3 years ago

In current reading of this issue, here is idea that occurs -- In addition to pywork/v02 make_xml.py program, write a make_html.py program that would generate as much as possible of the html code currently generated by php from the xml. And save this html in the the xxx.sqlite (instead of saving the xxx.xml in xxx.sqlite).

The php (basicDisplay.php, etc.) would then be much thinner, as most of the html would be already computed.

There would still be a few things that the html would not (could not?) contain directly --- notably the input/output/accent parameters. Possibly these run-time desiderata could be handled totally by Javascript.

drdhaval2785 commented 3 years ago

I agree. We keep the most feature rich version in HTML which goes in the sqlite i.e. with accent and all markup. Then javascript can change transliterations and show / disable accent etc.

gasyoun commented 3 years ago

Makese since. All my coders run away faster than I can catch them.