search

clarity20 / tql

Terse Query Language
1 stars 0 forks source link

Preliminary python work #47

Closed clarity20 closed 4 years ago

clarity20 commented 5 years ago

Justification:

Compromises:

Practicalities - Major points:

clarity20 commented 5 years ago

Work has begun porting the project to Python3.

The projected workflow and toolchain is bash2py plus manual work to complete the port, cython to generate C/C++ source from the python code, and gcc/g++ or MSVC to generate objects and executable binaries.

Cython extends python with special syntax and directives that tend to generate faster code than normal python, so efficiency concerns should not be a showstopper. This includes object type hints and mechanisms to allow some sort of combination of straight-up C code and python. Cython has been tried out on Windows and Linux and found suitable. Gcc is fine on Linux but not on Cygwin. MinGW gcc has its own problems. The Microsoft compiler should be fine, but cannot be downloaded and run on the trusty lapop, which cannot be upgraded to a newer Windows. The plan is to get a new one as soon as possible and to work exclusively on Linux until then.

We would like to make the python source files smaller than the bash files on average. Obviously this means more files.

As work progresses, opportunities will arise to flesh out the automated testing considerably. This should be taken advantage of.

clarity20 commented 5 years ago

Another advantage of migrating to Python is support for inline, object-oriented database connections, with the structured result sets they provide. This would give us better options for pretty-printing result sets than the roll-your-own text munging of displayResultSet(). Here is a Python routine from oreilly.com that pretty-prints a result set the "right" way. This would also enable us to migrate away from reliance on third-party command line utilities, most notably the command-line database administrative tools such as mysql but also the GNU utilities still used in displayResultSet().

clarity20 commented 5 years ago

A new repo has been started for the Python port. Some updates to the tql project are anticipated to assist in comparing the bash and the Python code. All code will probably be placed into subdirectories and the bash library code broken up into smaller files.

clarity20 commented 4 years ago

We're finally revisiting the question of translating TQL to python.

I had put TQL on ice this past spring in order to jerry-rig a python-based application that satisfied at least some small part of Miniquery's ultimate goal: a nontrivial application that was buildable to an ELF-format binary, the sine qua non of this big process of conversion and extension.

That application took on a life of its own after I discovered the open source projects SQLAlchemy and python-prompt-toolkit, most especially the latter. The potential gains it offered Miniquery in terms of both functionality and usability were far too many and far too good to pass up, and I ended up diverting from the initial plan of translating the shellcode to python in order to create a best-in-class "jacket" in which the not-yet-translated code could later be zipped up.

The result is a command-line program that takes SQL queries and does almost everything that standard TUI programs out there do with them. So nobody can say, "Why would I give up what I already have to hop on board your train?" because nothing substantial is given up! Meanwhile our program just awaits the magic that only we can provide ;-) with a translated and extended TQL.

So now, after about 7 months working on that part of Miniquery that lies outside TQL, it's time to return at last to translation.

It would be neat to catalog all the unforeseen goodness we've added since taking this detour. But continuing to work on the thing is time better spent.