TODOs for proper Python package

[slhck]

If we want this to be come a proper Python package that users can install via pip install sujson, we need to:

• [x] Provide full flexibility for parsing command arguments instead of just config files
• [x] Remove re-used code and put it in helper/library files
• [x] Put every script into a subdirectory, e.g. sujson
• [x] make that a proper Python package with __init__.py and __main__.py
• [x] add a setup.py file
• [x] allow multiple entry points (e.g. csv_to_sujson, xls_to_sujson, sujson_to_csv, …)
• [x] provide example data as additional package data with MANIFEST.in
• [x] come up with a license
• [x] come up with Python requirements (I suggest 3.6+; 2.7 is EOL in 2020)

… and probably other things.

I am happy to take care of this, particularly if you have not done this before, it can be a quite daunting task, but I have released a couple of packages on PyPI so I can lead this task.

Here is an example for a simple, command-line-executable and pip-installable package: https://github.com/slhck/scenecut-extractor

[Qub3k]

Thank you! Indeed, I have not had a chance to prepare any Python code to become a proper package. Nevertheless, I am willing to learn how to do it and help you move through this process.

[slhck]

@Qub3k I'd still like to make some more general changes that would be needed in order to make this a standalone Python package with a few entry points (scripts that can be run by the user) and library code (things that are shared by multiple scripts).

This would include some logic to have default command line arguments and additional config options through config.json.

Also, I would like to know what kind of scripts you plan on having. I imagine that in the beginning we will have:

• csv_to_sujson — parsing tidy and non-tidy CSV files
• xls_to_sujson — re-using the same code from above, just parsing XLS sheets
• sujson_to_csv — converting to tidy CSV files (https://github.com/LucjanJanowski/translator-to-suJSON/issues/6)

Can you let me know when you merge the next PR, so that I can go ahead and re-structure a little bit? I guess that whatever my change will be, it would cause conflicts if there is any work done in between. For example, if you tell me that there are not going to be huge changes in the week after QoMEX, I can do my changes then (i.e., starting from June 10th).

Thanks for your great work on this! 👍

[slhck]

We can actually invoke more or less the same code via different commands without having to provide separate Python files. See, e.g.:

https://github.com/wireservice/csvkit/blob/master/setup.py#L55

We could have these three entrypoints all pointing to the same sujson.parser class, and implement more parsers as we go.

[Qub3k]

I like the idea with a single parser (and three entry points). 👍 I will merge the pull request no later than the beginning of next week (around Monday - Tuesday).

We do not plan any major changes soon. Only some extensions to what is already present in the repo. 🙂

[slhck]

Fixed in #11