Transition ITables to the `src` layout

[mwouts]

In prevision of #245 I would like to transition ITables to the src layout.

@mahendrapaipuri do you think you could give me a hand on this?

This is what I am wondering:

• Where should the dt_for_itables node package go? Currently it is at itables/dt_for_itables, should I move it at src/dt_for_itables? Or at the root of the project? Note that the Python package uses dt_bundle.js and dt_bundle.css that are created by that package
• Can I create a src/itables_for_dash folder with more or less this content (e.g. a node package, a Python package, and optionally a R and Julia package)? Or should I move that somewhere else? I would much prefer not to create another repo for that.
• Should I move the html folder currently within itables (with the html table template) at some other location, or is the current location fine?

[mahendrapaipuri]

If you want to use src-layout, I think a sensible organization would be as follows

├── packages
│   └── dt_tables
│       ├── package.json
│       └── src
│           └── index.js
└── src
    └── itables

Basically that is how most of JupyterLab packages are organized. Then you will have to move the transpiled js assets to src/itables/js/<some_name>.js file so that you can distribute it with wheel and sdist. I mean the same strategy that we use for Jupytext.

You can keep html folder within the same folder as itables but during installation, it is more cleaner if you install it at sys.prefix/share/itables/html as they are data files and not package files. Similarly the transpiled js can be installed at sys.prefix/share/itables/js during installation.

Does it make sense to you?

[mwouts]

Thanks @mahendrapaipuri! Yes that makes perfect sense. I'll give this a try over the week-end, thanks for the advice!

[mwouts]

Hi @mahendrapaipuri, just tried something but the CI is pretty red so far :wink:

I guess I am not sure how to implement move the transpiled js assets in the pyproject.toml? (I've tried using a symbolic link but that's probably not a great idea).

Also, re the html and js files (and also the csv files), I see this recommendation to access the data files with importlib.resources. Does that comes in addition, or could it substitute your recommendation of installing the files in the share directory?

[mahendrapaipuri]

I guess I am not sure how to implement move the transpiled js assets in the pyproject.toml? (I've tried using a symbolic link but that's probably not a great idea).

This needs to be done in build script in package.json file. Something like

"scripts": {
    "build:js": "esbuild src/index.js --format=esm --bundle --outfile=dt_bundle.js --minify",
    "copy:js": "mkdir -p ../../src/itables/js && cp dt_bundle.js  ../../src/itables/js",
    "build": "npm build:js && npm copy:js"
  },

With such build script, it will move the tranpiled JS into src/itables directory. Seems like esbuild have outdir CLI option. You can check if you can directly use --outdir=../../src/itables/js instead of copying it with another copy:js script.

Also, re the html and js files (and also the csv files), I see this recommendation to access the data files with importlib.resources. Does that comes in addition, or could it substitute your recommendation of installing the files in the share directory?

This is another way of doing it as well if you would like to distribute html and js files with the package. If you would like to install them to share folder, you will have to change the source code accordingly to read those files from sys.prefix/share/itables/{html,js}. For example, here is how jupyterhub distributes its HTML templates and other JS assets.

I think a sensible approach is to distribute these files with package just like you are doing now (so that you dont need to do any major changes in src code) and once src layout works, we can use approach to install them into share folder in the second iteration. What do you think?

[mwouts]

Thanks @mahendrapaipuri ! That's perfectly clear. I like the use of multiple scripts that will let me continue to build the dt_for_itables package locally (I have published it on npm and we use it the connected mode). Thanks as well for the JupyterHub link, I will have a look!

[mwouts]

I think I am seeing a first version of this working! Thanks @mahendrapaipuri!