(Discussion) CLI command to populate the hxldash database with predictable sample data

[fititnt]

Issue started at https://github.com/SimonbJohnson/quickX3/issues/66

TL;DR: At this moment the command python manage.py migrate is used to prepare the databases (both initial and upgrades). This issue is about discussion about some additional CLI command that may be executed just after python manage.py migrate that may add fake (but predictable) data.

---

Disclaimer: at this point I don't have knowledge about potential more easy ways (or already standards on how to do it) with Python or Django. So this issue at this point may be more generic. If later I found some reference projects or documentation on this topic I may add.

All comments from here is about explaining user cases and things that may be relevant just later, but I'm adding upfront. But for anyone else reading, the 'TL;DR:' already explained what mostly likely could result from this.

Use cases

1. Aid (human) user testing

New people who may help with testing small changes, even if try to make a local copy, may actually not be fully aware on how to add all the dashboards options.

With such command, the person may actually be able to see some data and also when editing the code may be able to see if something breaks.

It may not replace human review, but it may be very useful.

2. Have potential to be used for automated testing

By having an cli command that preload the data in a predictable way, to a point of some URLs are expected to have some very specific content, it is the perfect scenario for later add automated testing. So at some point after having a command to populate the sample data, is possible to receive Pull Requests to add extra command to run and check this data.

While the sample data may have much more data (to help humans testing) depending on how the automated testing be implemented, this may force the testing code be updated more often. The good part is that sometimes this means do "a search and replace for strings" on both the sample data and the test command and may work at first try.

At least one sample data optimized for non-human

One way to optimize later these "search and replace" and also make easier automated testing is to from start have at least two spreadsheets that may be the same untill need to make more complex (or detailed) tests. The quickstart uses this one: https://docs.google.com/spreadsheets/d/1R9zfMTk7SQB8VoEp4XK0xAWtlsQcHgEvYiswZsj9YA4/edit#gid=0.

At some point on future, the sample data that would be used for automated testing could have differences like this

• Adm1 : "Admin1Header" or other prefix/suffix (this type of pattern for any column name like, as long as the name is unique and would not be generated by other parts of the test)
• Some parts of the spreadsheet start to have non-latin words that should also be unique.
  • This very likely will trigger some issues that are not form hxldash, but from average javascript libraries; some issues may be easier to fix, some not, so maybe could be better have a 3rd spreadsheet that is know to have bugs

Things that intentionally may be a good idea not change on sample data:

• Data related to P-codes (since they would require create even fake p-codes) may be better keep the names
• The numbers could be keep consistent
  • for example, if at some point there are not just one spreadsheet optmized for automated testing, places of the page that make calculations may be keep equal for years. Also the pages variations (e.g. public access, page that requires password to access) that may be using the same spreadsheets as base may have near full testing, but just the parts that changed receive may have different check.
• The URL to access each test page ideally should keep the same (if dont have a reason to change)
  • Sometimes a thing may break and take time to discover what caused the issue; since automated testing (even done on future) may actually be used to test older versions and see when sometime that was working stopped
  • This point may not be relevant if the hxldash can do redirects (if at some point is implemented pretty URLS, like /d/3-my-dashboard-name, but the older URLS, like /view/3, still redirect)
  • This also helps with documentation for the command that populate data (if the ID 1 is always the same spreadsheet UrlOfSpreadsheet, the documentation needs no update)
  • I suppose that, in practice, this mostly likely means that the primary key should not be changed if is not a complete different version

[SimonbJohnson]

Thanks for this. I'm hoping to find time this weekend to implement this at least for an online version.

[SimonbJohnson]

@fititnt - I've added the command populate_database which creates the 3 dashboards from the tutorials. It should be easy now to add further dashboards to data.json for testing purposes.

It is on branch adding-populate-database-command

Let me know if this is as you expected and I will merge.

[fititnt]

@SimonbJohnson Great! I will start testing right now.

[fititnt]

@SimonbJohnson I can confirm that the command works.

Here a test server, with link for the tree dashboards https://dash.hxl.etica.dev/view/1 https://dash.hxl.etica.dev/view/2 https://dash.hxl.etica.dev/view/3

suggestion 1: Sample with password

One thing that could be added (even if is not a different dashboard) is a sample with a pre-defined password. This later can be used as a automated test to check if the function stopped to work.

Suggestion 2: document the new command

Its a good idea to add python manage.py populate_database to the README.md. But the minimum functionality already works.

The only thing that I did not fully tested is using with non-SQLite database, but the command works.

[SimonbJohnson]

Added with suggestions https://github.com/SimonbJohnson/quickX3/pull/76