STL VTK support and dinamic tool details [WIP]

[TurBoss]

This Pull request is to discuss the 2 features which still in development

STL VTK support for load tool holder, tool, stock, vise and machine

Dynamic properties to store tool details like the required by the STL stuff (filename, model, etc..)

so basically you define in yaml the database structure example:

settings:
  tool_table.custom:
    value_type: dict
    default_value: {
      path_color: list,
      holder_model: str,
      tool_model: str,
      flutes: int,
      rpm: int,
      feed: int
    }

that can be addressed in python by

        self.tool_table_custom = getSetting('tool_table.custom')

        for setting in self.tool_table_custom.getValue():
            setting_id = 'tool-{:02d}-{}'.format(tnum, setting)
            data = self.data_manager.getData(setting_id)

            LOG.debug("{} {} {}".format(tnum, setting_id, data))

[TurBoss]

to insert data on the db here is an example:

        for i in range(1, 6):
            print(i)
            self.data_manager.setData('tool-{:02d}-{}'.format(i, 'path_color'), [0, 255, 0, 0])
            self.data_manager.setData('tool-{:02d}-{}'.format(i, 'holder_model'), "original.stl")
            self.data_manager.setData('tool-{:02d}-{}'.format(i, 'tool_model'), "original1.stl")
            self.data_manager.setData('tool-{:02d}-{}'.format(i, 'flutes'), 3)
            self.data_manager.setData('tool-{:02d}-{}'.format(i, 'rpm'), 7500)
            self.data_manager.setData('tool-{:02d}-{}'.format(i, 'feed'), 350)

[rodw-au]

Please do not use a flat file database for tool management. That would be a terrible travesty. Nobody really understands the importance of relational databases. This application is crying out for one! Please refer to this definition https://www.codecademy.com/articles/what-is-rdbms-sql#:~:text=A%20relational%20database%20is%20a,database%20is%20organized%20into%20tables.

Some attempt was made by Andy Pugh years ago and is documented in the Linuxcnc Wiki http://wiki.linuxcnc.org/cgi-bin/wiki.pl?ToolDatabase

I made an attempt to define plasma cut charts and have put it in a repo for it. https://github.com/rodw-au/plasmatools In this database, there are 9 tables. I don't have the skills to build a GUI to maintain this data (And it could be s seperate application). So it would be good to get you guys to do that and I'll define the database structure so you end up with a kickass system!

Before you rush in and code anymore, stop and regroup and start by defining all of the data (fields) that you wish to collect and then identify the data entities (tables) that this data belongs to. Then we define the relationships between each entity. Often this is done using data entity relationship diagrams. We need to think about many to one and one to many relationships when defining relationships. Then we can design the database and then the GUI to maintain it.

More on this after Christmas.

[TurBoss]

maybe move the whole persistent data manager to a 'sqlite3' db?

[silopolis]

Very much like the sqlite choice 👍 Maybe using a database abstraction layer could be interesting to not be linked to a specific db? For shop wide shared db, other dbms like MariaDB may be prefered.

That plasmacut app looks like the one running on sinumerik for mills in the video I shared earlier this week 😍

About tool data, that wiki is surely a great resource. Bonus would be compatibility with ISO 13399 / GTC.

[rodw-au]

maybe move the whole persistent data manager to a 'sqlite3' db?

I think that could be done pretty easily using Sqllite3 BLOBs (binary large objects). BLOBS can store any binary data such as images, documents, data structures and so on. So a tool table might be able to include an image of the tool. I have used them to store printer settings for a report by writing out the Windows DEV BLOCK structure to a BLOB.

Bearing in mind I don't know how or what data is being stored now.

You could save your pickled data direct into the database BLOB but there are also examples of using sqlite3.Binary to achieve similar results. This might be a quick and dirty fix to move data to SQL now so there is only one file (the SQL DB).

But it would be better long term to rewrite to use SQL tables and fields which might take more effort. Typically DB programs I've written in the past have a settings table that holds config data containing a single record that is read on startup. That record could include references to other default DB tables that need to be retrieved on startup. Eg current tool etc.

Of course once you have data in SQL, it would be fairly trivial to replace Sqlite3 with any SQL Db manager (MariaDB, Postgres et al) running anywhere, even on AWS in the cloud. But lets not get ahead of ourselves

[TurBoss]

to be reworked since #70 thank you all for the comments