Print Queue Management System Architecture

[MC42]

The print management queue needs to be capable of handling the intake, storage, and querying of current and past print jobs with some architectural system.

Scope This system should be capable of at the very least taking in the settings of the print job, any notes associated with it (such as quantity), and a file dropbox for storing and associating it with their Rutgers NetID. It should be capable of sending gCode to relevant printers (if this method is supported).

Architecture A web service of some sort will ask the user to log in with their NetID, and view the progress on their print jobs.

• Past jobs should be rendered with a "Done" marker of some sort, future jobs with a "Queued" mark, and an estimate if possible. Ongoing jobs should use the print spooler to evaluate how long the job has remaining by the last gCode status display update.

When a job is eligible to be printed, the gCode is loaded into the spooler, mirrored after the OctoPrint architecture for hardware resources, which will dispatch new gCode commands, and on "OK" from the printer, also inserting status commands to the front of the queue as needed to receive information and update the web interface.

Web Interface This is responsible for intake of new prints, correlating users with their print jobs, and offering a printerface to manage ongoing jobs (such as canceling, since the commands wouldn't be local to the printer to cancel.)

Potential Difficulties The individual gCode commandsets offered by printers is far from unified, and may not work across printers. For this reason we'd need to be able to infer the destination printer type from the Project File or have the user select the printer? this may be moot given that the MVP may just be a digital tracking board for our ongoing prints.

Another difficult potentially exists in the hosting workflow, especially as billing is relevant, but that may not be at this point.

MVP Takes files, stores them somehow. keeps track of who has been billed or not / whose print jobs are done successfully / had to be reprinted.

[MC42]

If we take a look at commands M27-M29, as well as M31 we can post the commands to a gcode file and not rely on it not dropping out, then delete the gCode off the "SD" afterwards. Super handy.

https://www.reprap.org/wiki/G-code#M27:_Report_SD_print_status

Whether the Ultimakers support this or not is TBD.

[MC42]

It should be fairly easy to check whether a user exists or not in CAS with this library, should Python be the basis we go with.

https://github.com/python-cas/python-cas

Also the CAS endpoints are documented here:

https://eas.rutgers.edu/?ht_kb=cas-local-implementation

[MC42]

First draft of Bootstrap-oriented framework uploaded to repo.

This may be useful, but manual click-add to "new job" will probably just work fine.

https://blueimp.github.io/jQuery-File-Upload/

[MC42]

Continued iterating the "home page" to view jobs, thoughts so far @dhayden7ru ?

[MC42]

Also do you have a photo of the pre-existing print queue slip for me to replicate digitally?

[MC42]

First draft of new job page added to repo, screenshot below. Still needs payment info recorded somewhere.

[MC42]

More information added, may still need a refactor as the project progresses given that the radio buttons are submitted as URL arguments.

[MC42]

And now we're POSTing!

[MC42]

More progress, now a visible workflow exists.

[MC42]

https://djangocas.dev/blog/python-cas-flask-example/ (example for CAS Attributes being gotten with the Python API so we can get contact emails)

Database Structure

jobs table job_name,VARCHAR job_date,DATETIME files,LONGBLOB class_id,VARCHAR status,VARCHAR submitter,VARCHAR total_cost,DECIMAL (unsigned) print_settings,JSON paid,BOOLEAN paid_method,VARCHAR

The print settings (as shown in prior screenshots) would have roughly modals configured per printer setup / stored in the JSON object and serialized / deserialized as needed. This makes forward-porting information relatively simple. The only difficulty is the cost row would have to be calculated / extracted from the JSON. Entirely possible though as a computed / virtual column.

Will keep pondering. The user model for permissions will likely involve querying the Canvas LMS API for users present in a given course for class-based submissions

[MC42]

The backend table for jobs should also have an auto-index to handle searching, given that jobs will almost always be chronologically submitted.

[MC42]

https://mariadb-corporation.github.io/mariadb-connector-python/usage.html#passing-parameters-to-sql-statements

[MC42]

https://www.reddit.com/r/3Dprinting/comments/7ehlfc/python_script_to_find_stl_dimensions/

[MC42]

Some more thought has gone into this and it may make sense to use a front-end like Vue calling against a JSON backend with authentication may make more sense than flask templating. The latter will work but it severely limits the ability to make a UI that looks pretty, depending on the use case. Will continue to research whether it's possible to use Flask-CAS in conjunction with VueJS and Vuetify (Vue Material Design implementation)

[MC42]

https://mariadb.com/resources/blog/how-to-connect-python-programs-to-mariadb/

On Ubuntu 20.04 (and likely on Debian) we also needed to install libmariadb-dev to get the command mariadb_config, which was a prerequisite for installing mariadb as a python dependency.

[Endoman123]

Upon further research and testing, we might actually want to use some type of full DBMS as alot of our data is relational.

e.g.: prints to logs, prints to printers, etc.

[MC42]

@Endoman123 We had briefly discussed that and the easiest conclusion we arrived at was that we absolutely can build a relational model for most parts of the system. Pending moving the whole thing to, say, sqlite or something functional and relational, we'd be reinventing the wheel a bit.

Do you want to look into sqlalchemy (python libary for managing relational database interaction in a transferable way) to see the feasibility?

[MC42]

https://flask-sqlalchemy.palletsprojects.com/en/2.x/models/

[Endoman123]

SQLAlchemy seems like the move; I'll do some more research on that and sqlite and try and get a good idea of what exactly we want to accomplish. Maybe we need to flesh out the entities in our architecture and the relationships involved. A diagram is probably in order.

[MC42]

I suspect we'll have our main table representing most of the json structure, with a long primary key, which joins to the status table which has an unused primary key. Can you think of other major things we need to represent relationally?

[Endoman123]

In my mind, I imagine we'd need to represent the relationships between the following:

• Printers and their currently printing jobs
• Print jobs and their log information
• Staff members (i.e.: users) and the jobs they author

[MC42]

Printers would essentially be a list with the "print job on it" and the status would have to be derived relationally from the rest of the job list.

Print jobs would be the primary "view", log information is again derived from the primary key.

Staff members isn't a model we necessary need to use; a generic account works unless this is also intended to maintain permissions and deploy for future infrastructure, which atm is beyond scope. Still not a bad idea to have the framework there, but it shouldn't be super complicated. We just need to be sure to not store passwords in paintext ever.