search

kitodo / kitodo-production

Kitodo.Production is a workflow management tool for mass digitization and is part of the Kitodo Digital Library Suite.
http://www.kitodo.org/software/kitodoproduction/
GNU General Public License v3.0
63 stars 63 forks source link

Extract file management #4327

Open matthias-ronge opened 3 years ago

matthias-ronge commented 3 years ago

The idea is—and we keep coming back to it—to do the file management in Production in an external unit, which runs as a daemon. The file management is very complicated to set up and does a lot with the operating system. It has to set permissions, has to execute scripts, has to manage an LDAP in order to transfer users to Samba servers from the Linux system, etc.

Tomcat is becoming more and more tightly bound into a corset by the ascending Linux distributions, which is correct at first, because Java is powerful and it can do a lot of damage, if malware attacks. You have to set extra permissions for writing to directories, for running script commands with sudo, etc.

Therefore (and because maybe it is better done in another programming language, something like C++) a separate component with a narrow interface can be implemented for file management. There is already the file management interface available for the file management in the source code, it can be solved in a standalone program. It can then also do different things and more and better, for example, when a task is marked as ended by a user, there was a requirement that the release of the symbolic link should be withdrawn until the upload is technically complete. Witch requires some kind of monitors. Something like that can be done better in your own software.

henning-gerhardt commented 3 years ago

In general your idea is not bad but with your solution with a daemon process and C++ I would run screaming away. My last approach to access a Java defined interface with an other programming language caused more trouble I could image and I will avoid and stay away to do something like that again.

Maybe this should be discussed in more detail until your idea / proposal is in a usable and implementable state.

matthias-ronge commented 3 years ago

It was just about being able to evaluate in this context whether Java is the best platform for this. If the answer to that is yes—and this may also be due to the interface—why not. The idea of a demon process that is addressed via HTTP requests from Production is however a strong friend of mine.

matthias-ronge commented 3 years ago

Perhaps it would be smart to use a standard GIT-LFS for file management. It would provide transaction security, changes and deletions would remain traceable and retractable, and GIT provides a network interface that could make the tedious configuration of Samba-with-LDAP or WebDAV unnecessary.