Missing source code: Only a README.md file

[pjotrek-b]

The source code tree only contains a README.md file - as does the fs-curator-0.4.0.tar.gz source code release file.

The documentation looks amazing - and this is exactly what I am looking for.

Grateful for any hints on how to get/use fs-curator. Thank you :)

[pjotrek-b]

@llowvxe: Wow. This pretty much looks like clear instructions to download and install malware. Thank you?

Since I'm on Linux, I dared to look at your proposed zip - which contains the following:

• x86_64-w64-ranlib.exe
• libssp-0.dll

THIS LOOKS LIKE A TRAP! - Please do not download the link proposed by @llowvxe

[KoolKeith]

I agree with pjotrek-b. The app sounds really cool, but without source code it's kind of a joke to be on github. I have been following it a long time, but never launched it once.

[pjotrek-b]

@unreadablewxy: If you are still reading issues/discussions here I would really like to get in touch with you.

I am seriously working on a very similar project, using xattrs for professional (archival) annotation purposes, as this is my job.

[unreadablewxy]

Sorry for the late reply, I disengaged for a while due to lack of interest from the public

If you wanna sync up on some designs I'm open to it.

I don't release source for my C++ projects because I am linking to proprietary libs developed by associates who don't want exposure to folks beyond our group. This github page exists to track issues, as I originally intended to integrate the project into some other more open projects.

If not having the source code is an issue then I'm afraid there's nothing I can do. The libs are much more stable & years more advanced than public libs.

[pjotrek-b]

Thanks @unreadablewxy for your reply! :smile:

So your "fs-curator" is in fact still in use and maintained? What are the proprietary-yet-more-advanced libs used for in your project?

I've discovered your project, while working on something similar: https://github.com/ArkThis/AHAlodeck

So if you've got time, I'd love to exchange experiences with using the filesystem for digital collections. So far, all my tests and prototypes seem very promising - yet I'm very slow and clumsy in coding GUIs :wink:

[unreadablewxy]

yes the program is deployed for both personal and at scale solutions by my associates & myself. Though the code has evolved to incorporate more features such as: porting to BSD; selectable hashing algorithms; more conflict resolution actions; batched file transforms (eg: invoke a cmdline to do EXIF stripping); "aliasing" (attributing multiple checksums to the same file); an audio signaturing algorithm they invented (i don't use this feature); and binary RPC through STDIO. I was collecting some of these features into a 0.5.0 release before my attention was needed elsewhere

the most notable components used by this project includes adapters for FFMPEG & OpenCV that leverage the latest features of the C++ standard (RAII, smart pointers) as well as our unique compiler extensions & QA strategy (I'm afraid I'm not at liberty to discuss the later in any further details). These libs are carefully authored to comply with licensing requirements and are kept proprietary solely because a majority of my associates believes the time cost to train contributors to meet standards out weighs the value of any would be contribution

thanks for linking your project, I see the similarities. For the sake of argument, say that the source code thing isn't a problem, what were you hoping to leverage from this project?

[pjotrek-b]

Thanks for your reply and insights. :smile:

Leveraging from this project: Exchange on experiences using xattrs for collection management.

I'm very experienced with large-scale professional data collection handling, and I currently feel like the only one in my domain (GLAM) suggesting to finally take xattrs seriously :grin:

I believe use of key/value annotation directly on the filesystem has been underestimated.

[unreadablewxy]

I agree that folks should give key/value annotations a fair shake. Its just that software for it don't really exist and people are too used to directory trees. But perhaps the newer generations who has grown accustomed to "super search" in everything can be convinced otherwise?

I'm not familiar with the acronym "GLAM" please elaborate

As for experiences on xattrs, here's some lessons my associates and I have learnt

• xattrs are typically allocated inside of FS meta data on most nix file systems, but on NTFS & reFS alternate streams (analogue to xattrs) actually cause block allocation, so lots of attrs are inefficient on windows
• i envisioned building indices based on xattr values so that prefix searches are possible using file paths, imagine creating a path /collection/ATTR1/ATTR2/ATTR3 while performance improvement depends on attribute cardinality, for attributes concerning the contents of personal media files, this approach achieves close to (log n) depth and thus allows very efficient querying with just directory enumeration APIs. Reason to do this is that disk space is cheaper than RAM
• hard links are a force multiplier in conjunction with the above. Every file is backed by file unique identifiers (UUID on windows, inode number or an emulation of it is on most nix FS) so exact same file checks can be conducted very efficiently. This allows for efficient multi-index intersection queries

[pjotrek-b]

Oh, sorry: See: Wikipedia GLAM (cultural heritage) - Galleries Libraries Archives Museums :smile:

So large scale meta+data collections - and all things that come with it :wink:

Thank you very much for your insights! I've been using xattrs on Linux-only so far (okay, and Haiku OS :yum:), and I'm fascinated!

It feels like xattrs were ahead of their time, and as underestimated/overlooked like "long filenames" before 1995.