Closed mszoek closed 3 years ago
Yes it would! open
could be written quickly in a hacky way as a shell script using XDG and some assumptions, but it really should be backed by LS.
Consider me super interested.
Status update:
Next up:
Is there a spec/schema for launchservices.db
yet?
Is there a spec/schema for
launchservices.db
yet?
It's mostly a blob store for the serialized LSAppRecord objects with columns for things that are searched (app URLs, extensions). The main table is applications
. The extensions
table transforms file extensions to UTIs and the typemap
table maps applications to the UTIs they can accept. Currently they look like this:
CREATE TABLE applications(url text, basename text, version int, apprecord blob);
CREATE TABLE typemap(application text, rank int, uti text);
CREATE TABLE extension(ext text, uti text);
rank
allows setting primary and alternate apps for opening a file type. There's currently no support for overriding the opener for a specific file - thinking about xattrs for that.
I also still need to write the launchservicesd
which will maintain the db and eventually handle actually opening stuff. :)
For helloSystem we are trying to implement the core concept in a way as simple as possible. So I am not sure we want to start dealing with UTIs when everyone but Apple is using MIME types. What would be the arguments to rethink this?
TBH I am not sure if I will stick with UTI internally or convert them to MIME. :)
But anyway, I don't think you need to start dealing with UTIs unless you are writing an Info.plist. I can read the MimeTypes field from a XDG .desktop file to see what it opens, then manage that internally with whatever schema I use. Would that work for you?
My view on desktop files is that they are a leagcy technology to be used as a fallback if we don't get a suitable application from the LS database. I imagine simplified application bundles to supply a list of supported MIME types, like GNUstep does (but most likely in somehting more modern/simpler than XML/plists). When encountering such an application bundle, Filer could add the information to the database. Does this make sense?
launchservicesd
Why a daemon?
My view on desktop files is that they are a leagcy technology to be used as a fallback if we don't get a suitable application from the LS database. I imagine simplified application bundles to supply a list of supported MIME types,
OK. I also consider them legacy but wasn't sure what your plans were. A bundle's Info.plist can contain a list of extensions and MIME types using the keys CFBundleTypeExtensions
and CFBundleTypeMIMETypes
in its CFBundleDocumentTypes
dictionary. These are deprecated in favor of LSItemContentTypes
but LS will respect them. (Currently it only looks for CFBundleTypeExtensions
but I'll add the other if you need it.)
What did you have in mind for an alternative plist format?
launchservicesd
Why a daemon?
The launchservicesd
is responsible for watching /Applications
and other well-known places for new or changed apps and maintaining the database.
What did you have in mind for an alternative plist format?
A simple format that one would use today and that is well supported by Qt. Probably json.
The launchservicesd is responsible for watching /Applications and other well-known places for new or changed apps and maintaining the database.
Couldn't Filer maintain the database? Then no daemon would be needed. The watching never works properly anyway (at least not as the only measure) because applications can be at random paths.
One of the original Mac OS X authors wrote:
We wanted the task of installing an application to be as simple as dragging an app icon from somewhere (a server, an external drive) onto your computer's disk and be done with it. To do this, all the information that the system needed to know to handle the application was stored inside the app bundle, including its icons, its version information, the type of file it could handle, and the type of URL schemes it could handle. Then this information for 'centralized' in the Icon Services and Launch Services database. To maintain performance, applications were 'discovered' on a limited basis in a few 'well known' locations: the Applications system and user folders, and a few others, as well as automatically when the user navigated in the Finder to a directory containing an app. In practice, this worked very well.
If you're OK with Filer being responsible for calling LSRegisterURL under certain conditions, I am happy to go that way! You're right about that being how macOS really does it.
JSON plists would be perfect. Reading them from a .app
bundle would not require any changes beyond a new flavor of NSPropertyListReader
.
Looks like we have a plan then.
The only thing I am not too keen on is to introduce non-Qt dependencies in Filer. Hence I am asking for how the database looks, so that Filer can use Qt to edit the SQLite database...
There are quite a few non-Qt dependencies already - libfm, glib, menu-cache :) My concern about Filer editing the database directly is it could become hard to keep LS and Filer in sync if I need to change the schema. And perhaps there could be problems with locking. But I guess we can try it 🤷🏻♀️
How can the open
command be compiled on helloSystem?
It will build as part of the LS framework, so you just type make frameworks
at the root of the airyx source tree. If you have the other frameworks built & installed, you can instead just do make frameworks FRAMEWORKS=LaunchServices
to make it faster. The output from either will be under ${BUILDROOT}
which is normally $HOME/obj.amd64/buildroot
You may need to adjust some of the Makefiles if you don't have the Airyx clang and ld, which understand -framework
. The equivalent to clang -framework Foo
is clang -I/System/Library/Frameworks/Foo.framework/Headers -L/System/Library/Frameworks/Foo.framework/Versions/Current -lFoo -Wl,-R/System/Library/Frameworks/Foo.framework/Versions/A
Thanks. That sounds straightforward enough! :+1
Running into
cp -Rvf /tmp/airyx/Frameworks/CoreFoundation/CoreFoundation.framework /home/user/obj.amd64/buildroot/System/Library/Frameworks
cp: /home/user/obj.amd64/buildroot/System/Library/Frameworks: No such file or directory
*** Error code 1
Stop.
make[2]: stopped in /tmp/airyx/Frameworks
*** Error code 1
Stop.
make[1]: stopped in /tmp/airyx/Frameworks
*** Error code 1
Stop.
make: stopped in /tmp/airyx
Easy,
mkdir -p ~/obj.amd64/buildroot/System/Library/Frameworks
seems to fix this.
But then:
cc -fobjc-nonfragile-abi -fobjc-runtime=gnustep-2.0 -fpic -DPIC -O2 -pipe -O0 -g -D__AIRYX__ -DBSD -DPLATFORM_IS_POSIX -DGCC_RUNTIME_3 -DPLATFORM_USES_BSD_SOCKETS -I/usr/include/freetype2 -I/usr/include/fontconfig -I/usr/include/cairo -fobjc-runtime=gnustep-2.0 -fobjc-nonfragile-abi -fPIC -I.. -I../libobjc2 -I../Foundation/Headers -g -MD -MF.depend.O2Encoder_JPG.pico -MTO2Encoder_JPG.pico -std=gnu99 -fstack-protector-strong -Qunused-arguments -c O2Encoder_JPG.m -o O2Encoder_JPG.pico
O2Encoder_JPG.m:5:9: fatal error: 'jpeglib.h' file not found
#import <jpeglib.h>
FreeBSD% sudo pkg install libjpeg
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
All repositories are up to date.
pkg.real: No packages available to install matching 'libjpeg' have been found in the repositories
Wondering whether it has to do with things like -I/usr/include/cairo
(Airyx) vs. -/usr/include/cairo
(FreeBSD, helloSystem).
mkdir -p ~/obj.amd64/buildroot/System/Library/Frameworks
seems to fix this.
make prep
should also fix it; that creates all the buildroot directories that are used later.
cc -fobjc-nonfragile-abi -fobjc-runtime=gnustep-2.0 -fpic -DPIC -O2 -pipe -O0 -g -D__AIRYX__ -DBSD -DPLATFORM_IS_POSIX -DGCC_RUNTIME_3 -DPLATFORM_USES_BSD_SOCKETS -I/usr/include/freetype2 -I/usr/include/fontconfig -I/usr/include/cairo -fobjc-runtime=gnustep-2.0 -fobjc-nonfragile-abi -fPIC -I.. -I../libobjc2 -I../Foundation/Headers -g -MD -MF.depend.O2Encoder_JPG.pico -MTO2Encoder_JPG.pico -std=gnu99 -fstack-protector-strong -Qunused-arguments -c O2Encoder_JPG.m -o O2Encoder_JPG.pico O2Encoder_JPG.m:5:9: fatal error: 'jpeglib.h' file not found #import <jpeglib.h> FreeBSD% sudo pkg install libjpeg Updating FreeBSD repository catalogue... FreeBSD repository is up to date. All repositories are up to date. pkg.real: No packages available to install matching 'libjpeg' have been found in the repositories
Wondering whether it has to do with things like
-I/usr/include/cairo
(Airyx) vs.-/usr/include/cairo
(FreeBSD, helloSystem).
Ooooo.. yeah. Your stuff is probably in /usr/local
, right? I guess we can include both paths in those Makefiles. I had removed them all during my conversion of the paths to make sure nothing got left behind accidentally, but it should be safe now to put /usr/local/include
back.
That would be great, yes. Thanks!
The vanilla-clang
branch it looks like the build proceeds further, until it ends at
install -U -T package=runtime,debug -d DBusKit.framework/Versions/A/.debug/
install -U -T package=runtime,debug -o root -g wheel -m 444 -U libDBusKit.so.debug DBusKit.framework/Versions/A/.debug/
Warning: Object directory not changed from original /tmp/airyx/Frameworks/DBusKit
rm -rf DBusKit.framework/Headers/dbus
rm -f DBusKit.framework/Versions/Current/libdbus-1.so*
cp -rvf /usr/include/dbus-1.0/dbus/*.h DBusKit.framework/Headers/
cp: /usr/include/dbus-1.0/dbus/*.h: No such file or directory
*** Error code 1
It would have to use /usr/local/include/dbus-1.0/
instead...
Ah, shoot! I missed that one. Can you just change it for now and I'll come up with some kind of conditional check later?
With
sudo ln -s /usr/local/include/dbus-1.0 /usr/include
sudo ln -s /usr/local/lib/dbus-1.0 /usr/lib/
in place as a workaround, make frameworks
succeeded on FreeBSD/helloSystem. :+1:
But open
was apparently not built?
FreeBSD% find /home/user/obj.amd64 | grep open
I moved open
out to a new bin
folder which you can build (once the frameworks are installed in /System/Library/Frameworks) with make bin
.
make -C bin all install
===> open (all)
clang -O2 -pipe -o open open.m -I/usr/home/user/airyx/Frameworks/LaunchServices -I/usr/home/user/airyx/Frameworks/Foundation/Headers -L/home/user/obj.amd64/buildroot/System/Library/Frameworks/LaunchServices.framework -L/home/user/obj.amd64/buildroot/System/Library/Frameworks/Foundation.framework -lLaunchServices -lFoundation -lobjc -Wl,-R/System/Library/Frameworks/Foundation.framework/Versions/A -Wl,-R/System/Library/Frameworks/LaunchServices.framework/Versions/A
In file included from open.m:31:
/usr/home/user/airyx/Frameworks/Foundation/Headers/Foundation/Foundation.h:37:9: fatal error:
'CoreFoundation/CoreFoundation.h' file not found
#import <CoreFoundation/CoreFoundation.h>
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
*** Error code 1
Looks like /usr/home/user/airyx
is wrong?
This is so much easier with my patched clang
:)
I forgot the transitive dependencies of Foundation. Add -I${TOPDIR}/Frameworks
to the Makefile and duplicate the -L
and -Wl,-R
parts for the frameworks CoreFoundation
and CFNetwork
.
There, I just pushed the change to vanilla-clang
. Hopefully that helps!
make -C bin all install
===> open (all)
clang -O2 -pipe -o open open.m -I/usr/home/user/airyx/Frameworks -I/usr/home/user/airyx/Frameworks/LaunchServices -I/usr/home/user/airyx/Frameworks/Foundation/Headers -L/home/user/obj.amd64/buildroot/System/Library/Frameworks/LaunchServices.framework -L/home/user/obj.amd64/buildroot/System/Library/Frameworks/Foundation.framework -L/home/user/obj.amd64/buildroot/System/Library/Frameworks/CoreFoundation.framework -L/home/user/obj.amd64/buildroot/System/Library/Frameworks/CFNetwork.framework -lLaunchServices -lFoundation -lCoreFoundation -lCFNetwork -lobjc -Wl,-R/System/Library/Frameworks/Foundation.framework/Versions/A -Wl,-R/System/Library/Frameworks/CoreFoundation.framework/Versions/A -Wl,-R/System/Library/Frameworks/CFNetwork.framework/Versions/A -Wl,-R/System/Library/Frameworks/LaunchServices.framework/Versions/A
In file included from open.m:31:
In file included from /usr/home/user/airyx/Frameworks/Foundation/Foundation.h:41:
In file included from /usr/home/user/airyx/Frameworks/Foundation/NSAffineTransform.h:9:
In file included from /usr/home/user/airyx/Frameworks/Foundation/NSGeometry.h:9:
In file included from /usr/home/user/airyx/Frameworks/Foundation/Headers/Foundation/NSObject.h:9:
/usr/home/user/airyx/Frameworks/Foundation/NSObjCRuntime.h:9:9: fatal error: 'objc/objc.h' file
not found
#import <objc/objc.h>
^~~~~~~~~~~~~
1 error generated.
*** Error code 1
Stop.
make[2]: stopped in /usr/home/user/airyx/bin/open
*** Error code 1
*** Error code 1
Stop.
It is in ./Frameworks/libobjc2/objc/objc.h
...
This is so much easier with my patched clang :)
How would one get it onto a FreeBSD/helloSystem machine?
Have you actually installed the frameworks into /System/Library with make installairyx
or by copying them out of ${BUILDROOT}
? libobjc2
should end up in /usr/lib
and /usr/include
. The system isn't designed to build on FreeBSD anymore - there's too much complexity; it has been self-hosted for a while now.
How would one get it onto a FreeBSD/helloSystem machine?
Grab https://dl.cloudsmith.io/public/airyx/core/raw/files/base.txz and extract the clang-related files from /usr/bin and /usr/lib/clang. It might be enough to just extract /usr/bin/clang and /usr/bin/ld. They're built against the FreeBSD stable/12 git branch.
Doing now...
FreeBSD% sudo mkdir -p /System/Library
FreeBSD% sudo cp -r Frameworks /System/Library
cp: Frameworks/DBusKit/DBusKit.framework/Versions/Current/libdbus-1.so*: No such file or directory
cp: Frameworks/DBusKit/DBusKit.framework/Versions/A/libdbus-1.so*: No such file or directory
Still getting
In file included from /usr/home/user/airyx/Frameworks/Foundation/Headers/Foundation/NSObject.h:9:
/usr/home/user/airyx/Frameworks/Foundation/NSObjCRuntime.h:9:9: fatal error: 'objc/objc.h' file
not found
The system isn't designed to build on FreeBSD anymore - there's too much complexity; it has been self-hosted for a while now.
Just my 2 cents here, isn't this approach unnecesarily holding back Airyx and fragmenting the landscape? I mean, don't you want any of your work to land in helloSystem and/or FreeBSD proper in some form or thoe other one day? (In helloSystem, we want to stay close to FreeBSD wherever possible for this exact reason - so, no custom kernels etc.)
Still getting
libobjc2 isn't in the Frameworks folder so you would still need to install that into /usr/lib and /usr/include. Look under ${BUILDROOT}/usr for those.
Just my 2 cents here, isn't this approach unnecesarily holding back Airyx and fragmenting the landscape? I mean, don't you want any of your work to land in helloSystem and/or FreeBSD proper in some form or thoe other one day? (In helloSystem, we want to stay close to FreeBSD wherever possible for this exact reason - so, no custom kernels etc.)
I'd be delighted for this to all end up in FreeBSD and other projects upstream, but it's the opposite case with respect to "holding Airyx back". If I constrained it to being 100% FreeBSD, I would be compromising on what airyxOS is and making it harder to develop. I'm not going to do that.
Also I really doubt many "unix purists" would want these changes :)
FreeBSD% sudo ln -s /home/user/obj.amd64/buildroot/usr/include/objc /usr/include
FreeBSD% sudo ln -s /home/user/obj.amd64/buildroot/usr/lib/lib* /usr/lib/
FreeBSD% make bin
make -C bin all install
===> open (all)
clang -O2 -pipe -o open open.m -I/usr/home/user/airyx/Frameworks -I/usr/home/user/airyx/Frameworks/LaunchServices -I/usr/home/user/airyx/Frameworks/Foundation/Headers -L/home/user/obj.amd64/buildroot/System/Library/Frameworks/LaunchServices.framework -L/home/user/obj.amd64/buildroot/System/Library/Frameworks/Foundation.framework -L/home/user/obj.amd64/buildroot/System/Library/Frameworks/CoreFoundation.framework -L/home/user/obj.amd64/buildroot/System/Library/Frameworks/CFNetwork.framework -lLaunchServices -lFoundation -lCoreFoundation -lCFNetwork -lobjc -Wl,-R/System/Library/Frameworks/Foundation.framework/Versions/A -Wl,-R/System/Library/Frameworks/CoreFoundation.framework/Versions/A -Wl,-R/System/Library/Frameworks/CFNetwork.framework/Versions/A -Wl,-R/System/Library/Frameworks/LaunchServices.framework/Versions/A
In file included from open.m:31:
In file included from /usr/home/user/airyx/Frameworks/Foundation/Foundation.h:41:
In file included from /usr/home/user/airyx/Frameworks/Foundation/NSAffineTransform.h:9:
In file included from /usr/home/user/airyx/Frameworks/Foundation/NSGeometry.h:9:
In file included from /usr/home/user/airyx/Frameworks/Foundation/Headers/Foundation/NSObject.h:9:
/usr/home/user/airyx/Frameworks/Foundation/NSObjCRuntime.h:123:2: warning: MIN is already
defined, MIN(a, b) may not behave as expected. [-W#warnings]
#warning MIN is already defined, MIN(a, b) may not behave as expected.
^
/usr/home/user/airyx/Frameworks/Foundation/NSObjCRuntime.h:129:2: warning: MAX is already
defined, MAX(a, b) may not not behave as expected. [-W#warnings]
#warning MAX is already defined, MAX(a, b) may not not behave as expected.
^
2 warnings generated.
ld: error: undefined symbol: __objc_exec_class
>>> referenced by open.m
>>> /tmp/open-bfaf27.o:(.objc_load_function)
ld: error: undefined symbol: __objc_class_name_NSMutableDictionary
>>> referenced by open.m
>>> /tmp/open-bfaf27.o:(__objc_class_ref_NSMutableDictionary)
ld: error: undefined symbol: __objc_class_name_NSPlatform
>>> referenced by open.m
>>> /tmp/open-bfaf27.o:(__objc_class_ref_NSPlatform)
ld: error: undefined symbol: __objc_class_name_NSMutableArray
>>> referenced by open.m
>>> /tmp/open-bfaf27.o:(__objc_class_ref_NSMutableArray)
ld: error: undefined symbol: __objc_class_name_NSString
>>> referenced by open.m
>>> /tmp/open-bfaf27.o:(__objc_class_ref_NSString)
ld: error: undefined symbol: __objc_class_name_NSFileManager
>>> referenced by open.m
>>> /tmp/open-bfaf27.o:(__objc_class_ref_NSFileManager)
ld: error: undefined symbol: __objc_class_name_NSURL
>>> referenced by open.m
>>> /tmp/open-bfaf27.o:(__objc_class_ref_NSURL)
ld: error: undefined symbol: __objc_class_name_LSAppRecord
>>> referenced by open.m
>>> /tmp/open-bfaf27.o:(__objc_class_ref_LSAppRecord)
clang: error: linker command failed with exit code 1 (use -v to see invocation)
*** Error code 1
Stop.
make[2]: stopped in /usr/home/user/airyx/bin/open
*** Error code 1
*** Error code 1
Stop.
make: stopped in /usr/home/user/airyx
Add -lobjc
to the link. Hopefully that will be the last thing!
FreeBSD% clang -O2 -pipe -o open open.m -I/usr/home/user/airyx/Frameworks -I/usr/home/user/airyx/Frameworks/LaunchServices -I/usr/home/user/airyx/Frameworks/Foundation/Headers -L/home/user/obj.amd64/buildroot/System/Library/Frameworks/LaunchServices.framework -L/home/user/obj.amd64/buildroot/System/Library/Frameworks/Foundation.framework -L/home/user/obj.amd64/buildroot/System/Library/Frameworks/CoreFoundation.framework -L/home/user/obj.amd64/buildroot/System/Library/Frameworks/CFNetwork.framework -lLaunchServices -lFoundation -lCoreFoundation -lCFNetwork -lobjc -Wl,-R/System/Library/Frameworks/Foundation.framework/Versions/A -Wl,-R/System/Library/Frameworks/CoreFoundation.framework/Versions/A -Wl,-R/System/Library/Frameworks/CFNetwork.framework/Versions/A -Wl,-R/System/Library/Frameworks/LaunchServices.framework/Versions/A -lobjc
In file included from open.m:31:
In file included from /usr/home/user/airyx/Frameworks/Foundation/Foundation.h:41:
In file included from /usr/home/user/airyx/Frameworks/Foundation/NSAffineTransform.h:9:
In file included from /usr/home/user/airyx/Frameworks/Foundation/NSGeometry.h:9:
In file included from /usr/home/user/airyx/Frameworks/Foundation/Headers/Foundation/NSObject.h:9:
/usr/home/user/airyx/Frameworks/Foundation/NSObjCRuntime.h:123:2: warning: MIN is already
defined, MIN(a, b) may not behave as expected. [-W#warnings]
#warning MIN is already defined, MIN(a, b) may not behave as expected.
^
/usr/home/user/airyx/Frameworks/Foundation/NSObjCRuntime.h:129:2: warning: MAX is already
defined, MAX(a, b) may not not behave as expected. [-W#warnings]
#warning MAX is already defined, MAX(a, b) may not not behave as expected.
^
2 warnings generated.
ld: error: undefined symbol: __objc_exec_class
>>> referenced by open.m
>>> /tmp/open-aba98b.o:(.objc_load_function)
ld: error: undefined symbol: __objc_class_name_NSMutableDictionary
>>> referenced by open.m
>>> /tmp/open-aba98b.o:(__objc_class_ref_NSMutableDictionary)
ld: error: undefined symbol: __objc_class_name_NSPlatform
>>> referenced by open.m
>>> /tmp/open-aba98b.o:(__objc_class_ref_NSPlatform)
ld: error: undefined symbol: __objc_class_name_NSMutableArray
>>> referenced by open.m
>>> /tmp/open-aba98b.o:(__objc_class_ref_NSMutableArray)
ld: error: undefined symbol: __objc_class_name_NSString
>>> referenced by open.m
>>> /tmp/open-aba98b.o:(__objc_class_ref_NSString)
ld: error: undefined symbol: __objc_class_name_NSFileManager
>>> referenced by open.m
>>> /tmp/open-aba98b.o:(__objc_class_ref_NSFileManager)
ld: error: undefined symbol: __objc_class_name_NSURL
>>> referenced by open.m
>>> /tmp/open-aba98b.o:(__objc_class_ref_NSURL)
ld: error: undefined symbol: __objc_class_name_LSAppRecord
>>> referenced by open.m
>>> /tmp/open-aba98b.o:(__objc_class_ref_LSAppRecord)
clang: error: linker command failed with exit code 1 (use -v to see invocation)
This isn't going to work with the unmodified compiler :(
After looking closer, I realized the issue is not missing libraries - it's that clang by default emits different symbols. The symbols in libFoundation look like this:
000000000028b068 R_X86_64_64 ._OBJC_CLASS_NSMutableArray
not like this:
__objc_class_ref_NSMutableArray
You might be able to make it work by messing with the option flags -fobjc-abi-version=3 -fobjc-nonfragile-abi-version=2 -fobjc-nonfragile-abi -fblocks -fobjc-runtime=gnustep-2.0
but I had to patch clang to get the right behavior here.
This is unfortunately out of reach for me, so for now it seems like I have to live without it. Or are there packages of launch
somewhere for FreeBSD 12?
Totally understandable - it is a lot. Every new build of the frameworks, tools, etc is available at
https://api.cirrus-ci.com/v1/artifact/github/mszoek/airyx/airyx/airyx/dist/airyx.txz if that helps. I could also add targets to package up just /System/Library/Frameworks and /usr/bin/open into pkg
packages.
I could also add targets to package up just /System/Library/Frameworks and /usr/bin/open into pkg packages.
Yes, this would be tremendous.
commit 8b4c29137e441dceeddc6d3394341d7a3f6c6d62 (HEAD -> main)
Author: Zoe Knox <zoe@pixin.net>
Date: Sun Sep 26 14:25:48 2021 -0400
Build FreeBSD pkgs of frameworks and /usr/bin for helloSystem
Going in for next build! You should then be able to get the latest packages from CI at
https://api.cirrus-ci.com/v1/artifact/github/mszoek/airyx/airyx/packages.zip
Hopefully that will make things a bit easier :)
Would that imply writing something along the lines of Launch Services to find out what to open something with? We are in desperate need for something like it.