search

asalamon74 / pktriggercord

Remote control for Pentax DSLR cameras
http://pktriggercord.melda.info
GNU Lesser General Public License v3.0
103 stars 38 forks source link

make fails on a Mac - sg.h not found #38

Open mccodesurfer opened 5 years ago

mccodesurfer commented 5 years ago

Hi, There seems to be a missing file in the repository. make fails with "scsi/sg.h not found" after cloning the repo and running make on a Mac.

asalamon74 commented 5 years ago

There is no SCSI Generic driver for Mac, that's why you get this error message.

And this is the reason why Mac is not supported.

mccodesurfer commented 5 years ago

That was my understanding but a guy supporting Kstars on the Mac said he had Gphoto2 supporting canon on the Mac. I guess they have some sort of USB driver for that. If he or I write that driver, could it be included with pktriggercord ?

Sent from Greg's iPhone

On Feb 5, 2019, at 2:07 AM, András Salamon notifications@github.com wrote:

There is no SCSI Generic driver for Mac, that's why you get this error message.

And this is the reason why Mac is not supported.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

asalamon74 commented 5 years ago

Maybe they are using some different kind of protocol. Can you share a link to the USB driver you are referring to.

mccodesurfer commented 5 years ago

I really don’t know much here. @rlancaste of indilib is going to going to look into it. He’s the expert. He may have something.

I’ve found some objective C code for USB access that I thought I could make a library out of, and provide a dylib. I’ve been able to access my Pentax K-5II with it but it would need some “scsi"_read/write functions to be written around it.

On Feb 5, 2019, at 2:41 PM, András Salamon notifications@github.com wrote:

Maybe they are using some different kind of protocol. Can you share a link to the USB driver you are referring to.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/asalamon74/pktriggercord/issues/38#issuecomment-460775555, or mute the thread https://github.com/notifications/unsubscribe-auth/AYVpszR7tved3V9EiuGNk_G5XgVID_Khks5vKd5NgaJpZM4aiLsg.

rlancaste commented 5 years ago

Yes, the INDI GPhoto driver is built on libgphoto2. This library builds on Linux, Mac, and even Windows. So we are able to provide INDI support on Macs and Linux for Canon, Nikon, and any other cameras that libgphoto2 supports. My understanding is that the library talks directly to the camera to instruct it to take exposures and that the issue with Pentax is that it doesn't include the ability to take exposures this way. Is that correct?

rlancaste commented 5 years ago

Does PKTrigger Cord completely control via USB?

rlancaste commented 5 years ago

So I am guessing that PKTriggerCord uses a different type of connection to the camera than libgphoto2?

asalamon74 commented 5 years ago

It seems that modifying the bsd port of the program is more promising. With a simple hack I was able to compile it on a Mac. It still has lots of warnings and need testing. I'll investigate it soon.

mccodesurfer commented 5 years ago

If you think you can gain access to the Mac USB then I'll hold off making a dylib. Let me know as soon as you can. We may incorporate pktriggercord directly into Kstars and bypass gphoto2 for Pentax.

Blueshawk commented 5 years ago

Yes, the INDI GPhoto driver is built on libgphoto2. This library builds on Linux, Mac, and even Windows. So we are able to provide INDI support on Macs and Linux for Canon, Nikon, and any other cameras that libgphoto2 supports. My understanding is that the library talks directly to the camera to instruct it to take exposures and that the issue with Pentax is that it doesn't include the ability to take exposures this way. Is that correct?

I don't have a Mac but maybe I can help explain the state of things. The way the indi_pentax driver works is ekos>>indi>>libgphoto>>pktriggercord. The indi driver wraps to libgphoto which is in turn wrapping to PkTriggercord for calls to the camera using "MSC" (usb:scsi) through the usb port to the camera instead of the PTP used by the other brands. Cheers, Ray

rlancaste commented 5 years ago

Ok, does libgphoto2 run pktriggercord as a separate program where it executes commands or does it use it as a library with functions that libgphoto2 calls?

asalamon74 commented 5 years ago

They import the source code of pktriggercord, you can find it here: https://github.com/gphoto/libgphoto2/tree/master/camlibs/pentax

They sync the source code sometimes. Occasionally there ask for changes in pktriggercord to make it easiere to import it, but it's quite rate. For instance: #22

rlancaste commented 5 years ago

Ok I got it, so the pktriggercord code in libgphoto2 just goes into one of the camlibs in the camlibs folder. You don't have to launch a separate program that gets called. Thank you for the clarification.

mccodesurfer commented 5 years ago

Thanks @asalamon74 . I looked at the camlibs/pentax folder and it doesn't include USB/SCSI driver so gphoto2 guys must provide one, which makes me wonder why its not working on the Mac, unless they also use a generic scsi driver. I'll see if I can find it but so far its looking like I'll have to write one.

asalamon74 commented 5 years ago

I created a hack to compile for Mac OS X. The macosx branch contains it: https://github.com/asalamon74/pktriggercord/tree/macosx

Basically I tried to use the BSD port of the program. Since scsiio.h was missing I copied the file from BSD. It is possible to compile the program. GTK is a bit tricky so it's better to compile it using make cli.

When I connect my camera (K-x) the system recognized it, at least I can get info using ioreg -p IOUSB -l. The BSD port was assuming /dev/rsd* device which is mising from Mac OS X. The new devices after I connect the camera: /dev/disk2, /dev/disk2s1, /dev/rdisk2, /dev/rdisk2s1. I've tried all the devices but only got Device open while querying: and IOCTL failed in query error messages. The first one can be eliminated by umounting.

I don't know too much about Mac OS X ioctl, so any help would be appreciated.

mccodesurfer commented 5 years ago

That’s excellent! I have written the ioctl part. I was just trying to test it. Seems to be working but your Mac build will help a lot. Now I can try sending actual camera commands. I copied the scsiio.h also but I think in the end I won’t need it. Question though. My stuff is written in objective C. Can I add it to pk and compile? How to compile with C ?

mccodesurfer commented 5 years ago

Cause I’m using the Xcode environment. Will gcc recognize objective C?

rlancaste commented 5 years ago

You could do this in terminal on OS X, it worked fine for me in an experiment:

mkdir ~/Desktop/test cd ~/Desktop/test git clone https://github.com/asalamon74/pktriggercord.git cd pktriggercord git checkout macosx make cli

asalamon74 commented 5 years ago

@mccodesurfer Please only use C, not objective C. Gcc might recognize objective C, but we need to compile pktriggercord on different platforms, so the best is to use only ANSI C.

mccodesurfer commented 5 years ago

May have to provide a library then because the code is not written from scratch. Uses macOS frameworks and libraries. The way I understand it. I just have to put the dylib file in the /usr/local/lib but not sure how to link it in the make file

asalamon74 commented 5 years ago

@mccodesurfer If you provide a library then the language of the library is not that important, only the few calls should be written in C.

Just hack together something with using the -I -L -l switches (-I for the directory containing the headers, -L for the directory containing the lib, -l for the library linking) and I try to clean it later.

mccodesurfer commented 5 years ago

Ok. Will do. It may take a week or two. I’m not on it as much as I’d like.

mccodesurfer commented 5 years ago

@asalamon74 I am close with this macOS driver - getting status from camera failing on commands- I haven't been able to relate your USB description "pentax_scsi_protocol.md" to this doc: https://www.beyondlogic.org/usbnutshell/usb6.shtml#SetupPacket

In particular, the pslr_init sets up ioctl for BSD. I have a similar ioctl that uses the setupPacket structure. Your cmd structure is similar (I guess) but I haven't figured out how to form it into the packet structure the macOS uses.

mccodesurfer commented 5 years ago

Any advice would be great. Thanks

asalamon74 commented 5 years ago

@mccodesurfer

pentax_scsi_protocol.md describes a higher level protocol. It describes the byte arrays we send to the camera, the bytes array read from the camera. Probably you don't really need this.

The low level interfaces define scsi_read and scsi_write methods to send/receive data.

The pslr_init tries to find the device and initialize it. Probably you are referring to the get_drive_info method called by pslr_init. Is that correct? On OpenBSD it uses ioctl, but it's possible that on Mac there are other ways to get the information. For instance the linux version reads the info from the /sys filesystem without using ioctl. Unfortunately it seems to me that there is no such filesystem on Mac.

mccodesurfer commented 5 years ago

Yes, I'm trying to emulate the low level SCSI_read, SCSI_write that call ioctl (in bsd version). In particular, I replace ioctl - this seems to be where the rubber meets the road - with some macOS interfaces. One called "DeviceRequest" use is very similar to ioctl; build a USB byte stream structure with USB byte codes (per the link I gave you) and call DeviceRequest on the structure.

From what you're saying, I guess I need to encapsulate the Pentax protocol into the DeviceRequest protocol. I was thinking they were the same level. They seamed to be the same in many respects, like the F0 and TT codes look like bmRequestType and bRequest. Since F0 code defines vendor specific codes to follow, it looks like this byte stream protocol is USB, not Pentax specific. However, F0 in particular does not exactly match the USB protocol in the link - if F0 where bmRequestType, it would be a reserved code. Just thinking out loud here.

I guess, I have to find out more about using DeviceRequest.

mccodesurfer commented 5 years ago

I decided to shift the interface with Wireshark on a pc and see that indeed there is 30 bytes or so preamble to the pentax command codes... still working on it!

mccodesurfer commented 5 years ago

Got Wireshark on Mac now. Looks like I have to make "interface" transactions, which I had working a while ago but its not working right now. Anyway, just letting you know the status.

rlancaste commented 5 years ago

Good to see you are making progress! I’m wondering if you have given any thought yet as to whether we want to get the code for controlling this into libgphoto or whether we want to make a new INDI driver for pentax entirely ? Obviously the library you are writing could work either way, but I’m just trying to plan ahead for when you finish. Personally I think the best route due to encapsulation and simplicity would be if we can get it into gphoto

mccodesurfer commented 5 years ago

Thanks Rob! I like the idea of a Pentax driver. Its been almost a year and I don't believe GPhoto2 has yet been updated with some significant fixes in pktethercord. For the future, Pentax has finally come out with an SDK for their cameras but only support releases from the first k-1 and later. But future ports from their SDK could be truly full featured.

tempelmann commented 5 years ago

Any news on this project? It's not clear to me whether there's so far an actual working Mac version of pktriggercord.

I tried rlancaste's instructions, but when I run it with sudo, I only keep getting a "Resource busy" message.

I also wonder if SIP has to be turned off for this to work (mine is off).

I'm testing on 10.13.6

rlancaste commented 5 years ago

Which command gives "resource busy"

mccodesurfer commented 5 years ago

I'm still working on it...slowly. Ran into a roadblock on the USB access and got some help on Codementor. The guy steered me in a new direction using new high level Apple API for camera access and control, which looks really easy to use but I'm just getting going on it. It seems to be undocumented - very new I guess.

tempelmann commented 5 years ago

@rlancaste I get this error when I invoke the cmd without arguments.

@mccodesurfer Typical for Apple these days. They can't keep up any more. But that API sounds like it won't use SCSI cmds, so are you sure you're not going in the wrong direction there? Or are you implementing a "driver" for that new API?

I'm still investigating the "macosx" branch. Am currently getting the project to run in Xcode to make debugging easier.

tempelmann commented 5 years ago

Here's what I've found out so far:

this: fd = open(deviceName, O_RDWR); fails always (10.13.6) with error "Device busy".

Changing it to fd = open(deviceName, O_RDWR | O_SHLOCK); makes it work.

However, the next call fails again: ioctl(fd, SCIOCCOMMAND, &screq) It gives "Inappropriate ioctl for device" as the error msg.

@rlancaste On which macOS version did you test this? And where is _IOWR('Q' documented?

This SO question about using SCSI cmds on OSX suggests we'd need a IOKit driver for that: https://stackoverflow.com/a/7349373/43615. So, am I missing something? Do I need to install an extra kext first for this to work? @rlancaste Could it be that you already have some 3rd party SCSI-supporting driver installed that facilitates this, while I do not?

An interesting thing is that the cmd ioreg -r -c IOSCSIPeripheralDeviceNub lists my attached Pentax as a SCSI device. So, macOS know about SCSI, it just doesn't let us talk to it via SCSI-Pass-Through. According to all the info I could find so far, the only (and proper) way would be to write a IOKit driver that matches on Pentax vendor codes and then injects itself as a block device driver, with providing the added functionality for SCSI ioctl calls. I've done basic IOKit drivers before, but I have no idea how to do the SCSI part of it as I do not understand the interaction of it all.

mccodesurfer commented 5 years ago

@tempelmann It looks like the API will give access to low level commands that will allow me to tunnel the camera commands through the USB interface. So its experimental.

rlancaste commented 5 years ago

So just to be clear. I just test compiled the code. I cannot actually test the code because I don't have a compatible camera. Originally the code would not compile, so I was just verifying that it could be compiled.

tempelmann commented 5 years ago

@rlancaste Oh, bummer then. That means my assessment is correct. Gotta hope @mccodesurfer can get something working, then. Good luck!

mccodesurfer commented 5 years ago

Well, I'm sorry guys but I've given up on the trying to port to the Mac. It seems every turn ends up a dead end with lack of information. I'm not prepared to put any more time into it. I think the fact that the Pentax is supported on linux is something I can live with. Since Pentax has released new software dev tools, maybe there will be another solution soon.

tempelmann commented 2 years ago

I had another look after learning how to use Apple's native SCSI APIs:

  1. There is really no pass-thru method in macOS via ioctl(), so the BSD access method won't work here.
  2. To use the native API (via <IOKit/scsi/SCSITaskLib.h>), the device must not be claimed already by a macOS driver, which is will if the Camera is in MSC mode: Then it'll present itself as a mass storage device, and macOS will take over, and thereby denying access via aforementioned SCSI APIs.

Which means we'd need to use PTP mode so that macOS won't claim the camera's access for itself. But AFAIK, pktriggercord does not work with the camera in PTP mode. So that's not an option.

So we need the camera to be in MSC mode - but then macOS takes over. A way out of this dilemma might be to make a simple IOKit KEXT that prevents macOS from inserting its mass storage driver. Technically, the kext probably won't need any code, but only a plist that reserves itself for the Vendor ID of Pentax cameras (0x25fb, I think). That should prevent Apple's IOKit drives from taking over.

But: Apple has announced that KEXTs won't be allowed in macOS in the future, for (in this case unnecesary and counterproductive) security reasons. Still, with older macOS versions (before macOS 12 "Monterey"), a KEXT might be possible, maybe even still thereafter; Apple is quite vague about this (they say that some types may still be allowed).

So, let's assume we can make such a kext. Then the next problem is that the SCSI driver won't get loaded either, I believe, and therefore the SCSI API won't be usable either.

But without the SCSI API, we'll have to talk to the Camera via the USB API instead.

So the big question is: How do we send SCSI commands over USB?

That's the part I don't understand yet. How does one use the lower level USB protocol to talk SCSI-over-USB? Are there special USB channels involved in that, or do the scsi commands get wrapped somehow? Does someone have insight in this? Or does pktriggercord even do this already in cases where there's no SCSI-pass-thru?

rlancaste commented 2 years ago

So, one thing that might help here is that we already disable the PTP Camera program because we have to do that to control Canon cameras and I think other DSLR's as well. That means that it does not get taken over by MacOS and is left open for gphoto control. That's already a part of KStars https://github.com/KDE/kstars/blob/ff1ac9cfeb1f11b4fb8725b643ae9e0c8950ab09/kstars/ekos/manager.cpp#L857

I don't know if this will help resolve the issue you mentioned here, but maybe.

tempelmann commented 2 years ago

@rlancaste Thanks for keeping an eye on this topic. The code you've shown is handling a user-owner process, though, which is easy to deal with (i.e. simply force-quit it) in order to release any hold on the PTP device. With the block storage device this is more difficult because it's macOS (i.e. the kernel) that takes over control of it. And there are no APIs to force it to give it up.

I am currently exploring a different path, though. I have been able to open the endpoints to my K-5 when it's in PTP mode (but not in MSC mode - there I can only open the "usb device" services, but not the "usb interface" services). I've also found the docs on how to send SCSI cmds over USB, at least for the "BOT" mode.

Can all supported Pentax cameras be used with pktriggercard in PTP mode, or do some of them only work with MSC?

56Tbird commented 2 years ago

I have had no luck with tethering my Pentax Cameras to Lightroom. Ricoh website was not helpful, hoping that they will work things out.

-----Original Message----- From: Thomas Tempelmann @.> To: asalamon74/pktriggercord @.> Cc: Subscribed @.***> Sent: Wed, Feb 9, 2022 4:01 pm Subject: Re: [asalamon74/pktriggercord] make fails on a Mac - sg.h not found (#38)

@rlancaste Thanks for keeping an eye on this topic. The code you've shown is handling a user-owner process, though, which is easy to deal with (i.e. simply force-quit it) in order to release any hold on the PTP device. With the block storage device this is more difficult because it's macOS (i.e. the kernel) that takes over control of it. And there are no APIs to force it to give it up.I am currently exploring a different path, though. I have been able to open the endpoints to my K-5 when it's in PTP mode (but not in MSC mode - there I can only open the "usb device" services, but not the "usb interface" services).Can all Pentax cameras be used with pktriggercard in PTP mode, or do some of them only work with MSC?— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you are subscribed to this thread.Message ID: @.***>

rlancaste commented 2 years ago

No problem, sorry I can't be more help with it, I don't have as much experience working with lower level stuff. I modified a couple of INDI drivers to make them work better on Macs, but they didn't need the kind of work you are talking about. But if you get the pktriggercord program working on Mac, I can help by getting it in the 3rdParty build and I can get it building with the rest of the MacOS build for KStars and INDI on craft. I'm currently working toward the goal of getting it to do nightly builds on the server.

tempelmann commented 2 years ago

Anything on my question whether pgtriggercord works with the camera being in PTP mode instead of MSC? I can't find a definitive answer in the docs or the code on this.

mccodesurfer commented 2 years ago

This PKTriggercord doc shows that it uses MSC - I tried to work on this a long time ago and gave up. I later employed Codementor help but it seems one would have to use ImageCapture (or its successor) to talk to the Camera. But it would be great if someone found an alternative pentax_scsi_protocol.md .