Online mode

[roelj]

It seems like online-mode is possible including pressure data after all: http://forum.wacom.eu/viewtopic.php?f=11&t=18686

Does anyone have AutoDesk Sketchbook Express and could confirm that the Inkling can be used in online-mode with pressure data?

[roelj]

My guess is that the data sent in online mode is the same as the data sent in offline mode. The only thing that needs to be done to get online mode working would be a kernel driver that can initialize the online mode and pass the data to the program (or act as a mouse driver). Could it be that this would be almost identical to one of the Wacom tablets?

[xuv]

Have tested the online mode on Windows and with Autodesk Scketchbook Express, and here is some review about it.

First, I must say that I'm really excited by this online mode. The inkling is a totally usable and cheap "tablet" replacement. It's maybe a little buggy here and there. But it's a great tool to draw. And it actually works on any surface. It would be the ideal tool for me as a "portable tablet" replacement.

I was surprised to see that it works just like a mouse. The position of the pen (if picked up by the receiver) is streamed constantly, even when no pressure is applied and when the pen tip is hanging 1cm above the paper. A click and drag event is sent when pressure is applied to the pen tip (aka when drawing on the paper). A short tap of the pen on the paper is equal to a click. A double tap on the paper is equal to a double click.

After installing the Autodesk Sketchbook Express, I tried all the possible tools and parameters I could find but never seemed to have anything resembling pressure sensitivity. So, AFAIK, there is no pressure data sent. I would guess that pressure is just used to trigger "click (and drag)" or "no click".

[xuv]

Confirmation by Wacom that the online mode doesn't have pressure: http://forum.wacom.eu/viewtopic.php?f=11&t=17718#p64111

[xuv]

As stated here.

As there is no special driver installed, I think this works with mouse data and does not include pressure information.

There is no windows driver? So it behaves like a usb mouse? How do you set a usb device to behave like a mouse in linux? Would it be that easy?

[roelj]

Hmm.. I'll try using libusb to make it behave like a mouse.

[roelj]

What I've found out so far:

• The inkling device implements two interfaces, one is a "Mass Storage" interface, and the other is a "Human Input Device "HID" interface". What this means is that we can use the storage and HID simultaneously.
• The protocol is not standard, so attaching a standard mouse driver probably won't work.

I hope we can add Inkling support to the already-existing Wacom device driver. I'll look into it and try to get that driver working with the Inkling. If that works, we could have it working like a regular wacom tablet.

[marcoblancas]

Hello Roel... did you was able to use the Inkling with the online mode on Linux. I'm exited to be able to use it for my maths lectures screencasts recorded with xournal and kazam in linux.

[roelj]

Hello marcoblancas. Thanks for showing your interest.

I haven't been able to get it working. Mainly due to my lack of knowledge about the USB subsystem.

[marcoblancas]

I'm writing in the Wacom Linux Project some message related to this.

http://sourceforge.net/p/linuxwacom/mailman/linuxwacom-discuss/?viewmonth=201410&viewday=28

[roelj]

@marcoblancas That's a great move. To my understanding, when plugging in the Inkling, two modes are exposed: Mass Storage and HID. See the output of lsusb -vv on the Wacom Inkling device: http://pastebin.com/bnPqxpqD

Do you want me to enter the discussion as well. Or could you pass along this information?

[roelj]

Adding "usbhid.quirks=0x056a:0x0221:0x40000000" to the kernel line looks promising. dmesg shows it sees the HID device interface.

I'll have a look at the existing Wacom drivers and see if I can the Inkling to work with it. This might take a couple of weeks before I can give a status update though.. Any help is appreciated.

[marcoblancas]

Today I sent a new message to the thread in the Wacom for Linux Project...

http://sourceforge.net/p/linuxwacom/mailman/linuxwacom-discuss/thread/CANRwn3Q_MmmqSpDrWy-Dav_O4XfBypeakysb_kdnODo7mRo7mQ%40mail.gmail.com/

with some test they required. You can enter in the discussion if you want. I'm not a very advanced Linux user and I can just a little... testing and things like this...

[roelj]

Not much news in the last couple of days. Unfortunately I'm in the process of bringing some internal changes to InklingReader so it can handle more complex converting later.

Have you got the Inkling device to work as a USB mouse device? I tried with the hid-recorder as well but I had no success. I looked at the wacom-input driver but I couldn't really make up where I could add the device descriptors to get the Inkling to be recognized (and that would probably not even work straight away).

So once again I have to postpone this feature request.

As far as your usage. You may find the replay functionality useful along with automatic coloring of layers.. This would give you at least some control over what is shown at a specific time when doing math videos or real classroom teaching.

[roelj]

I remember the SketchManager sends a sequence of USB packets to the Inkling device before it starts sending the coordinates to the computer. I will try to "replay" these packets and see if it's just as easy as sending those packets to enable it to behave like a mouse. From there the generic mouse driver could pick it up.

[roelj]

I identified the USB HID packets sent to the device by the Wacom Inkling SketchManager software. I'm going to try to send the same packets using libusb from InklingReader. Anyone with libusb experience, please help :)

[bakercp]

Just sending you some positive energy @roelj ! Looking forward to seeing what you come up with!

[roelj]

I submitted my USB control bits. I haven't git "online-mode" to work yet. I'm trying to send the same packets as the official software, but it doesn't seem to be any of the USB HID packages (or I am doing something wrong). So I'll have to look further at which packets are missing.

[xuv]

As @bakercp, I'm also sending positive energy. I've been lurking this thread and I'm sorry I can't be much of any help, except maybe to try to look for someone with more knowledge in this matter.

[roelj]

I've been studying the USB packets and I think my approach was wrong so far. I think the Inkling can be triggered to give a coordinate using an interrupt.

A snippet of the data I can gather is the following:

02:00:00:e6:03:08:00:00:a5:0d
02:00:00:e2:03:08:00:00:a5:0e
02:00:00:dd:03:08:00:00:a5:0f
02:00:00:d9:03:08:00:00:a5:11
02:00:00:d5:03:08:00:00:a5:12
02:00:00:d1:03:08:00:00:a5:14
02:00:00:cb:03:08:00:00:a5:15
02:00:00:c5:03:08:00:00:a5:17
02:00:00:c0:03:08:00:00:a5:19
02:00:00:bd:03:08:00:00:a5:1c

These are hexadecimal values grouped in bytes. Possibly 02:00:00:d5:03 is the X and 08:00:00:a5:12 is the Y? If anyone has a good idea what this data could be, please let me know.

[roelj]

Hmm, I didn't realize this before, but there's a pretty complete interface description in the "lsusb -vv" output I posted before.

I think I'm almost at the point of getting this to work..

[roelj]

Ok guys. I think we're almost there. I succesfully received blocks of 10 bytes from the interrupt packets. There are a few problems left:

1. In order to get the packets I need to let the official software do the initialization. This can be fixed by analyzing which packets activate the HID interface and send them out from the InklingReader program. I have a complete packet log so this is just a matter of time.
2. I don't know what the blocks of 10 bytes are.. I'll try to make sense of it once I can get step 1 to work.
3. IF we know how to translate the blocks of 10 bytes to coordinates, it will only work within InklingReader. I could however attempt to modify an existing Linux mouse driver to do the custom initialization and make sense of the blocks of 10 bytes.

In conclusion: I think I made progress and I will continue my work this weekend to get step 1 and 2 sorted out. Step 3 seems to be a step too far yet. About a week ago I knew very little about USB devices and controlling them, so I am probably lacking a lot of knowledge about the Linux USB subsystem (for drivers) and USB device control (all I can do now is look at the packets sent by the official software and do the exact same thing they do..).

[roelj]

Just to let you know. I figured out step 1. In my local development version I can receive data through interrupts in real time :D.

I think I've got a pretty good idea about the X and Y parts, so I'll try to plot it and see if it looks anything like my pen movement.

Whether I can get the X/Y data from it or not, I'll clean up my code and publish my findings somewhere tomorrow.

EDIT: We've got:

• X/Y data
• Stroke begin/end data
• Pen pressure data

Looks like step 2 is done: I've figured out what the data is.

I'll hack up a simple GUI to do some demo drawing.

[roelj]

I need some help to get it right. I've uploaded a plot of X/Y data. It's supposed to be a circle, but it is split in three parts. If anyone has an idea of why it could be split..

Circle plot (PDF) file.

[marcoblancas]

I don't know how to help you. I have asked for some help in the wacom for linux project. I think the can give you some ideas.

[roelj]

I subscribed to the list as well. I'll participate there to streamline the communication.

[roelj]

I figured it out. We get values between 0 and 1920 on both X and Y.

[roelj]

Could someone try it out?

1. Connect your Inkling device as usual.

2. Run the latest version with:

   ./inklingreader --online-mode

3. Draw with your pen. Now Touch / X / Y / Pressure / Tilt X / Tilt Y data should appear in your terminal screen.

This will probably only work on GNU/Linux..

[marcoblancas]

Hello Roel. I was not able to compile in Ubuntu 14.04 because gtk version is lower... you know #37 But my collogue with debian was able and it's great to see it works and we receive the coordinates.

We saw pressure, x, y and xtilt an ytilt ( I'm not sure what are xtilt an y tilt).

Great job...

[roelj]

On the Linux Wacom mailing list Chris gave me some valuable information on where to go on from this point. I'm a bit pre-occupied right now, but in the weekend I will probably be able to come up with solutions.

Good to see someone else being able to receive the coordinates too. That means the handshaking is not limited to my device only!

[roelj]

I haven't been able to get a working driver or an emulated device with libevemu. I haven't given up yet, but I just haven't been able to get it working this weekend.

[roelj]

I finally found a way to get the online-mode working (at least partly) as a virtual mouse device. This only works on systems running the Linux kernel (I think). Unfortunately there is no pressure support yet. Even though the information can be gathered from the Inkling device, I don't see a way to expose it to the virtual mouse driver.

Could someone please use it and post the results?

EDIT: To use it run these commands (as root): modprobe uinput ./inklingreader -j

[roelj]

Pressure and tilt are now exposed to the uinput driver as well. Does anyone know how to write udev rules so InklingReader doesn't need root access to use online-mode? If so, then I can integrate it into the graphical user interface.

[xuv]

This is such a great news. I'm in the process of moving from one place to another now. So can't really test anything as my inkling reader is in a box for now. But I'm impatient to test this as soon as possible.

[roelj]

I'm looking forward to see whether it works for you.

Today I made some progress on the integration with the existing wacom kernel driver (because that would be much more awesome). Benjamin Tissoires is providing help to achieve this.

[xuv]

@roelj Thanks for the notification. Integration in the existing Wacom Driver make me even more impatient. Not related, but if you have some time or think this could be interesting, you might want to submit a proposal to #LGM2015 regarding the development of this project: http://libregraphicsmeeting.org/2015/submit-a-proposal-for-lgm-2015/

[marcoblancas]

Great job Roel. I was able to compile with no problem on Ubuntu 14.10 and I have tested the online mode too. It's fantactic... my cursor moves with my pencil but the movement is not very smooth and is no usable at the moment to write texts or resolve math problems for my screencast. By other hand, everything seems to be working OK while I have my pencil in the area recognized but if I put the pencil far an later I write over my draw the same position is not detected. I have recorded a video with my test... you can see that the circle I draw over the mouse in the paper it's draw between the eyes in my picture.

http://youtu.be/nNyouNvzUQM

It's strange... but the mouse settings in ubuntu for the slow or fast movement are affecting to the pencil smoth.

I was not able to test the presure in Xournal. With bamboo wacom tablet it works in xournal but with wacom inkling and your online mode I always see the same presure on the screen...

I can't help you with the UDEV... may be next week I could ask to somebody here.

Thank you very much, great job.

[roelj]

Thanks @marcoblancas for the video. I had expected a lot more trouble with getting this little bit of code to work on other GNU/Linux distributions. It seems to work smoother on your machine than it does on mine.

I'm working on a solution to all three problems: the absolute coordinate problem, the smoothness of the mouse device and pressure support.

I thought pressure support would be exposed when sending the data to the virtual mouse device, but it doesn't seem to work..

I believe the Wacom driver for the Linux kernel already has a lot of code in place to deal with these problems, so I want to give that a try first.

[roelj]

@xuv Thank you for the link to #LGM2015. Unfortunately I don't have the financial resources to go to Toronto.