[EPIC] Drag and Drop support

[dr1rrb]

Epic for

https://github.com/unoplatform/uno/issues/995 (same issue but specific to the ListView) https://github.com/unoplatform/uno/issues/4274 (TabView) https://github.com/unoplatform/uno/issues/4311 (macOS) https://github.com/unoplatform/uno/issues/4312 (Wasm) https://github.com/unoplatform/uno/issues/4313 (Android) https://github.com/unoplatform/uno/issues/4314 (Tizen and Gtk) https://github.com/unoplatform/uno/issues/4315 (tooltip UI) https://github.com/unoplatform/uno/issues/4316 (Default UI / element preview) https://github.com/unoplatform/uno/issues/6293 (Wasm external to app drop)

What would you like to be added:

UIElement.< DragEnter | DragLeave | DragOver | DragStarting | Drop | DropCompleted > events, and the matching overridable methods Control.< OnDragEnter(DragEventArgs) | OnDragLeave(DragEventArgs) | OnDragOver(DragEventArgs) | OnDrop(DragEventArgs) >

Why is this needed:

Cause I like to drag and drop stuff ... and it matches the UWP API.

For which Platform:

• [x] iOS
• [x] Android
• [x] WebAssembly
• [ ] Windows

[abrull]

Are there any updates on this?

[robloo]

Funny, I just thought of checking up on this myself. It's a pretty major deficiency still in the platform.

[jeromelaban]

We've been making progress on the support for pointers events, pointer capture and other related features, making the implementation of this feature closer.

Make sure to upvote this issue to make it visible!

[robloo]

This seems more important now with the Skia backends for Linux/Win7. It seems it wouldn't be possible to use native drag-and-drop like Android/iOS in this case as Uno is rendering everything and managing the controls themselves.

[robloo]

I need this for a macOS port as its critical to the user-experience in a desktop environment. It makes more sense at this point to just implement WinUI-style drag-and-drop rather than making platform-specific code for my use case.

I'm happy to make myself available as a resource on this. If someone from nventive that has more experience cross-platform led the development it probably makes more sense than if I tackle it only for macOS right now. But I would be happy to help adding all classes, enums, etc. where they are needed. Please just let me know how I can help!

[francoistanguay]

If all goes well, @dr1rrb is starting on it this week. Talk about timing!

[robloo]

@dr1rrb I quickly outlined the events/classes used in drag/drop.

UIElement Events

• public event DragEventHandler DragEnter;
• public event DragEventHandler DragLeave;
• public event DragEventHandler DragOver;
• public event TypedEventHandler<UIElement,DragStartingEventArgs> DragStarting;
• public event DragEventHandler Drop;
• public event TypedEventHandler<UIElement,DropCompletedEventArgs> DropCompleted;

Classes

• public delegate void DragEventHandler(object sender, DragEventArgs e);
• public sealed class DragStartingEventArgs : RoutedEventArgs
• public sealed class DropCompletedEventArgs : RoutedEventArgs |
• public sealed class DragOperationDeferral
• public sealed class DragUI
• public sealed class DragUIOverride |
• public sealed class DataPackage
• public sealed class DataPackageView
• public sealed class DataPackagePropertySet : IDictionary<string,object>, IEnumerable<KeyValuePair<string,object>>
• public sealed class DataPackagePropertySetView : IEnumerable<KeyValuePair<string,object>>, IReadOnlyDictionary<string,object> |
• public enum DataPackageOperation

I know it can be tricky working with someone on features like this -- especially remotely. However, since the API and functionality are already defined it simplifies dividing up tasks. For example, I would be happy to work on DataPackage-related functionality or drag-and-drop events/classes to get the scaffolding in place before you start. Then you can focus more on connecting the pieces together. Again, just let me know if there is something you would like me to do.

[dr1rrb]

Hi @robloo thanks for your help ❤. I'm about to start the work on this.

My plan was mainly to focus to the Drag and Drop within the app, but apparently everybody want's also between apps, so lets see if we can also do that :)

So my first task would be to wire the routed events (Drag<Starting|Enter|Over|Leave>, Drop[Completed] and their args), so we will be able to fire them and they will bubble properly (monkey coding here, just a setup that I'm used to and which requires to not forget some hidden places). I think that once this work is completed we will be able to split the work.

If it makes sense for you, then I'll would focus to raise the events within the app (I expect to have some tricks to do in pointers events to raise D&D events even if the pointers events are handled, or if the drag gesture conflict with manipulations), while you could focus to convert the data from/to the native events (i.e. inter-apps) to match the UWP DataPackage and raise the managed events.

[robloo]

but apparently everybody want's also between apps

Ah, ok. I was looking at that functionality too but excluded it as it seems data-transfer/share was a slightly different feature. It gets a bit more complicated too. For sure drag-and-drop of files from the OS into an app would be great though and I use that as well.

while you could focus to convert the data from/to the native events (i.e. inter-apps) to match the UWP DataPackage and raise the managed events.

I could definitely help out with this on the mac and android. Other platforms are a bit outside my scope though and I don't have setups for them. That's why I was initially thinking I could help with some of the monkey-work creating the classes and things common for all platforms. I'll help where I can though -- I just can't invest a lot of time for platforms like Skia or Wasm that are outside my use case right now.

[dr1rrb]

Quick update:

In addition of the classes you've listed @robloo , I made a deep investigation of the UWP behavior. Here are my notes:

• The args.OriginalSource is null for DragStarting and DragStarted ==> I assume this is a bug, and I won't follow this
• The routed events are actually routed only to elements that have set the CanDrag / AllowDrop flags …
• … and in fact some events involved for D&D are not routed events at all (even if docs states the opposite ... will try to fix the MS doc)
• UWP does have a default UI for D&D !!! ==> We gonna have to make a plan for this as it implies to render a portion of the UI in a WriteableBitmap, which is not supported on WASM
• UWP automatically provides meaningfull DataPackage for some controls like TextBlock and Image
• There are some specific timing considerations to avoid conflicts with the hold gesture (context menu)
• There is a global event which requests for a drop target (for within and accross app D&D). If there is a event handler it must provide a target otherwise D&D does not work. If no handler, then the UI is used (i.e. the UIElement.Drag**)
• The DataPackage can be constructed asynchronously
• The DataPackage is not thread dependant, but deferal and event args are.
• The UI is not redered until the defereal has been completed (only minimalist UI)
• The ListView has some specific properties and event to handle the D&D between items ( CanDragItems, CanDragItemsProperty; DragItemsStarting; DragItemsCompleted; IsDragSource ==> This is considered as out-of-scope for the current work.

I've completed the wiring of the routed event and pushed it on a feature branch: https://github.com/unoplatform/uno/tree/feature/drag-n-drop if you want to have a look.

For the remaining parts, I think that the shared code to properly raise the events within the app won't be really easy to split/pair, so it would be really much more efficient if you can investigate the support for the native cross-app D&D, especially on macOS if you are setup for it 🙂 If you agree, the meeting point should probably be located in the CoreDragDropManager: when we receive an event from the OS, we create a DataPackage and push it to this manager which will then trigger the event like it will do for in-app D&D.

GitHub

unoplatform/uno

Build Mobile, Desktop and WebAssembly apps with C# and XAML. Today. Open source and professionally supported. - unoplatform/uno

[robloo]

@dr1rrb Great deep dive and info!

UWP does have a default UI for D&D

Yes, by default the draggable element is rendered to a bitmap and made semi-transparent and shown under the cursor. This is really useful but as you said I'm not sure how well this transfers to other platforms. It might be easier to attempt the glyph and caption support first in the DragUIOverride. If the only limitation on WASM is writable bitmap though I think that needs to be added. If you mean you can't render a UI element to a bitmap at all... well I wouldn't worry about it for the first implementation.

it would be really much more efficient if you can investigate the support for the native cross-app D&D

Sure, I'll investigate this for macOS to start.

[DierkDroth]

Hi,

Curious: what's the expected ETA for drag and drop on WASM?

Thanks

[dr1rrb]

@DierkDroth

The inter-app drag and drop support is expected to be merged this week (note: we didn't implemented the a ListViewBase specific events yet).

For intra-apps drag and drop, there is no ETA for each platform yet:

• Skia-WPF: It's already working and will be merged with the in-app D&D
• Skia-GTK (i.e. linux): No plan yet
• iOS: no plan yet, hopefully I'll be able to do it while babysitting my D&D PR
• Android: @robloo is working on it
• macOS: @robloo is also working on it
• WASM: no plan yet, hopefully I'll be able to do it while babysitting my D&D PR

[DierkDroth]

Thanks for the update @dr1rrb

@jeromelaban we needed intra-day drag and drop in WASM e.g. when dragging tabs on a tab view to re-arrange/re-position them. Please let me know if you guys needed specifics and I could get them from my team.

[jeromelaban]

@DierkDroth If you can open an issue with that specific scenario, it'll be easier to track. Also, make sure you're able to do it using a normal UWP app.

[DierkDroth]

@jeromelaban pls see here: https://github.com/unoplatform/uno/issues/4274 Note, we're using WCT TabViews until drag&drop works properly with WinUI3 TabViews (which it doesn't at this point): https://github.com/microsoft/microsoft-ui-xaml/issues/2715

[robloo]

• For Android I may not get to external (inter-app) drag-and-drop support. My goal there was to get it working within Uno itself and that looks complete. Those commits are already in the initial D&D PR.
• For iOS it might be best to wait until the macOS version is done. That might save some time and any improvements in one can likely apply to the other in some capacity.
• For macOS I should finish this soon (by next week). Drag-and-drop is more complex than I first thought and that combined with poor documentation for macOS, some undocumented Xamarin features that needed to be used, and very long build times on my mac dev machine (that hinder trial-and-error) have all contributed to delays.

[dr1rrb]

The intra-app drag and drop is now supported (https://github.com/unoplatform/uno/pull/4270) 🎉🎉🎉

This issues has been transformed as an EPIC to track remaining features.

[DierkDroth]

Thanks for the update @dr1rrb

[DierkDroth]

@dr1rrb I had this tested by my team. Unfortunately it's not working when dragging tabs of the WCT TabView (to re-order them).

What are we missing?

[robloo]

@DierkDroth While waiting for an answer from @dr1rrb I can add some clarification. TabView, like ListView drag-and-drop is control-specific. In other words, these controls handle drag/drop in a customized way. While intra-app, generic drag-and-drop is now complete TabView (#4274) must be added separately and the WCT TabView is a further separate but similar topic. I expect having the base drag-and-drop support complete will make these control-specific implementations much easier.

[jeromelaban]

@DierkDroth this issue https://github.com/unoplatform/uno/issues/4274 is still opened for tracking the WCT TabView. This addition of Drag and Drop is for the general behavior, now we'll look at specific behaviors for controls.

[DierkDroth]

Thanks for clarification.

[XeonG]

drag drop file upload not possible without this right?

[dr1rrb]

@XeonG Some parts of this epic has already been implemented, so far (as of 4.8).

Internal (a.k.a. intra-app)

> Supported on all platforms

By "internal" we refer to the ability to drag and drop content (could be anything that is supported by the DataPackage, including files) within the same app window.

External (a.k.a. inter-app)

> Supported only on WASM, MacOS and Skia WPF

On the other hand "external" refers to the ability to drag and drop content to and from another app. In that case we usually support only standard types like files, image, text and uri.