Kinect for Windows v2

Kinect for Windows V2 is a powerful piece of kit that can allow developers to integrate with both new and existing projects providing a new, natural way of interacting with the software. As hardware gets updated, so does the developer tools that go with it and this means that things often change. One in particular change is the way that you enable your custom controls with the Kinect sensor.

How custom manipulation evolved

If you’re familiar with Kinect for Windows 1.7, you might recognise these lines of code that you may have used in one of your projects:

KinectRegion.AddHandPointerGripHandler(this, this.OnHandPointerGrip);
KinectRegion.AddHandPointerGripReleaseHandler(this, this.OnHandPointerGripRelease);
This will use the Kinect region to understand that the custom control that you’ve created can be manipulated with a hand pointer. You’d then write the code that would move the control you’re manipulating like you would with touch manipulations.

However, Kinect for Windows 2.0 takes a different spin on this making it easier for you to get up and running with custom, Kinect enabled controls. For the tutorial purposes, I’m going to show how you can create a manipulatable container control that you can put any content into.

Creating a manipulatable Kinect control

The first thing you’ll need to do is create two classes, one being the custom control and the other a Kinect controller.

public class DragDropElement : Decorator, IKinectControl
{
}

public class DragDropElementController : IKinectManipulatableController
{
}
The IKinectControl interface gives us two properties and a method for creating the controller. The two boolean properties are IsManiuplatable and IsPressable. You can define which you’d like Kinect to handle. For this tutorial, we will set IsManipulatable to true and IsPressable to false.
public class DragDropElement : Decorator, IKinectControl
{
    public IKinectController CreateController(IInputModel inputModel, KinectRegion kinectRegion)
    {
        return new DragDropElementController(inputModel, kinectRegion);
    }

    public bool IsManipulatable
    {
        get { return true; }
    }

    public bool IsPressable
    {
        get { return false; }
    }
}
We now have our custom control ready to be used in our XAML but we now need to set up the KinectController. The CreateController method is called when the KinectRegion detects the custom control is within the XAML that it is currently displaying. The IInputModel interface holds a reference to the custom control itself so that we can manipulate it using our controller. As you can see above, I’ve already shown which parameters our default constructor is going to take so let’s go and set that up. This is where we do the fun stuff!

Setting up the Kinect manipulations

The first thing we need to do is set up the default constructor with the two parameters we passed through from the custom control. We’ll also need to keep hold of them so we will set up some variables.

public class DragDropElementController : IKinectManipulatableController
{
    private ManipulatableModel _inputModel;
    private KinectRegion _kinectRegion;
    private DragDropElement _dragDropElement;
    private bool _disposedValue;

    public DragDropElementController(IInputModel inputModel, KinectRegion kinectRegion)
    {
        _inputModel = inputModel as ManipulatableModel;
        _kinectRegion = kinectRegion;
        _dragDropElement = _inputModel.Element as DragDropElement;
    }

    ManipulatableModel IKinectManipulatableController.ManipulatableInputModel
    {
        get { return _inputModel; }
    }

    FrameworkElement IKinectController.Element
    {
        get { return _inputModel.Element as FrameworkElement; }
    }

    protected virtual void Dispose(bool disposing)
    {
        if (!_disposedValue)
        {
            _kinectRegion = null;
            _inputModel = null;
            _dragDropElement = null;

            _disposedValue = true;
        }
    }

    void IDisposable.Dispose()
    {
        Dispose(true);
    }
}
Now that we have everything we need, we just need to add events that are fired by the ManipulatableModel for moving the control. If you’re creating a pressable control, you’ll use the PressableModel and add events for when the item is pressed on using the Kinect hand pointer.

For this tutorial, I will show you how you can pick up your control using grip and move it around a canvas. Note that when you’re moving something around a screen, it needs to be relative to something else which in this instance is going to be a parent Canvas element in our XAML.

In our constructor for the controller, we will create the event to be fired for manipulation updated as so:
_inputModel.ManipulationUpdated += OnManipulationUpdated; And then we will add out method in with the functionality to move the object relative to the parent Canvas as so:

private void OnManipulationUpdated(object sender, KinectManipulationUpdatedEventArgs e)
{
    var parent = _dragDropElement.Parent as Canvas;
    if (parent != null)
    {
        var d = e.Delta.Translation;
        var y = Canvas.GetTop(_dragDropElement);
        var x = Canvas.GetLeft(_dragDropElement);

        if (double.IsNaN(y)) y = 0;
        if (double.IsNaN(x)) x = 0;

        // Delta value is between 0.0 and 1.0 so they need to be scaled within the kinect region.
        var yD = d.Y*_kinectRegion.ActualHeight;
        var xD = d.X*_kinectRegion.ActualWidth;

        Canvas.SetTop(_dragDropElement, y + yD);
        Canvas.SetLeft(_dragDropElement, x + xD);
    }
}
You now have a custom control that is capable of being moved around a Canvas. This can be represented in your XAML like so:
 <kinectDemos:DragDropElement >
     <Rectangle Fill="Green" Height="200" Width="200" / >
 </kinectDemos:DragDropElement >
        
 <kinectDemos:DragDropElement Canvas.Left="50" Canvas.Top="100" >
     <TextBlock Text="Hello World!" FontSize="26" / >
 </kinectDemos:DragDropElement >
        
 <kinectDemos:DragDropElement Canvas.Left="200" Canvas.Top="200" >
     <CheckBox Content="Option 1" Style="{StaticResource CheckBoxStyle}" / >
 </kinectDemos:DragDropElement >
If you’re adding Kinect into an already feature complete project, you can port across your manipulation code from touch or mouse input easily with making minor tweaks to accessing the cumulative translations or velocities through the Kinect’s manipulation event arguments.

Download Kinect Custom Control Demo Source Code

If you have any questions, please leave a comment below and I will get in touch as soon as possible!

Author James Croft

James is a .NET developer for Black Marble Ltd. with over 5 years app development experience in Microsoft platforms. He enjoys blogging, helping others learn to code, making YouTube videos, spending time with his girlfriend but most of all, penguins.

More posts by James Croft

Join the discussion 7 Comments

  • Payal says:

    Hello ,
    The drag and drop works fine. I would want to drop a button in a specific area like a rectangle in the centre of the canvas, and this shouöd trigger an event and navigate to the next page. How should I implement this.

    Help!

    • FI says:

      Hi,
      You need to create the event for “Manipulation Completed” same as it is done above for “Manipulation Updated”. In the ManipulationCompleted event handler you can navigate to the next page. To know either the hand pointer is inside the rectangle or not, you need to implement “KinectPointerPoint”.

  • FI says:

    Hi,

    I have implemented a scrollviewer which can be scrolled up or down by gestures “Up” or “Down”, the user can drag elements and drop them out of scrollviewer. When a user gestures “Up” or “Down”, I use TranslateTransform and DoubleAnimation to animate the elements in the scrollviewer up or down respectively.

    When a user drags an element out of scrollviewer I want to keep the original element and drag the copy of it. Unfortunately I am not able to do so because I can’t set IInputModel.Element to the copy of original element as it is read only. In other words I can’t set IKinectController manually to other elements.

    Any solution or workaround will be appriciated. I am using WPF Framework, C# / XAML, Kinect SDK v2, .net 4.5.

  • neeta says:

    i have placed image inside dragelement.if i drag image outside canvas then its goes way.How would i restrict boundry using kinect.

  • neeta says:

    very nice!!! its worked. i can able to drag drop image.thank u so much……

  • Phil says:

    Hi James,

    great article and nice job !
    However, Have you tried to use together IsManipulatable set to true and IsPressable set to true ?
    It seems difficult to fire an event when “kinect hand” is enters or leaves a user control , and make this user control also draggable.

    Regards

Leave a comment