Kinect Unity



After installing the Kinect v2 SDK from here http://www.microsoft.com/en-us/download/details.aspx?id=44561 you can also download the supporting Unity 3D plugins here http://go.microsoft.com/fwlink/?LinkID=513177. Note that the plugins require Unity 3D Pro and they expose APIs for Kinect for Windows core functionality, visual gesture builder and face to Unity apps. The zip file containing the Unity packages also contains two sample scenes Green Screen and Kinect View. Lets take a look at KinectView first:

Even though the Kinect plugin for Unity doesn’t work in UWP (and the Kinect cannot be plugged into a HoloLens device in any case), it can still run when deployed to Windows or when running in the IDE (in which case it is using the.NET 3.5 framework rather than the.NET Core framework). Kinect xbox one with Unity. GitHub Gist: instantly share code, notes, and snippets. Microsoft Xbox One Kinect Bundle 500GB Black Console (7UV-00239) + 2 Controllers. $132.00 0 bids + shipping. Xbox One S 500Gb SPECIAL EDITION Great Condition with Extra Assorted Games. Unity - Limited Edition (Microsoft Xbox One, 2014) BRAND NEW. $15.99 + shipping. Paper Mario: The Origami King - Standard Edition (Nintendo Switch, 2020.

So, I stumbled around a bit on the next step as initially I tried opening the KinectView scene from it’s existing location and this doesn’t seem to work very well. The result I got was that when I examined the game objects there was an error on each of the scripts. After some head-scratching I watched the video here http://channel9.msdn.com/Series/Programming-Kinect-for-Windows-v2/04 and at about 10:54 the presenter shows copying the scene locally to the current project. That fixed the issue for me so I was back up and running. You can plug in your Kinect sensor and run the game and then explore the scripts to see how the data is retrieved from the sensor and applied to GameObjects in your scene.

My idea was to create a very simple Kinect sample which will move particle systems around following the positions of the users hands. So lets step through what I did.

First, I chose File > New Project to bring up the Unity project wizard:

I added the Visual Studio Tools as I’m more comfortable editing and debugging in Visual Studio. You can download the tools from here https://visualstudiogallery.msdn.microsoft.com/20b80b8c-659b-45ef-96c1-437828fe7cf2.

Once the project is created I imported the Kinect Unity package; Assets > Import Package > Custom Package… I navigated to where I installed the Kinect package and chose Kinect.2.0.1410.19000.unitypackage.

I left everything selected and imported the various libraries and scripts. The editor shows that I have an empty scene just containing a Camera.

Kinect Unity

The first thing to do was to try to get the Kinect sensor data into the scene, so to do this I created a new c# script in the project and copied the code from the KinectView sample in the file BodySourceManager.cs. The code retrieves the sensor and opens a reader on it for the body data and also reads the current frame’s data in the scripts update() method. The update() method is called as part of the game loop so we are ‘pulling’ the Kinect skeleton data each time we are called. The class also has a method GetData() to allow the data to be accessed by other scripts.

I created an empty GameObject called BodySourceManager and associated the BodySourceManager.cs script with it so the code to set up the Kinect would be executed. Next, I created another empty GameObject and associated a new script which would get the body data and position the GameObject depending on the position of one of the joints in the skeleton. I made the joint type a variable so I could set up the positions of various GameObjects via the Unity editor. The intention was that if I then add a particle system as a child of this new GameObject then it would follow the tracked position of say, the left hand and I could add others for each joint that I wanted to track.

The code for the update loop showing how the game object position is set from the joint.

  1. void Update ()
  2. {
  3. if (_bodySourceManager null)
  4. {
  5. return;
  6. }
  7. _bodyManager = _bodySourceManager.GetComponent<BodyManager>();
  8. if (_bodyManager null)
  9. {
  10. return;
  11. }
  12. Body[] data = _bodyManager.GetData();
  13. if (data null)
  14. {
  15. return;
  16. }
  17. // get the first tracked body…
  18. foreach (var body in data)
  19. {
  20. if (body null)
  21. {
  22. continue;
  23. }
  24. if (body.IsTracked)
  25. {
  26. var pos = body.Joints[_jointType].Position;
  27. this.gameObject.transform.position =newVector3(pos.X, pos.Y, pos.Z);
  28. break;
  29. }
  30. }
  31. }
Store

The next step was to add two new GameObjects, one for each hand and set their joint types to HandLeft and HandRight and then add particle system’s to each as a child. Then the particle systems had their parameters tweaked until they looked how I wanted. It seems to me that the combination of Unity and Kinect is a pretty powerful one and I’m looking forward to seeing the results in the Windows Store. Please find the sample project here http://1drv.ms/1tyKlIT.

Tutorials // Unity3d


*main image: Kinect GreenScreen example scene

Setting up Kinect with Windows 10
– connect to pc (using adapter)
– Driver installs automatically and you are Done 🙂
– You can test it with these apps:
3d scan: https://www.microsoft.com/en-us/store/p/3d-scan/9nblggh68pmc
3d builder: https://www.microsoft.com/en-us/store/p/3d-builder/9wzdncrfj3t6

Using Kinect XBox One with Unity 5 (I tested with Unity v5.50b5)
– Download & install Kinect SDK https://www.microsoft.com/en-us/download/details.aspx?id=44561 (i’m using v2.0_1409)
– UnityPackage is here https://msdn.microsoft.com/en-us/library/dn782041.aspx (package download)
– Import Kinect.2.0.1410.19000.unitypackage into your project
– Also Copy GreenScreen/ and KinectView/ folders from the zip to your project (they contain the sample scenes)
– One shader is giving this error message: “‘Shader error in ‘DX11/GreenScreenShader’: Fragment program ‘frag’: sampler ‘SampleType’ has no matching texture and will be undefined”

Fix is in the unity forums by @Michal

Kinect Unity Sdk

– To fix rest of the errors, run Unity Api Updater if it didn’t run automatically (from menu: Assets / Run Api Updater..)
– Open sample scene: GreenScreen/MainScene or KinectView/MainScene
– Hit play and it should work!

—

More Resources:
– Kinect experiments: http://www.kinecthacks.com/

Tags: adapter, kinect, package, plugin, sdk, unity, windows, xbox

Related Posts

1 Comment + Add Comment

  • Nice one, I hope to see more of your posts! If some of you are looking for even more tips about creating a game, I recommend this blog too: blog.theknightsofunity.com

Unity

Recent posts

Discord Chat

Join UnityLibrary Discord chat https://discord.gg/cXT97hU

Recent Comments

  • on Using RenderDoc with Unity
  • on ffmpeg stream raw video into Unity Texture2D
  • on Deploy to Android device with wireless connection
  • on Vector3 maths for dummies!
  • on Waves Shader
  • on Matrix Playground Shader
  • on Vector3 maths for dummies!
  • on Flood Fill Algorithm

Azure Kinect Unity 3d

@unitycoder_com

My Tweets

Subscribe to Blog via Email

Azure Kinect

Kinect

Kinect Unity Character

Tag Cloud

Kinect Unity Plugin

Connect