This week we are getting our hands deep into the sdk and the how we will use it to control Stargate Gunship.
We here at Code-Monkeys are taking the crawl, walk, run approach to the creation of the Perceptual Input Manager, PIM for short. The first iteration of the PIM, is simply to create a hands only approach to the current Stargate Gunship control schema. The current crawl format is simple in nature. I (John McGlothlan) always hold true to the simple coding strategy K.I.S.S. Which stands for Keep it Simple Stupid. A lot of people make the stupid pointed, aka don't be stupid, but for me it always had a slash. All code can be generated to be simple and easy to follow (Aka stupid), The major purpose of the PIM is to make controls easy to access and simple to understand. To that end, the PIM is using a Unity3d practice called “instance” where the actual class makes a static reference to itself. This makes it possible to call the class anywhere at any time without needing instantiation. Which is powerful and makes for readable code.
Currently the crawl version has the following variables available to access.
public bool perceptualEnabled; public bool collectingData; public Vector2 leftHandPosition; public Vector2 rightHandPosition; public Vector2 leftHandDelta; public Vector2 rightHandDelta; public bool leftHandOpen; public bool rightHandOpen;
This makes the ability to access your “hands” in real time without needing any more code than
PerceptualInput.instance.leftHandPosition
That is the whole requirement. This means upgrading our current control schema to include Perceptual input is normalized to simple coding schemes. Because we have made sure to include perceptualEnabled and collectingData, we don't need to run extra cycles if the camera is missing or someone is not actively using the device. Also it means we can progressively enable and disable the controls available to the user. If a perceptual camera is available use that, otherwise use a touch interface and finally if touch is missing use a keyboard and mouse.
We started PIM from the Unity example, as such I will skip a majority of the camera handshake as that can be found simply from the SDK examples. I will go through the code we are currently using to populate the public interface variables we have declared above.
foundLeft = false;
foundRight = false;
if (pp.QueryGeoNode(PXCMGesture.GeoNode.Label.LABEL_BODY_HAND_LEFT,out leftHand)) {
foundLeft = true;
}
if (pp.QueryGeoNode(PXCMGesture.GeoNode.Label.LABEL_BODY_HAND_RIGHT,out rightHand)) {
foundRight = true;
}
if (foundLeft && foundRight) {
collectingData = true;
// Determine left vs right
if (leftHand.positionImage.x > rightHand.positionImage.x) {
// Left is actually left.
leftHandDelta.x = leftHand.positionImage.x - leftHandPosition.x;
leftHandDelta.y = leftHand.positionImage.y - leftHandPosition.y;
leftHandPosition.x = leftHand.positionImage.x;
leftHandPosition.y = leftHand.positionImage.y;
if (leftHand.openness != 0) {
leftHandOpen = true;
} else {
leftHandOpen = false;
}
rightHandDelta.x = rightHand.positionImage.x - rightHandPosition.x;
rightHandDelta.y = rightHand.positionImage.y - rightHandPosition.y;
rightHandPosition.x = rightHand.positionImage.x;
rightHandPosition.y = rightHand.positionImage.y;
if (rightHand.openness != 0) {
rightHandOpen = true;
} else {
rightHandOpen = false;
}
} else {
// Camera is confused over left vs right flip data
rightHandDelta.x = leftHand.positionImage.x - rightHandPosition.x;
rightHandDelta.y = leftHand.positionImage.y - rightHandPosition.y;
rightHandPosition.x = leftHand.positionImage.x;
rightHandPosition.y = leftHand.positionImage.y;
if (leftHand.openness != 0) {
rightHandOpen = true;
} else {
rightHandOpen = false;
}
leftHandDelta.x = rightHand.positionImage.x - leftHandPosition.x;
leftHandDelta.y = rightHand.positionImage.y - leftHandPosition.y;
leftHandPosition.x = rightHand.positionImage.x;
leftHandPosition.y = rightHand.positionImage.y;
if (rightHand.openness != 0) {
leftHandOpen = true;
} else {
leftHandOpen = false;
}
}
} else if (foundLeft) {
collectingData = true;
leftHandDelta.x = leftHand.positionImage.x - leftHandPosition.x;
leftHandDelta.y = leftHand.positionImage.y - leftHandPosition.y;
leftHandPosition.x = leftHand.positionImage.x;
leftHandPosition.y = leftHand.positionImage.y;
if (leftHand.openness != 0) {
leftHandOpen = true;
} else {
leftHandOpen = false;
}
} else if (foundRight) {
collectingData = true;
rightHandDelta.x = rightHand.positionImage.x - rightHandPosition.x;
rightHandDelta.y = rightHand.positionImage.y - rightHandPosition.y;
rightHandPosition.x = rightHand.positionImage.x;
rightHandPosition.y = rightHand.positionImage.y;
if (rightHand.openness != 0) {
rightHandOpen = true;
} else {
rightHandOpen = false;
}
} else {
collectingData = false;
}
Here is the full breakdown of that whole code. Get ready if you're not reading very close. You will miss it like those tiny towns on rural highways, blink and poof gone. Use the sdk to get the current position of the users left and right hand. Check to see which hands were present. If both hands were found, make sure the the sdk actually is telling you that the left hand is on the left (you would think that would be always true but our testing showed that to be false). Set the delta's on each hand, then set the current position. If no hands were found, disable collectingData.
Yep that is it, simple right? Told you I always stand by K.I.S.S. crawl step done! …. Now to walk, use the heads motion to generate targeting. That is the start of the process to the holy perceptual grail. When we get the walk step done I will be sure to add another technical blurb to our weekly blog on head tracking.
John Wrote a cool tool to start seeing the controls in Unity 3D. We are sharing for everyone here (zip file). Feel free to download and share.