A test mashup of Vuforia Image Tracking on HoloLens 2 using Unity with “clipboard bound” MRTK buttons. The buttons control a custom hexapod robot (Rocco) via serial commands over Bluetooth. It’s great on PC but slow on lens. I bet there are options within Vuforia, or it needs interpolation for this use case. Pardon his broken leg.

Hand tracking as fake controller input for automatic grab using MRTK. This is an experimental adaptation of Earnest Robot’s “Auto Hand – VR” on the Unity asset store and its XR demo scene. Can’t wait to see multiplayer carnival games at virtual conference networking events!

A prototype handsfree SteamVR simulation created in Unity for testing weight distribution and wearability of custom VR helmet(s). The user controls the speed and direction of a virtual motorcycle using the gyroscopic sensor in a Vive VR headset or cell phone via RiftCat. Developed for and in collaboration with Cyreana Foulk and Chris Foulk.

A prototype demonstration of physics based running water with hand interaction in Unity on HoloLens 2 using the Screen Space Fluids (SSF) asset (particle system + compute shader).

Testing Leap Motion controller for hand tracking and hand physics inside the Unity editor (play mode). Deploying to HoloLens 2 takes a few minutes each time and a specific process to prevent errors. Less deployments is better, especially if you just want to tweak something small.

Still working on this guy… sorry no video at this time. He was assembled enough to execute some motion presets that I added to the code, but there are some interference issues and possibly one or two servos that don’t work quite right. I took him back apart to work on shielding but got pulled off the project for a while. So close to getting back to the code.

Partially Assembled

Circuit Diagram v8 (WIP)

Alive! Well, the light is on…

Mostly Assembled

Servo Hook-Up Board – Bottom

Servo Hook-Up Board – Top

Servo Hook-Up Board – Side

PC Power Supply with Voltage Regulator (Arduino wants 7-12V, I gave it 9V)

Controls

The videos below are demonstrations of the potentiometer and a few other commands.  The potentiometer controls the interval value when turned left or right (as does “i##”).  You can also see some examples of the other commands that Rocco accepts as input in the first video.  I am pasting a sequence of motion pre-sets while using the potentiometer to change his movement speed.

“[” or “]”   |   Increase/decrease servo speed – Rocco continues executing a motion every “servo_speed” milliseconds until motion is complete

“s###”   |   Set a three digit servo speed (default is 25ms, maximum 999ms)

Hardware potentiometer   |   Increase/decrease servo_interval – Rocco moves each servo “servo_interval” number of degrees until executed motion is complete

“i##”   |   Set a two digit servo interval (default is 32 which is the maximum value)

“p”   |   Toggle usage of potentiometer for reading the servo interval (potentiometer is used by default)

“j##”   |   Select a servo using a two digit servo number

“,” (the “<” key)   |   Select previous servo

“.” (the “>” key)   |   Select next servo

“k”   |   “Knock” selected servo (move +5 degrees then -5 degrees just to see which servo is selected)

“a###”   |   Set the angle of current servo, requires three digit angle (add leading zeros if necessary)

“g”   |   Get angle of selected servo

“m##” (00-08 currently)   |   Execute pre-coded motion

“+”   |   Selected servo plus 5 degrees

“-”   |   Selected servo minus 5 degrees

“d”   |   Disconnect selected servo (reset required to reconnect)

“r”   |   Reset

Connection

I am using an HC-06 Bluetooth serial module for wireless control.  It accepts 3.6 – 6V and I’m using 5V from the Arduino.

Module:
http://www.amazon.com/dp/B0093XAV4U/ref=pe_385040_127541860_TE_dp_1

The transmit/receive pins should be wired as follows:

Arduino “TX0” -> HC-06 “RXD”
Arduino “RX0” -> HC-06 “TXD”

Other markings on the module:  JY-MCU, BT_BOARD V1.06

The HC-06 is a slave device only and cannot start communication on its own (it cannot be paired with other Bluetooth modules, only “masters” like a computer or smart phone)

I am using the Real-Term application to connect to the module, which can be downloaded here:
http://realterm.sourceforge.net/

Real-Term is also referenced on the page below, which is a good source of information on serial communication in general:
https://learn.sparkfun.com/tutorials/terminal-basics/all

Below is an example of how to connect to the HC-06 module using Real-Term:

Baud: 9600 (this value should match what is coded in the Arduino)
Port: “6 = \BthModem0” (you can find which COM port HC-06 is using in Windows 7 by “Devices and Printers”, “HC-06”, “Hardware”)
Parity: None
Data Bits: 8
Stop Bits: 1
Hardware Flow Control: None
Press “Change” to connect

I am also able to connect to the Bluetooth module using my Nook tablet and the “Bluetooth Terminal” application.  Any Android tablet will work, but iPads and iPhones will not as they don’t support this kind of connection.

Additional links:
http://embeddedprogrammer.blogspot.co.uk/2012/06/ubuntu-hacking-hc-06-bluetooth-module.html
http://homepages.ihug.com.au/~npyner/Arduino/GUIDE_2BT.pdf
(if the link breaks just search for “a child’s guide to basic two way bluetooth” – that is how the document is titled)