A test mashup of Vuforia Image Tracking on HoloLens 2 using Unity with “clipboard bound” MRTK buttons. The buttons control a custom hexapod robot (Rocco) via serial commands over Bluetooth. It’s great on PC but slow on lens. I bet there are options within Vuforia, or it needs interpolation for this use case. Pardon his broken leg.
Hand tracking as fake controller input for automatic grab using MRTK. This is an experimental adaptation of Earnest Robot’s “Auto Hand – VR” on the Unity asset store and its XR demo scene. Can’t wait to see multiplayer carnival games at virtual conference networking events!
A prototype handsfree SteamVR simulation created in Unity for testing weight distribution and wearability of custom VR helmet(s). The user controls the speed and direction of a virtual motorcycle using the gyroscopic sensor in a Vive VR headset or cell phone via RiftCat. Developed for and in collaboration with Cyreana Foulk and Chris Foulk.
A prototype demonstration of physics based running water with hand interaction in Unity on HoloLens 2 using the Screen Space Fluids (SSF) asset (particle system + compute shader).
Testing Leap Motion controller for hand tracking and hand physics inside the Unity editor (play mode). Deploying to HoloLens 2 takes a few minutes each time and a specific process to prevent errors. Less deployments is better, especially if you just want to tweak something small.
- Unity 2018.4.24f1
(2019.3.15f1 works too)
- MRTK v2.4 + Hand Physics Extension
- Leap Motion SDK + Core Assets 4.4.0
- HoloLab Inc’s LeapMotionInputForMRTK: https://github.com/HoloLabInc/LeapMotionInputForMRTK
- One backwards dusty hat guy
Still working on this guy… sorry no video at this time. He was assembled enough to execute some motion presets that I added to the code, but there are some interference issues and possibly one or two servos that don’t work quite right. I took him back apart to work on shielding but got pulled off the project for a while. So close to getting back to the code.
Circuit Diagram v8 (WIP)
Alive! Well, the light is on…
Servo Hook-Up Board – Bottom
Servo Hook-Up Board – Top
Servo Hook-Up Board – Side
PC Power Supply with Voltage Regulator (Arduino wants 7-12V, I gave it 9V)
The videos below are demonstrations of the potentiometer and a few other commands. The potentiometer controls the interval value when turned left or right (as does “i##”). You can also see some examples of the other commands that Rocco accepts as input in the first video. I am pasting a sequence of motion pre-sets while using the potentiometer to change his movement speed.
“[” or “]” | Increase/decrease servo speed – Rocco continues executing a motion every “servo_speed” milliseconds until motion is complete
“s###” | Set a three digit servo speed (default is 25ms, maximum 999ms)
Hardware potentiometer | Increase/decrease servo_interval – Rocco moves each servo “servo_interval” number of degrees until executed motion is complete
“i##” | Set a two digit servo interval (default is 32 which is the maximum value)
“p” | Toggle usage of potentiometer for reading the servo interval (potentiometer is used by default)
“j##” | Select a servo using a two digit servo number
“,” (the “<” key) | Select previous servo
“.” (the “>” key) | Select next servo
“k” | “Knock” selected servo (move +5 degrees then -5 degrees just to see which servo is selected)
“a###” | Set the angle of current servo, requires three digit angle (add leading zeros if necessary)
“g” | Get angle of selected servo
“m##” (00-08 currently) | Execute pre-coded motion
“+” | Selected servo plus 5 degrees
“-” | Selected servo minus 5 degrees
“d” | Disconnect selected servo (reset required to reconnect)
“r” | Reset
I am using an HC-06 Bluetooth serial module for wireless control. It accepts 3.6 – 6V and I’m using 5V from the Arduino.
The transmit/receive pins should be wired as follows:
Arduino “TX0” -> HC-06 “RXD”
Arduino “RX0” -> HC-06 “TXD”
Other markings on the module: JY-MCU, BT_BOARD V1.06
The HC-06 is a slave device only and cannot start communication on its own (it cannot be paired with other Bluetooth modules, only “masters” like a computer or smart phone)
I am using the Real-Term application to connect to the module, which can be downloaded here:
Real-Term is also referenced on the page below, which is a good source of information on serial communication in general:
Below is an example of how to connect to the HC-06 module using Real-Term:
Baud: 9600 (this value should match what is coded in the Arduino)
Port: “6 = \BthModem0” (you can find which COM port HC-06 is using in Windows 7 by “Devices and Printers”, “HC-06”, “Hardware”)
Data Bits: 8
Stop Bits: 1
Hardware Flow Control: None
Press “Change” to connect
I am also able to connect to the Bluetooth module using my Nook tablet and the “Bluetooth Terminal” application. Any Android tablet will work, but iPads and iPhones will not as they don’t support this kind of connection.
(if the link breaks just search for “a child’s guide to basic two way bluetooth” – that is how the document is titled)