Iteration 1 Dominik - Mapping movement speed to train sounds played on a phone.

Steam Punk theme:

Hardware: 

Microbit, steam punk costume 

Software:

This iteration uses the microbit bluetooth HID extension from make code extensions. https://github.com/bsiever/microbit-pxt-blehid. It is straight forward to use and provides many options on controlling a bluetooth device by sending it bluetooth codes some which are common to all devices like play - pause - skip songs. The thing to keep in mind when using this library is to disable the radio functionality of the microbit. 

I used a serial window on Mac by first listing my usb devices /dev/cu.*  and later using this command to start a serial terminal screen /dev/cu.usbmodem1302 115200. On Windows we used a tool called https://os.mbed.com/users/borislav/notebook/serial-port-plotter/ which is very good for plotting a graph. We used this to calculate the different accelerometer values for when a person is running or walking. This microbit project creates a 3D acceleration vector https://makecode.microbit.org/_3EfHos45bciX that is used here.

The sounds that were used came from this YouTube clip at around 5min mark.https://www.youtube.com/watch?v=Sgkq1Kiz80I and the split sounds can be found here: https://drive.google.com/drive/folders/10utZi9UFfIeH449arsU7tO55-T4Z6Rfz?usp=share_link 

The Problem: 

To add sound to a persons step. 

First try : Beep with microbit on every step. The step threshold was 1300 for walking and 2000 for running. This worked well with little to no latency.

Second try: Connecting to a phone with bluetooth and playing a sound. Worked well the best function for playing a single sound to use was previous because on most media player apps it unpaused the sound. 

Third try: Using a step counter with the running time function on the microbit to calculate the time in milliseconds between each step. Problem here is the first step will always be missed as you can stay idle for some time and that time reading will throw the sound off. This step was useful for figuring out the step values which is 700 -800ms for waling and around 300 - 400ms for running.

Fourth try: Combining values of the accelerometer with the values found to be the step times and map them to each other. In code it looks like: 

if _3D > 1250 and _3D > 2300 and oneShot == 0:

serial.write_string("" + str(_3D) + " time " + ("" + str(Math.round(Math.map(_3D, 1250, 2300, 400, 270)))))

serial.write_line("")

oneShot = 1

for index in range(2):

media.send_code(media.keys(media._MediaKey.NEXT))

basic.pause(Math.round(Math.map(_3D, 1250, 2300, 400, 270)))

elif _3D < 1250:

oneShot = 0

I am here mapping 1250 - 2300 accelerometer value to 400 - 270 ms meaning the faster you go the smaller the delay should be between playing the next track.

I am using the next song operator as I made a playlist with all 4 train tracks and I have them all on repeat so they don't pause. My microbit file can be found here: https://makecode.microbit.org/_JKx9rbYzDgA9

Reflection on this iteration: 

The below video shows a quick demo. It is not the best but you can see sometimes the timing will be almost perfect. There are latency issues with using bluetooth and we found that older phones also impact the latency. 

My solution to this would be to use a wired approach and playing files from something like a DF robot mp3 player wired to the microbit. In my opinion It would be more responsive.