Month: October 2013

DM2009 lesson missed :(

I missed the Performance and Interaction lesson last week because I have to attend a company information session in the evening at Centennial Tower.

Anyway, I have to review on my previous assignment 1 and see how I can improvise it into a better functional project using Arduino.

Hmm let’s see. That project included sound as performance and some keyboards pressing as interaction. Therefore, for this second assignment, I could use some button switches or sensors to play different music. Doesn’t sound too creative though. Argh. I heard from my classmates that last week’s lesson was about Arduino and sound. 😦 Missed it!

Argh, brain’s dead. Too much assignments, quizzes, and job application deadlines to meet! 😡


DM2009 Introduction to Arduino

In this week’s lesson, we were introduced to Arduino Mega 2560. It is a microcontroller board which has 54 digital input/output pins and 16 analog inputs. It is very easy to use – simply connect it to a computer with a USB cable – plug and play.

In order to allow interaction between MAX and Arduino, a library called Maxuino was also installed. Before we could start tinkering with anything, there were some configurations to be done:

  1. In Arduino 3, select the right board and port used. (Tools -> board -> Arduino Mega 2560 ; Tools ->port -> usbmodem.??.tty
  2. Open up the Standard Firmata under Examples and upload it into the board

Then get a copy of SparkFun Inventor’s Kit for Arduino and get started!

Yi Xiang and I grouped together and we had fun connecting up the wires and components. I found that it is easier to follow the schematic diagram than the illustration. In our exercise, we used a photoresistor (as analog input), a LED (as digital output), a 330 ohms resistor, a 10k ohms resistor, and some jumper wires. The connection is shown in the diagram below:

Arduino and breadboard

Arduino and breadboard

After opening up Maxuino-gui, update the board and port used. Then we tested the input and output of Arduino in Maxuino patch with GUI. The GUI is very useful in letting us to control and observe the arduino’s input and output in real time. To program this exercise, the patch was modified as shown in the figure below:

Maxuino Interface

Maxuino Interface

Basically this is how it works:

  1. The program takes in the input value from the photoresistor (which change its resistance according to light intensity)
  2. The input value is compared with a certain value (e.g. 0.5)
  3. When the value exceeds 0.5, it will trigger the ouput to go to “on” state (LED connected to that output pin will light up)
  4. At the same time, the input values are also converted to larger number using “scale”. The converted values are fed into “mtof” and “cycle~” to produce a sinusoidal wave. Finally a speaker is connected to output the sound waves. Therefore, the photoresistor has a direct effect on the sound wave produced.
  5. Later on, we modified it to allow the control of movie playback using the photoresistor.

Overall, it was a fun lesson!

DM2009 Performance!

Our class had many wonderful performances today! I feel so inspired by many of my classmates. But first, let me talk about my own project.

The initial idea was to use the keyboard to play different notes of music. Later, I realized that it is very much similar to a virtual keyboard piano. Surprisingly there is a guy in the class who really did that virtual keyboard piano! The class seemed to like the idea. May be I should search more in the tutorials because I remember seeing a piano symbol in one of the tutorials before.

I think that there are so much things that could be simplified (and I overthought them). Perhaps there are already piano notes in MAX but I still downloaded each of the piano notes from outer source. Due to the file size and loading issue in MAX, I could only afford to load 9 musical notes correspond to the numerical 1 to 9 on the keyboard. And the fact that I thought people would find a virtual keyboard piano boring (but it doesn’t seem to be the case just now), led me to think of make it more quirky. But given the time and limited prior-knowledge to try something quirky may not always work out.

So the quirky idea is to detect colour of the shirt of the user interacting with the program. I used the color look up table and jit.findbound. I couldn’t make it work on the first try, second try, and third try. Considering the time constraint, I gave up. Apparently, many of my classmates could use that function well. =/

So instead of using the live webcam, I used the recorder to detect the surrounding sound (e.g, clapping) . When the sound level exceeds -20dB, the counter will be increased by 1 and the output counter number is fed to the “cue” which will play the preloaded music note with “sfplay”. The patch file is shown in the image below:

MAX project screenshot

Cue Number from Counter

But it would be very boring if it keeps playing the same old music cuing from 1 to 9 every time. So I added a “random” number generator (with a metro to send the bang to it)to get random cue number.  With this, the music generated will be more unique. The patch file is shown in the image below:

MAX project screenshot

Random Cue Number

Before the performance time, I wanted to try something different by using “select” which I suppose can open 1 file at a time according to the number generated. In this case, I can standby many single music files (which will be opened singly instead of preloading all of them at one time) and not worrying for MAX to hang again. But I wasn’t able to make it work within the time constraint.

MAX project screenshot

Another attempt

A summary of this project


Gim’me Some Musics

Features and Operation:

  1. User can play music files with keyboard (1-9)2013-10-09 18.48.52
  2. User can make noise (clapping or talking or hitting something which is attached to the contact mic). The noise will be picked up by the mic which channels it to the levelmeter. The levelmeter will in turn quantize the signal and the signal is compared to a preset value (e.g >-20). If the result is True, it will trigger the counter to add another count or the “random” generator to send a number to the cue. The cue will set which music file to be played.


  1. Generate music from the noise. Suppose the recorder is placed in a highly noisy place (street, construction site, etc) and the music player is placed at somewhere else. The noise generated will be converted into a nice melody.
  2. In a confined performance place, a contact mic can be attached to an object and user can play different musics with that object (not influenced by surrounding noise).
  3. This can also be used to interact with small kids. When they clap,(using a counter) a music note will be played. A full song, e.g. Twinkle twinkle little star, can be played when they clap continuously.

Further Improvements:

  1. Select some nice musical notes from nicer musical instruments.
  2. Have a few cascaded counters and a few sfplay functions to play different musical instruments simultaneously.

Here is a little video of the demonstration of the performance:


i think it is important to understand how the system works than to just make it works. This project has led me to a more creative thinking and to have a fresher view about performance and interaction.

We were introduced to contact mic, which is a very useful tool for this project. It only picks up sound wave (vibration) of the object that it is being attached to. Therefore it is rather immune from the surrounding noise.

There are some interesting performances by my classmates. I have learned a lot from them too. There are much more things to explore in MAX, and thanks to my classmates, we have seen different functionalities and applications of MAX. Most of them are applying the idea of play and pause a video or music file with object detection. I think it is kind of cool. Cant wait for the Arduino session next week! 😀

DM2009 Progress (Week 7)

The recess week is here! i have no lab session before the class and so Wen Yang and I went to the Performance and Interaction class earlier. I guess we were a bit too early as the door was still locked. So we headed over to the library. It has been my favourite place ever since I first stepped into it 2 months ago. Haha.

The second time we went to the class, the door is still locked. But thankfully a staff walked pass and granted us access. Surprisingly Mr. Dirk was already inside, with some other students as well. Oh uhh and I don’t like being late again :((

I went on to proceed with my project idea by downloading some music sample files from here.  I constructed a Max project based on some of the tutorials, which allows the user to play a musical note when a key on the keyboard is pressed. Sorry that I still couldn’t figure out how to perform printscreen on Mac. I know it is Command+Shift+3 but where is the file? I tried to paste it in Adobe photoshop’s new file but it doesn’t seem to work like Window’s printscreen which have to be pasted somewhere. Anyway, I use the function “key” to get the pressed keyboard value then “split” it from 48 to 59 (in ASCII) so as to just extract (detect) numerical value 1 to 9. Then i set up a “cue” function and used a music file player to “open” the first music note and “preload” key 2 to 9 with 8 other music files.

My first intention is to have different preload files such as piano, guitar, cello, and saxophone. Then using some sort of color detection function to get the colour of the shirt that the user is wearing. Different colours will open and preload different files and so when that user presses on the keyboard, the music being played will be different. I tried to use the “jit.findbound” function and colour lookup table but I couldn’t seem to make it work. 😦

At last, I changed the idea to something simpler. If the surrounding noise level is high, preload a set of louder music; if it is more quiet, preload softer music. Hmm, i guess the noise level fluctuated too frequently and the preloading process could be too intense for Mac to handle that it hung for several times.

Mr. Dirk gave me a better idea after that. So instead of just preload the music, I use a counter to count the number of times that the noise level exceeds a certain dB. It was quite fun to hear the music being played in a funny way according to the noise level but i think the choice of the music notes plays an important role now. In the end, i gave it 2 options. If the noise level is more than -30dB, it plays a music note according to the counter. If it is not, a user can play the notes using the keyboard (9 keys).

Improvement that can be done: Instead of just 9 musical samples, i could include more (but smaller file size) music samples. To make it more interactive, I can install some sensors at the pathway so if a person walk pass, it counts “1” and plays the first music sample. Then followed by the second person who walks pass, it will play the second samples and so on. This is actually very similar to my very very first few ideas after my first DM2009 lesson! 😀

  1. Install at least 2 sensors at a certain distance apart(e.g. 2 meter). When a person walks pass the first sensor, it triggers a timer. When the second sensor is triggered, the timer is stopped. If the time, let say, is less than 3 seconds, it means the person is walking very fast or nervous. So a more soothing and relax music is played along the walkway. The intention is to calm down the person. Same wise, if the time taken is more, an energetic music is played because the assumption is that the person is feeling depressed or dispirited. So a lively and faster music can cheer the person up.
  2. Different sensors are attached at different corner in the room. When each of them is triggered, a musical note will be played. So the more people in the room, the merrier! Everyone can just walk or run around the room, triggering different sensors and thus different musics can be performed.

HAHAHA, this actually ignites my passion for engineering again! 😀



HAHAHA today is a good day to try all those crazy stuffs. But it was a super normal day: woke up late because I was so immersed in a dream, studied my comp security (slow as ever), checked my whatsapp and knight still sleeping, ate cereal with milk for lunch, write some blogs for FYP, cleared MIT’s course exercises for this week, checked whatsapp and knight still sleeping, called knight and he’s still sleeping, wrote an email to knight to blackmail him, did some figurobics (pls, for like 10 minutes?), continue MIT course, finally knight woke up, nagged like an old mother, then went dinner.

I tried the new stall in the canteen – what a nice BBQ fried chicken leg with rice. Satisfied? No, i decided to reward myself with a can of coke after I returned hall. And guess what? I found a packet of MacDonald’s salt in my cupboard. Well, they said salt with coke cools down the body. So I opened up the can, drink a sip and pour in the salt.


To my amaze, my rootbeer fantasy came true, non stop. Volcanic eruption. I stared at the phenomenon blankly. The foam, bubbles and gas are everywhere on my desk. My first reaction is to sip using the straw as hard as possible. CHEERS! Then the coke started dripping down the desk and all i could do is to use my hand to stop it. =.=

Oh well, obviously that didn’t help much. So I end up mopping the floor with coke…. CHEERS!

I text knight and all he could do is laughing non-stop.