Experiential Design & Personal Project 1

Tan Shin Yih (0334900)
Experiential Design & Personal Project 1

Week 1

For the first class of experiential design & personal project 1, Mr. Mike brief us about the module and told us that we will have a collaboration project with RMIT students. He also explains to us how the collaboration will be going on and what we need to do for it. For let us understand more to the project we need to do, Mr. Mike also asks us to visit the previous work of RMIT student (http://rmitdmcc.com/2018/). Mr. Mike also sends a survey to us to know everyone specialization so it would be easier to split the group later. We also being told to work with the Australian students for 2 weeks, and we will need to come out of a prototype by this 2 weeks time. We need to work with the RMIT students by this 2 weeks time from 10 am to 6 pm at CODA if we don't have any classes to attend. Before class ended, Mr. Mike brief us for a one day trip with the Australian students by next week on Thursday at Ipoh.  

-----

We have no class in the second class as we will need to wait until the RMIT student reaches Malaysia. Our lecture asks us to use our time to do more research and improve our own skill by these few days to prepare for the project.


Week 2

(Day 1 - Monday)
Today is the first day we meet up with the RMIT University students. We first introduce ourselves with RMIT student, then we choose our specialize from animation, interactive, video sound, and VR/AR, then only we choose our teammate base on our specialized. And this is my new friend from Australia and group mates.  

Then, we start with the brainstorm of what we wanted to do and also what we can do based on our title "Cities". We were given some time to have a short conclusion for what we plan to do and present to the lecture. At first, we planned to do something like animation or interactive stuff. Then, my groupmate Chris suggest us to try to work on projection mapping. Our Australia friend Andrea also has an experience of creating a projection mapping before, so we plan to have a try on this method. Andrea show us her work before was a paper 3D model that did by her and her groupmate. 

We start to brainstorm on how we create the 3D model like may be using the same method like hers but we think of if create something big, it will be hard for them to bring back to Melbourne. Then I come out with an idea to create the 3D by using Maya so we can send them the file after we finish the model. They can use 2 projectors to project the final outcome to the audiences. But after we present to our lecture, he gave us a comment like if we using 2 projectors to project something then there will be no meaning doing the projection mapping. We then drawing out the mind map to have another new idea for the project.

While they draw the mind map, based on the 3D paper model I came out with a new idea which is doing a paper art that I always wanted to try but yet haven try before. I find some sample and show it to my group mates and I felt so happy that all of them likes paper art things. While I research for the paper art thing, I came out with more another idea about paper art so we start to create the slides for the presentation later. My English is not really good so I explain my idea to my group mates and they create the slide.

- the idea of combination between Melbourne and KL

- the idea of a turning wheel show Melbourne and KL differences

- the idea of holding a hand likes the collaboration between us

- the idea of exploring typography and patterns.

- the idea of creating some contrast between both countries with two side art.

- the idea of combine two cities.

- the idea of the process/ timeline of the RMIT student explores Malaysia.


* using paper art
* layering paper art
* experience on how projection mapping works on layer paper art. 

After presenting our idea to the lecture, they like about our idea of using paper to create silhouette a lot, but we will need to brainstorm on what is our concept and what is the projection mapping will be projected, like maybe a short storyline, interactive art or what. Since we have no idea of what our concept will be, so we plan to have a research on ourselves on what is projection mapping and what can projection mapping do and some inspiration video and discuss on Tuesday.

Few samples of my inspiration video of projection mapping:
1. https://www.youtube.com/watch?v=FcIDQSn26fw
2. https://www.youtube.com/watch?v=UYFWjUj4RPQ
3. https://vimeo.com/13337006
4. https://www.youtube.com/watch?v=Yin4-EVcR34
5. https://www.youtube.com/watch?v=lX6JcybgDFo
6. https://vimeo.com/45569479
7. https://vimeo.com/41486619

(Day 2 - Tuesday)
For the second discussion day, our lecturer told us that we will have a final concept and idea presentation of our project to the panel. After discussion, because the paper silhouette art has a lot unknown thing so our group mate decides to change the layer thing to a paper model.

At first, Andrea and Lily give us an idea that maybe we can try making the Malaysia signature building KLCC and because KLCC has 2 building so we will project Australia thing on one building and Malaysia on another building. We present to the RMIT lecture Ms. Li Ping, she told us that KLCC was already known by everyone so we might need to try to find another building to make it more special. She also suggest us for using Australia animal as a model, project Malaysia thing on it and using Malaysia animal to display Australia video, switch the things will make it interesting.

We continue our group discussion on what model we can make for the project. I suggest maybe we can make a lot of 3D house model of Malaysia and Australia building, then project light on top then make a contrast of both cities. Andrea suggested that we can also like switch the video for both cities, for example, Australia video on Malaysia building. Lily ask like maybe we can make people can walk through the city? Her words remind me of a science museum that I went to before when I was a kid. We can play around with the model around us and the model surrounds with light. So we can have a concept like "Explore two city in one time". Then, Chris told me like if we just make a lot of small building on the floor, it's hard to let people feel like "wow, two cities at the same time". Then, I suggest putting 2 whiteboards on the side of the building.

Something like the photo shown. At first the small 3D building house we arrange in a square size, because we need to have a way for people to walk through so we can cut the square into half become triangle, one side triangle will be building at Australia and another is for Malaysia and the screen will have video, we can also make something interactive like when people walking through the sensor, there will be some information on both city shows based on the people walking. My groupmates like my idea, Lily and Andrea suggest that we can have a drawing on outside of the whiteboard for display and Chris said that just pop out information will be boring, maybe we can adding some special effect and video thing but add the talking narration or music. He also suggests us to try to make the interactive on people footprint when they walk through space. We were happy with this idea so we present to Ms. Li Ping again. After the consult with her, she said that we lost the meaning using the projection mapping, and ask us to focus on the concept idea, the meaning of why we doing this thing.

We getting confusing after receive all the feedback, so we start to think whether what messages we wanted to bring out. The reason why we are doing this thing and what is our goal. While we thinking the concept for our project, Chris shares a video to us, that was my inspiration video that coming out the final idea. (https://www.youtube.com/watch?time_continue=2&v=rPqKlJd2SAc) After I watch the video, I come out with an idea in my mind "What we see is not the truth". Before the brain stuck, we had our break on lunch. While eating lunch, I come out will a concept direction, by the video I watch just now, we can find something that was normal for both countries but with adding something, the things will also having its own meaning like food! Like a normal plate, it is just a normal plate and we can see it everywhere, but if we project different food on it, it shows the differences between both countries.

We start our mind mapping on what is the differences between both cities and also start thinking of what method we can use for the paper silhouette. So we went back one full circle just to decide we're gonna do what we said in the start. After we explain our idea to our lecture, they were very happy with our final idea. They ask us to explore more and also giving us some useful suggestion. For the video part, we can use more unique thing rather than a famous thing and we also can use the experience to create a story. So we start to do the presentation slides and separate the job for each person to do at home.

- Chris: Slide information
- Lily: Figure out the new software VPT8
- Edrick: Technical stuff (what equipment we needed)
- Andrea: Mood board
- Me: Touch up the slide, research for material that suitable for projection mapping & art style

We also had another outsider lecture introduce and explain for us about Virtual Reality. He explains the VR art, showing us his previous work and also the technical aspect of VR. He also tells us how we cut out the VR screen into a flat art and also draw some sample of the VR art from different perspective angle to let us understand more on it.

(Day 3 - Wednesday)
We confirm the art style as we want to use the Neon color. Before the presentation, we have a first test of the technical part to see how the paper art looks while we project something on it. We realize that the distance between the projector, box and also the paper silhouette might affect the size of the shadow. So I will have research on how to create the box, do sketches, layout and the decide the distance between the paper silhouette to make sure everything works well.

After our presentation, the panelist was happy with our full circle back to the original paper art project. We proposed that we will finish 1-2 complete box and scene by these 2 weeks, but the panelist suggests us to finish all the 7 boxes and 1-2 complete scene by these 2 weeks.




As we will have a one day trip on Thursday to go Ipoh with the RMIT students to explore the cities, so I create a simple timeline that not too free and also not too tight so that we can follow the timeline to finish everything by these 2 weeks.

(Day 4 - Thursday)
One day trip at Ipoh. We go to Tapah to have our breakfast first, then we went to a museum called Kinta Tin Mining.

The second place we go to was the Perak Famous Tanjung Tualang Tin Dredge Ship #5.

After visiting the two place, we went to Sekeping for lunch and everyone can walk around to explore the Ipoh street ourselves. We also going to the Perak Cave Temple to visit but I didn't take a lot of pictures because we usually didn't take the photo inside Chinese temple because my grandmother always tells us if the people's momentum isn't strong enough then you take the picture, you will get sick.

The last station was a night market of Ipoh at Gerbang Malam. We can walk around and take pictures at that place. After having our dinner, then we went back to campus.


(Day 5 - Friday)
We will need to go to the art shop at Curve to buy some materials for the paper silhouette so we had our discussion at Starbucks to confirm our final art style, transition, and storyline.

Below are the storyboard of our project and we also changed some of the items for the paper silhouette to represent each scene. We also having some brainstorm of what text should be shown for the audience after the animation is done.

After coming up with our final storyline, we planned to do the video using animation. We also went to Art Friend to buy some materials for the paper silhouette box testing. Based on my research, we decide to buy art card paper and mounting board as our paper silhouette materials as the thickness of the paper is suitable for us yet it is not so expensive.

In order to make it easy for the RMIT student to find the materials, we went to IKEA to buy a suitable box for our paper silhouette box which is called "pappis".

(Day 6/7 - Saturday/Sunday)
On Saturday Sunday, we have our own time to do our own assignment for other subjects, we also separate our project works to:

- Chris: doing the sound thing
- Edrick, Andrea & Lily: start to draw some animation based on the storyboard
- Me: Draw the sketches for the 7 boxes and also finish 1 box of paper silhouette

I didn't do a tunnel book before so I did a lot of research about how to do the actual box thing and also did some research on how to draw the layer stuff on to art card.

I did some simple sketch for each layer, but because I lacking paper as we just bought 1 for testing so I also figure the technical stuff for the box.

I found that a lot of the people in the tutorial using the woodcut small box and some using the polystyrene foam to adding distance to the layer to make the lightbox. But because of those materials are very expensive so I decided to make a box for the distance thing myself. And after the try out of the box and everything, I plan to do 14 rectangle box in total for supporting the layered scene, 2 for 1 layer and total 7 layers.

Week 3

(Day 8 - Monday)

We show our progress did by Saturday Sunday, the messages and the storyline we did last week to Ms. Li Ping when we reach class. Ms. Li Ping is happy with my tunnel box thing and she asks me to try to finish this thing and testing on the projection first only draw for the other scenes layer draft. She also happy with the messages to the audience that written by Chris but some of them we still need to rephrase because she thinks that is too common and some lost the meaning. Ms. Li Ping also asks us to speed up and finish the animation because we still don't have any animation to show yet. 

As they haven't started the animation yet so all of us decide to do the animal scene first for the prototyping. And the photo shown is the storyboard of the animation. We separate our group for different tasks so that we can speed up the progression.

- Chris: sound & messages
- Edrick: Hornbill 2-5 second animation
- Andrea: Kollar bear 2-5 second animation
- Lily: Kangaroo 2-5 second animation & opening scene
- Me: start drawing the layer, buy art card and finish the whole tunnel box

We plan to do around 15-20 second for each scene including the messages to the audience and also changing the art style from neon color to vector art style and more cartoonish. As we couldn't finish our task that I listed just now, so we continue the task at our home.

I just finish the 14 boxes and the drawing scene in class, so I bring back all the art card and cut it at home. I cut the layered scene by following the sequences so I can still adjust and redraw some of the scenes to make sure that every layer can be seen after we position all the layer.

This is the 7 layers for the animal scene. Then, I try to arrange it with the 14 rectangle box inside the big box "TV" to see how it feels.

This is how the overall of the tunnel box looks like after I placed everything inside. I also try to use the torchlight to try out the feels of the box after the light goes through. I found out that the shadow of the paper cut out will be affected by the distance between the torchlight and the box. The second problem is when I move the torchlight to a different angle, the shadow will move and block something. It just likes how our shadow works when we stand under the sun.  

(Day 9 - Tuesday)

We have a presentation on today 2pm, so we need to try out everything before 2pm. After Andrea combine all the animation into a video, we had our first testing of the projection mapping with using the mad mapper software.

For the first video, we use a grey color background for the paper cutting thing and everything looks dark, although we adjust the projector brighter. So I suggest trying using some bright color, maybe using white color for the first layer, then maybe the second layer only put the other color so that the light inside the box is bright enough because our tunnel box already blocks some light so we need some bright color to make animation pop up.

So Chris will adjust the screen size for the projection, Andrea and Lily will try changing the different color of the layer to see what works better. While they doing the technical things, I start drawing the template and also a details background for the animal scene.

I try to draw the first layer more details and just adding some plain color for other layers because I scare the outcome will be very messy. After they finish the position of the projector and also the projection, I found that the layer in the animation is flat, so when we project on the box inside the box we can see everything is flat, big and the perspective was wrong. So I suggest adding the camera for the animation in After effect so that we can make the layer at the behind part smaller and further.

After our presentation, all of the lectures ask us to try to make the box bigger so that everything looks clearer and bigger and we can add more details inside too as our outcome for this prototype is just an A4 size box. We have a group discussion after the presentation and we decided to increase the sizes of the paper cut into A2 size so everything will look clearer and easier for us to project more thing on it. We also changing the position of the messages to the audiences to the bottom line of the "TV", so that the words will be clearer.

We also try to project the background that I draw just now to see whether the color works well not without scaling the size. We also plan to improve the animation to make sure the animation move smoother. Before we went home, we separate the task for everyone to do at home. Basically, we will just create a new tunnel box like I did before just with A2 size.

- Edrick: the outer card box of the paper silhouette

- Chris, Andrea, Lily: trace the 7 layers and cut out

- Me: the 14 rectangle boxes for supporting

(Day 10 - Wednesday)

On the morning, we cut out the 7 layers of the paper silhouette and also stick the supporting box. Then, Chris starts to study the projection mapping thing, and the rest of my group mates continue with the 4 animals animation. I measure and draw out the cover and the bottom part of the outer card box.

Ms. Li Ping asks us to speed up for the paper silhouette thing first so we can have more time for the projection mapping. I ask my groupmate to help me to cut the box with me.

To stick the first layer paper cut out, I use sticky tape to stick first so that I can restick it if I paste it wrong. Then, I realize that there are a bit out as we can see a bit hole from the top of the paper cut out, so I measure another mounting card and cover it.

After cover the hole, we use masking tape to tape the whole thing so that everything more stable. I also use the UHU glue to stick the 4 sides of the cover.

Then, we also use a hot glue gun to stick the bottom of the box. Before we back home, we went to spray the box into black outside the foundation class.

(Day 11 - Thursday)

Friday would be the final presentation of our prototype to the panelist, so we need to make sure everything done by today. I recut some part of the layer paper cut out because some part of the paper cut out not smooth and also use sandpaper to sand the edge of the paper. We also use white acrylic paint to paint the side of the paper because we accidentally spray some black color on the white paper yesterday. 

After that, I stick all the layer into the box using UHU glue. While we sticking the layering paper into the box, we found a problem that the middle part of the paper will band a bit because we have a wrong calculate for the box supporting part so we couldn't stick the supporting box on the side and bottom part. We only can stick on the bottom part, so the supporting box is not enough strong. Then, we add some styrofoam to support the middle part of the paper as the styrofoam is light enough and will not adding other weight for the supporting box.     

We plan to use back the old prototype box as one of the screens for the final presentation as a sample. The picture above is the final looks of the paper silhouette box. For the animation part, because they couldn't finish the animation of the animal, so I will redraw the animation by tonight.

The rest of my groupmate continue with the projection mapping position.

I did some research and redraw the kangaroo and add a few frames for the animation so that the jumping animation would be smoother because the previous people that doing this animation only have 1 frame for the walk cycle animation so the animation looks weird.

Besides, Mr. Kannan also suggests us to make the leaves on the first layer moving to help the animation looks more interesting. So I also make the leaf moving by using puppet tools and add in the ease in ease out transition on the leaf to make it move smoother.

(Last Day - Friday)
I continue the animation by combining all the animals into the animation. Because of lacking time, so some of the animal I using the old art that draws by my groupmate but has some changes in the position and also the animation of how the animals move.

The timing for the hornbill landing on the brunch also not suitable, so I ask Edrick to send me the original file and make some changes on the timing.

While I combine the animation, Andrea and Lily create the sample of the overall how our project works. Edrick edits the videos to project on the other 6 boxes and Chris set up all the projection mapping thing and sound effect thing.

Before 2pm, all of us start to confirm the final animation and projection thing. We also prepare for the presentation.

And here are some pictures of the final looks of our prototype projection mapping.

 And I also take the pictures of how the paper silhouette works with the projection mapping.

After our presentation, the panelist is happy with our concept but they also gave us a lot of useful idea such as using the real old TV as the box. They also suggest us to try to use different item and method to do projection mapping so that our concept will be more strong and deeper.

Our 2 weeks collaboration with RMIT students ends by today after all the presentation finish. For the next step of the project, we will continue to discuss from online or the VR system.

Self Reflection:
This is a very meaningful project as I can explore how the projection mapping works on a different method. I also very happy with the project because I can make something that I always want to do but didn't have a chance to do it. It is really hardcore and tired weeks as the progression of part of my groupmates are very slow so I need to do more to cover their part. Unfortunately, I have no chance to access and learn the projection mapping thing because of the slow progression so I took most of my time to create the 2 paper box silhouette. Based on these 2 weeks, we will need to rearrange everyone's job scope so that we can work with more effectiveness. 

Week 4
No class as our lecture giving us a short break after the two weeks collaboration.

-----

For the experimental class, we went for Mr. Razif's workshop that collaboration with Indonesian student to have a card game section.

We will need to design a card game and also redraw the card for our game.

This is our card game design and our card game called "ghost turtle". We upgrade the Turtle card game by adding 6 power card and two dice. The game in Malaysia called Turtle and Indonesia called ghost, so we call it turtle ghost. All player row the dice while his/her turn, when total got 2-4 or 10-12 only can get power card.

Power card:
- Call: Player can call any number card (only 1 number), who has the card that called by the player need to throw it out.- Exchange: Player can choose 1 player to see his/her card and decide what card to change, although doesn't have any card he/she wants also need to change.- Clone: Player can use the power card to clone any number to pair up. (so will have 1 more card doesn't have any pair after the power card used)
- Destroy: Player can throw any number card (only 1 card) that he/she doesn't want. (so will have 1 more card doesn't have any pair after the power card used)
- Add 2: Player need to random take 2 cards from 1 player or 1 card from 2 players.
- Drop 2: Player can random give 2 cards to 1 player or 1 card for 2 players.

Before the workshop end, every group will need to explain their card game for the other group.

Week 5
We have our group discussion about the changes in our project based on the industry expert's comment. Due to their suggestion, we wanted to change some of our paper silhouette boxes into other materials. Besides, we also planned to change our art style to a more abstract style as the old animation is too common.

Here are some example of our references:
1. https://www.youtube.com/watch?v=66QQ-gFSme8&feature=share
2. https://www.youtube.com/watch?v=7prNBMWfhKg&feature=share
3. https://www.youtube.com/watch?v=UYFWjUj4RPQ&feature=share


-----


For today's class, we had our group discussion for the collaboration project. Based on the feedback we collect, we redesign the whole thing for each scene. We sketch out the draft and also decide the material that we will need to use for the box. The art style that we will use for projection mapping is line art.

We also break down the task for each of us and come out with a timeline for everyone to update their progression so that we know everyone's progress.

My task is to create a mood board for our project and also research the visual look of how we plan to do for our project to RMIT student so they can understand our new storyline and the overall looks of the project then only they give us the actual measurement of the boxes size for each scene.

During the weekend, our groupmate had found the 7 boxes for our project use.

We think that putting the 7 boxes without design the layout is too traditional and boring, so I also did some research on how we can arrange for the 7 boxes to the RMIT student to help them explore and brainstorm on how we can do for the 7 boxes.

Week 6
While waiting for the RMIT students template for every scene, we have our test out for the animation and we try to explore different types of the line art animation. 

There is some delay on this week, as follow the progression timeline we will get our measurement by this week but the RMIT students fail to send us the actual template.

Above is the measurement we got from them. We also have some argument on this week as we request to remove the light on the cloud and street scene because we think that the light of the box might affect the projection mapping but the RMIT student wanted to remain it. We also try to make the art style of animation more consistent by this week so that everything will look neat and nice.

We also had another argument on the end of this week as the RMIT student ask us to change the art style of the line art to a watercolor style.

We think that last-minute changes for the art style make the timeline more slow as the whole week 5 and 6 they didn't finish the scene boxes and waiting for our animation to continue while we also waiting for the actual size.

Week 7
As the RMIT lecture request, we redo another set of animation with watercolor style for them. But the outcome was not really good as our art style is all look different and the RMIT lecturer asks us to give some animation task for the RMIT student too.

We separate the animation job scope for everyone
- Lily, Andrea: All 7 scene art asset
- Chris: All the sound asset, art and transport scene
- Edrick: Whether, street scene
-Me: Food, animal and language scene.

We did the 2 scene using the art asset that provides by the RMIT student by this week and confirms the new animation flows and effects too as they need one week for the tasks.

Here are the art assets that provide by them.

Week 8
On week 8, we do the animation for the RMIT student based on the art asset that they provide.

After sending them the animation file, they will adjust the final outcome by using the mad mapper themselves.

Week 9
My task for the group project is to research what method can be used for the interaction of the project. And explore how to make it works. Here are a few methods that I have found.

1. Kinect Projector
2. QR code scanner
3. Touch board / conductive paint

Before the research, I have a discussion with Zikri then I realize that it was hard for us to make the people touch the balloon to interact with and change the animation with using the touch board and conductive paint. As this method, we will need to connect with the wire and plug and this will make the balloon easy to found. Pasting the touch board or pressure sensor will make the balloon easy to explode and it is also a waste if we color the whole balloon with conductive paint and hang the touch board on the wall.

After discussion with my groupmate, I suggest changing the interactive from touching the balloon to stepping with footstep. By using this method, we can hide the wire on the floor and cover with something.

• Kinect Projector (not suitable)

During class time, I had a discussion with Mr. Razif about the Kinect projector. He told me that the Kinect projector that we have on the campus is the old version and the function that I need is using the new version. Besides, to make the interactive work, our concept is to detect the pressure but the Kinect is only can detect for movement.

• QR code scanner (not suitable)

Besides, I saw an interactive projector project of a people was using the QR code, so I wanted to verify whether the QR code is suitable for our project. After my research, I found that with using the QR code, the place of the QR code must be consistent, and we need to have more sensor of the QR code base on the quantity we need for the real balloon. Plus the windy factor that causes the balloon to swing and also the sensor will make the real balloon get to find easily, so I ruled out the choice of this method.

• Touch board / conductive paint

This is the method that Zikri group use last time. After asking for Zikri advice of how the things work, I planned to buy the touch board set, we can buy a black paper and put on the floor, draw the footprint on the black paper with the conductive paint. Then connect everything to the laptop and control using the laptop. I ask the tutorial of the coding link from Zikri and plan to study the coding.  

Before the class ended, Mr. Mike said that the touch board set is expensive and actually we can try to use other things that have the same function to replace it so I will also need to research on the Arduino and raspberry pi as I don't have any knowledge or experience of these things. I also need to study on the coding of the touch board used by Zikri group last time.

After I study the tutorial link that sent by Zikri, I realize that actually, this method is not suitable to our project as the capacitive touch sensor of the conductive paint will need to have contact with bare skin, only the sensor will react. Zikri groups project is using hand to touch, ours is through stepping. So it won't work if we step with our shoes on. Even we use the other type of sensor (resistive sensor), it will work but the sensor pad will cost a lot.

So I discuss with my groupmate whether we can change to motion sensor instead of the touch sensor. Because that will probably save us some work and easier setup. I had done some research on the motion sensor. The concept is like when an object (could be anything that is solid) pass through the sensor, the sensor will then activate and the projection will change (through coding).

To understand more to the sensor things and Arduino, I did some research and studies on these things.

1. https://www.youtube.com/watch?v=nL34zDTPkcs

2. https://www.makeuseof.com/tag/arduino-technology-explained/

3. https://www.arduino.cc/en/Guide/Introduction

4. https://www.dfrobot.com/blog-657.html?gclid=Cj0KCQjwrdjnBRDXARIsAEcE5YmaNr1GFl6l_Cn13fNKcssGxpBDhPwovKX7hamnhHwABkDkcma4EMUaAmHfEALw_wcB

I also found a Taiwan Chinese Youtuber that posted some Arduino tutorials that guide about Arduino and the coding part.

1. https://www.youtube.com/watch?v=B4T8hzO0IYc

2. https://www.youtube.com/watch?v=TJ8IqbyuTu4

3. https://www.youtube.com/watch?v=6oh5jG1e-gQ

4. https://www.youtube.com/watch?v=hY33--luLYs

5. https://www.youtube.com/watch?v=b-Dnn5djs9o

6. https://www.youtube.com/watch?v=cWEJMusT-hI

7. https://www.youtube.com/watch?v=8lfrjiA4nAg

8. https://www.youtube.com/watch?v=QqjvsVVlRiQ

Motion sensor

• Ultrasonic Sensor

The ultrasonic sensor is a type of acoustic sensor divided into three broad categories: transmitters, receivers, and transceivers, Transmitters will convert electrical signals into ultrasound, receivers will convert ultrasound into electrical signals and transceivers can both transmit and receive ultrasound. Ultrasound transducers are used in systems which evaluate targets by interpreting the reflected signals.

• PIR sensor (backup)

PIR sensor, which is also known as Passive Infrared Sensor, is an electronic sensor that measures infrared light (temperature of living things) radiating from objects in its field of view. It does not radiate energy for detection purpose, only detects general movement and do not give information on who or what moved.

*It will be connected to the Arduino UNO

• Arduino

The Arduino that is used is Arduino UNO, which is an open-source microcontroller board based on the Microchip ATmega328P microcontroller and developed by Arduino.cc. It acts as the brain for the PIR sensor to communicate with the computer. Connecting the PIR sensor to the Arduino UNO will allow it to perform its programmed job.

*It will be connected to both the PIR sensor and the computer

*Will need a power supply to function

*code using Arduino Web Editor

• Arduino Kit (Resistor, Power supply cable, USB cable, Connecting wire)

Actually, both ultrasonic and PIR sensor can be used for our project, so we will be using the ultrasonic sensor and make the PIR sensor as a backup.

Week 10

I ordered the ultrasonic sensor, Arduino and Arduino kit from Lazada. Due to Raya break, it took me almost a whole week to receive the parcel. While waiting for the parcel, I start to figure out the coding for the Arduino.

In order to understand how Arduino kit could connect to each other, I have used several ways including google, youtube and also Arduino website to do the research. It took me a few days to understand how they could connect to each other and what are their functions.

• Breadboard

It is a solderless device used for a temporary prototype with electronics and test circuit designs. Most electronic components in electronic circuits can be interconnected by inserting their leads or terminals into the holes and then making connections through wires where appropriate.

• Jump Wire

A jump wire is an electrical wire, or group of them in a cable, with a connector or pins at each end, which is normally used to interconnect the components of a breadboard or other prototype or test circuit, internally or with other equipment or components, without soldering.

• Resistor

A resistor is a passive two-terminal electrical component that implements electrical resistance as a circuit element. In electronic circuits, resistors are used to reduce current flow, adjust signal levels, to divide voltages, bias active elements, and terminate transmission lines, among other uses.

• Led Light Bulb

LED light bulb is an electric light for use in light fixtures that produces light using one or more light-emitting diodes. Used to test if the circuit is working.

• USB cable

Used to connect Arduino to the computer. Coding can be uploaded from a computer to Arduino through this cable.

• Power Supply Cable

Used to provide power supply to the Arduino. It is connected to Arduino and battery on the other end. Arduino will function according to the coding uploaded as soon as power is supplied.

Later on, I started working on the coding part. I started learning to program for Arduino, which I have taken another few days to learn. I managed to learn basic coding by the end of the week.

Week 11

This week, we propose the “tunnel” idea to Mr. Kannan and Mr. Mike. Mr. Kannan is okay with the idea and suggests us to try out. We can also make the tunnel to a smaller size as long as the thing works. Mr. Mike then told us that the actual size for the tunnel idea is quite small and it might be a difficult part to bring the experience to the user. He suggests us to maybe think of using the VR so that the visual part and the spacing area can be brought out to the user. But the only disadvantage for this is the user can’t touch the things. This might stray away from our original idea, as we wanted the user to touch and feel to explore something.

So we think for a second plan for this project. We will maintain the method of using projector mapping and also sensor thing, but the outcome will be a different way. We plan to make everything in a small table, the projector will project the visual illustration from the top, and there will also project some button and there will be something for the user to touch and move. We will have a storyline, but the storyline will change based on the button that the user press.

I have decided to test if the Arduino is going to work. I have built a basic circuit where the led light bulb will keep blinking once the Arduino is supplied with power. I have coded a basic code, which is then uploaded onto the Arduino for it to function as programmed. After I have done all the preparations, I connected the Arduino with the power supply, the led light bulb blinks. Thus the first test is a success.

Week 12
For this week, as we came up with a new idea to add in the capacities touch sensor, we went to a shop to purchase the sensors. Hence, I have to further my research as touch sensor will now be added into the circuit.

My objective is to test if the touch sensor will work as programmed. I programmed it to light up the led light bulb once the touch sensor is activated. I spent several hours to complete the circuit. I then spent another few hours on the coding part on the next day. After everything has been prepared, I then uploaded the coding onto the Arduino, where the Arduino will work as programmed as soon as it is connected to the power supply. I then tested the touch sensor and the led light bulb lights up whenever I touched the sensor.

During the class section, we showed our ideas to Mr. Mike. I also showed the progress on the coding part. Soon after that, we realized that the touch sensor is quite sensitive because it can be triggered by a lot of things such as stone, table, metal, and book. This is a new discovery as I thought the capacitive touch sensor can be only triggered by our hand. After some additional research, I found that the sensor can be triggered by anything that is conductive or has dielectric different from air. Due to this problem, the idea we had earlier to manipulate the table will need to be changed so that the table won’t affect the sensor.

Also, during class time, we have tested the other two different type of touch sensors. The results were then compared to the initial sensor. The larger sensor came out to be inverted. For example, the initial sensor will be triggered once it is touched by hand and the led bulb will light up, but the led bulb for the larger sensor is constantly on, the light will only go off when the sensor is triggered. Moreover, the smaller sensor came out to be more sensitive compared to the other two. The sensor can simply be triggered by placing hand close enough and not touching it. Before we could test the smaller sensor, we had to borrow the soldering kit from the engineering lab to connect the sensor to wires as the sensor doesn’t provide any connecting part, unlike the other two.

Besides, as the sensor can be triggered by the table, we have decided to cover the table with a piece of cardboard to avoid the activation of the sensor. After we have measured the actual size of the projected area (table), my task is to draw a complete circuit with the estimated length of each wire and the sensors placements. In addition, I will need to help on some handcraft asset (five hand boxes, one sandbox).

After class, I went to Art friend to buy the cardboard and make the 5 hand boxes and also the sandbox.

Week 13

For this week we start testing the layout. I figure out the materials that we might use for the 5 boxes and also a sandbox with Chris. We arrange the motion sensor and also placement for the 7 touch sensor. After we confirm all the materials, we start designing the layout for everything based on the storyline and the boxes I have made. After all the simple setup I measure the length and design the pathway for the wire, Arduino and also for the touch sensor. So that everything will be easier and also easier for me to set up the wire with the touch sensor.

After the changes, I success to make the capacitive touch sensor work with the boxes.

I also complete the coding for the motion sensor by calculating the distance of our hand with the sensor and send signals to the led light. But I also met a problem as the coding for motion and the capacitive sensor is crashing so it will affect the signal whether can send out completely. Might need to change again for the coding.

Week 14
For this week, we did the 70% setup for the project. We confirm the placement for the projector and did the final setup for it. Due to some materials need some time to collect like the sandbox so we couldn't show the complete one. 

I success to make the motion sensor sends a signal to the led light when the covered box taking further than 20cm and also reset when the box is being put back.

I also fix the problem of the crashing of two different sensors playing in at the same time. To make sure myself won’t get confused and also the wire won’t get messy, I paste all the wire together. 

Besides, I also help Chris to illustration the 5 dragon head for the 5 boxes. 

Then I face the final part of my task, which is to send the signal to mad mapper with Arduino when the touch sensor is triggered. Actually, I have started to research for this since the project started but until now haven’t got the correct code for it. In order to make it work, I have used a few methods. According to the research I have done, I have tried using Processing to act as the media that could communicate with Madmapper. It should read the serial messages from the Arduino and translate it, then send it to Madmapper. However, the code is not working and it failed. I have also tried inserting the video into Processing, but it failed too.

After that, I tried to import the keyboard function from the library into the Arduino app. It should allow Arduino to send a signal that was translated into alphabets to the computer. I thought this would work as I could set Madmapper to read the alphabetic signal. But unfortunately, the Arduino I have bought doesn’t support the keyboard function. It is supported on most Arduino devices, but not the one I bought.

Then, while I was searching the internet for more methods, I have found an expert on programming, who has done similar research to what I was trying to do, but not many details were included. Hence, I have contacted him and he replied to my email. However, he wasn’t familiar with Madmapper, and he suggested me to skip Processing as it is not really needed in my project. He wasn’t able to give any advice other than that.

Finally, I have decided to have another attempt on Processing, since it sounds more feasible to me. I will need to look into more tutorials on the coding since Processing is using Java, and it’s quite tough for one who has not learned it before. Hopefully, it would work.

Unfortunately, it didn’t work as planned. I failed to link Arduino to Madmapper through coding. I have decided to change from Arduino board to Bare conductive touch board in the very last minute. We rent the touch board with the person that Zikri's group rent for two days. To be honest, it is easier for me to use touch board as I have gone through some research about it and I could find tutorials and examples for it to connect to Madmapper.

The touch board is totally a new thing. It uses conductive paint instead of a capacitive touch sensor, but the theory is the same. As long as one touches the sensor, it will trigger the board and thus sending a signal to change the animation. The wiring is the same, I’ll just need to change the capacitive touch sensor to cardboard painted with conductive paint and to change the Arduino board to touch board. They both use the same software, which is the Arduino software.

Before the day of the final presentation, I went to campus to set up everything including the decoration, projector set up and all the wire and copper tape to final testing the coding to make sure everything works.

At first, I randomly put some preset on the mad mapper to check everything whether works a not.

At first, I try to move the copper tape and also double-check with the decoration that I used to cover the touch sensor to found out why the sensor keeps triggering. Then, I realize that when I insert all the 8 touch sensor thing on the touch board it might make all the touch sensor doesn't work because of the touch board too sensitive. When the touch board only left like 5 to 6 wires, the code on works well and the sensor won't keep triggering. So, I remove all the "wrong" choice and only left the correct button to make the project works.

After the final testing for the coding and touch sensor thing, everything works well. I managed to complete the codes within the time given and hence the project is a success.