Final(ly) Evaluation

What I have Learnt

Throughout this project I have learnt the fundamentals of Design Iteration, going from point to point; Analyse, Design and Testing. Having been reminded of the process which I need to take when designing a piece of work and documenting it’s progress. I have also learnt parts of Processing code and how to manipulate works to portray an image I aspire. These processes have widened my knowledge of design processes and given me an insight to the lengths I need to go to to come to a final piece which works; following the track I took the development of lighting, motion and colour was expanded into and understanding of spaces, and social interacts. Touching on the knowledge I already possessed from life experiences I adapted what I was taught together to create a piece of work which questions the motion of people and the motion of life (what happens around us as people).

What I have Achieved

I achieved the adaptation of a piece of processing code which lead to me creating a digital environment that portrays the physicality that there is motion in everything, even if it can’t be seen. I developed an “Almost Particle System” which represented a particle system that allowed passers to see the environment they stood in evolve in to something which is constantly in motion. The System had a main focus on what was going on around people rather than focusing sourly on the person viewing and when a motion was performed by people the particles would react, showing that every action has a reaction. Symbolising that that there is always a cause from an effect. I also achieved a new found knowledge of depth and concept understanding, relating the two things together because of how my concept was affected due the depth of the space I was using as well as text size.

What Went Wrong

The biggest mistake I made was not taking into account how big the space was in the foyer; I had been doing all my experiments and designs in close proxemics to myself and when it lead to me setting up in Weymouth house the particles did not respond to the environment as well. The particles would not react with each other as well as when the camera was focused on something closer, and the code could not blank out the entire person when they walked past. This had a massive effect on the final piece as a whole, leading to me installing the system and changing the code as I went along, giving different interpretations of what could be used. The benefit of this was a wider knowledge of how the space was effect by the particles, giving me an insight into how cameras react to a large and small space as well as the motion of people walking by.

What I Would Change For Next Time

Next time I would write the Particle System myself from scratch, at the point of deciding to write the system my knowledge was not acute enough to create it. I would analyse the space to a proper degree and begin experimenting in either the space I would be installing in or in a similar space . I would like the Particle System I would create to react to people when they walk on screen, first starting off a showing the whole person and then them evolving into Particles. I would also do wider research in other systems which are related to particles, giving myself a wider insight into what I could have adapted my skills to leading to a better result.

Feedback

Being down in the open space there wasn’t many people walking through at the time I had chosen to set up and rumor on the DMD street was that a few of my course mates where on the second floor, so I decided to take the installation up to them and get some undergraduate feed back.

Reminds me of the abstract pictures that phychatrists show their clients “What do you see?”

The closer you get makes it feel like your walking into a portal.

The Pixels make it sometimes hard to understand what is going on in the screen. – Sil 

 As a media student, the installation’s deconstructs us into pixels as individuals, an interesting concept for me as it communicates all users of media essentially becoming products of technology. – Becky

I think this piece was really interesting idea, I instantly thought of ‘every action has an equal and opposite reaction’ when interacting with the sketch. – Bob

I’m very fond of this idea, but I don’t think you considered the size of the room enough for it to work in the way you wanted. Still a very nice piece and show’s part of you concept very well. – Alex

My thoughts on the feedback are very light hearted, it’s a shame people didn’t really understand to concept fully, but that is probably to do with the unexpected reaction with the bigger environment and the code, this will add to what I would change next time. From what I understood thought; my classmates thought the piece was really nice and a simple, but a well done manipulation of AsciiVideo this could be pushed much further.

I could image a piece of work like this being expressed in away which could be displayed in a location like a museum (Maybe Tate Modern), having an entire wall project people looking at the piece, and they see themselves is the Ascii format.

Down in the Open Space

Here is the first image of a person walking by the system, as you can see the particles are constantly moving which is what I wanted to be portrayed; showing that we are not the only things which are moving in the world, even if you can’t see it, it doesn’t mean it hasn’t got a response, the camera is showing this.

The problem here is the way the particles are responding to each other; because the camera has to pick up a much bigger space there aren’t enough particles to cause a big enough response to react to the man walking by. This can easily be fixed by adding more particles.

Here I flipped the code adding lines, as seen previously in my designs. The benefit of this was the reaction the camera had towards the person on screen as well ash the particles responding much better to his movements. Expressing what I wanted to a much better light.

Here is someone reacting to the system, I think the same person who had noticed the work and come back to see what happens. The person was eager to see how the particles reacted by doing star jumps and moving closer to the camera, this helped show the space been left blank where the persons body was, but the particles around still staying in motion.

This is the biggest jump with the code, having made the particles bigger and also reversing the negative space as seen in previous experiment I wanted to see how to bigger space would react. It in fact not being beneficial to the initial idea at all, the particles don’t have the motion which always shows movement anymore and the bold colours of the persons body isn’t strong enough to draw attention towards the particles.

I would describe this as the worst out come for the digital environment; it lead to the camera lagging once again and the particle where not responding enough to represent my concept, but the colours are nice and you can see the Tardis very well.

Overall it was a shame that there weren’t many people in the foyer at the time because my response may have been more deeming, but the way the almost particle system symbolised the space looked very nice and portrays my concept ideally, but not to the standard I want.

Edited code for second installation below –V–

Capture video;
boolean cheatScreen;

String letterOrder =
  "------------------------------------------" +
  "0oO0oO0oO0oO0oO0oO0oO0oO0oO0oO0oO0oO0oO0oO";
char[] letters;

float[] bright;
char[] chars;

PFont font;
float fontSize = 1.8;


void setup() {
  size(640, 480);

  filter(GRAY);
  // This the default video input, see the GettingStartedCapture 
  // example if it creates an error
  video = new Capture(this, 160, 120);
 
  
  // Start capturing the images from the camera
  video.start(); 

That One PARTicle That Just Can’t Keep Up

for (int x = 0; x < video.width; x++) {
int pixelColor = video.pixels[index];

int r = (pixelColor >> 16) & 0xff;
int g = (pixelColor >> 8) & 0xff;
int b = pixelColor & 0xff;

Here I have made a slight changes to the colour, I have changed the attributes from “r,g,b,” to “h,s,b,”. The reason for this is to add a clearer image to what is been projected, allowing people to see what is going better. At the moment I feel the way the particles are looking doesn’t look clear enough and show the blank spot where the person is well enough, changing these attributes will hopefully improve this.

for (int x = 0; x < video.width; x++) {
int pixelColor = video.pixels[index];

int h = (pixelColor >> 16) & 0xff;
int s = (pixelColor >> 8) & 0xff;
int b = pixelColor & 0xff;

Also changing the font size again, this time too 2.5.

As you can see I have encountered a little problem

The problem I have encountered here is the camera cannot keep up with my movements, this is a big deal because of the pace people walk through Weymouth House, they will not notice that there bodies are being blanked out. Trying to fix this problem I added in a fps command;

frameRate(60);

This had no response to the work; nothing changed so I fluctuated between higher and lower frame rates and this still had no difference….So I tried cutting the amount of letters (particles) in half to reduce the collisions between them, again this had no difference. This problem is a tricky one to figure out, so I’m going to move from w119 and head down to the walking space and see if this makes any difference.

What you’re seeing here is the first view of the production in the Weymouth House walk through; me stepping away from to webcam to see if it is clear that I am standing there. The lag on the image isn’t as bad as it was up in W119, that could be due to the wider every of space, I did also reduce the size of the particles to allow more space for movement, hence dark spots.

I’ve Made A Decision

I’ve decided to use more than one of the code manipulations in this space, considered as further experimentation I am going to explain it as; “which code reacts better to the bigger environment.”

The Line Between Two Designs

The next step to try a different interpretation of what I am doing, before I get really deep into the code I need to state which direction I want to go in, either working the particles like the plan has always been or delving into something else, working with lines rather than dots or maybe even something completely different. Options being numbers or playing around with size of the letters, not like before, in turns of absolutely massive or really tiny. I will have to prepare myself whether lighting will have a big effect on the sizes and if this could effect the distance the camera could see.

I’ll just get straight into trying out the lines, code used below –v–

String letterOrder =
"-__--__==__--__==__--__==__--__==__--__==__--__==__--__==__--__==" ;
char[] letters;

Using equals signs and similar line shapes I have replaced all the zeros, here are the results –v–Yes! That is just a black screen!

There is obviously a problem here….

Doing further insight into what’s gone wrong here and not having any errors the problem is with density; the letters need to have a curtain density for them to placed onto the screen. The problem with using lines and lets which are very thin is the density of the symbols isn’t computed by the code leading to the blank screen.

Because I have come across this problem I am going to mess about a bit with the code and see what I come up with.

PFont font;
float fontSize = 3;

Increasing the size of the font the 3, usually being 1.5 you can see that the size of the letters is cause a bigger confrontation with each other dismantling the image and making it look quite (inaway) pixelated.

I not sure if I really like this effect, it has quite a lot of lag which is shame and does portray my concept to it’s full degree.

String letterOrder =
" .`-_':,;^=+/"|)\<>)iv%xclrs{*}I?!][1taeo7zjLu" +
"0oO0oO0oO0oO0oO0oO0oO0oO0oO0oO0oO0oO0oO0oO0oO0oO";
char[] letters;

This is the reverse of the one before; having the particle on the inside, to do this I used what I learnt about density in the letters and made the top like of letters all lines and the bottom line ‘O’s’, creating what is above.

String letterOrder =
"------------------------------------------------" +
"0oO0oO0oO0oO0oO0oO0oO0oO0oO0oO0oO0oO0oO0oO0oO0oO";
char[] letters;

 Here I have kept the top line of text which has worked out quite well for me; the way the pixels are reacting to my body shape and the outside if very similar to one of the drawings I have.

Here I was trying to show the difference I want between the body and what’s outside the body, and with the size of the text and the way code is laid out in the form of letter density order it has resulted in the main focus being what’s happening outside of the structure. Another thing I have noticed is the way the shade black doesn’t have any letters, it is a completely blank space when on camera. When I move my body the letters (particles) react in such away that it show’s what is happening when the motion takes place, even if it can’t be seen the camera is showing it, exactly what my concept is.

The Next Step

I want to make the particles clearer, giving the image more clarity and nicer imagery. My next step of experimentation is to mess about with the colour attributes and how the particles are represented.

The A to Zero Design Process

From the starting point my main objective is to achieve the best representation of what a particle system would look like by manipulating the code, here are some sketches of what I’m trying to show.What we have with the original piece is all the letters of the alphabet and grammar creating the environment

String letterOrder =
" .`-_':,;^=+/"|)\<>)iv%xclrs{*}I?!][1taeo7zjLu" +
"nT#JCwfy325Fp6mqSghVd4EgXPGZbYkOA&8U$@KHDBWNMR0Q";
char[] letters;

The code above ^

I am going to experiment with these letters and change them into symbols like lines and dots; trying to portray the movement in the environment to best of my ability.  In the sketch above I have tried to show little circles which I can develop the code into, I’ve changed the code see above to have ‘0’ to see what would happen.

String letterOrder =
"000000000000000000000000000000000000000000000000" ;
char[] letters;

As you can see the results are not what I want, the ‘0’s are not moving like particles and the image looks more like a stained glass effect. The way that I may fix this is if I replace the ‘0’ with different size round shapes e.g ‘O’ and ‘o’.

The results made no different, so I started playing around with the font size of the letters, replacing the code with this:

String letterOrder =
"0oO0oO0oO0oO0oO0oO0oO0oO0oO0oO0oO0oO0oO0oO0oO0oO" ;
char[] letters;

float[] bright;
char[] chars;

PFont font;
float fontSize = 1.5;

The Results –v– Changed fontSize from ‘2’ to 1.5.These are the results I’m looking for, what’s great about this is the way the “Particles” are knocking against each other, portraying the beginnings of my concept. The simple change in size of the letters and made such a big difference that a static image has become alive.

The smaller the particles the more that are needed to fill out the image. The expression here relates to the concept, but there are a few problems.

The Problems

I’m not convinced with the way motion is being shown, yes it is the particle portrayal that I want, but it’s something about the colour that I’m not really enjoying looking at. It’s important that I get the colour the way I want it, so what I’m going to do is mess about with which colours are being used, the code I am going to manipulate is below –v–

pushMatrix();
for (int x = 0; x < video.width; x++) {
int pixelColor = video.pixels[index];
// Faster method of calculating r, g, b than red(), green(), blue()
int r = (pixelColor >> 16) & 0xff;
int g = (pixelColor >> 8) & 0xff;
int b = pixelColor & 0xff;

// Another option would be to properly calculate brightness as luminance:
// luminance = 0.3*red + 0.59*green + 0.11*blue
// Or you could instead red + green + blue, and make the the values[] array
// 256*3 elements long instead of just 256.
int pixelBright = max(r, g, b);

As well as messing about with the colour I’m also going to see what the piece would look like if I where to use lines rather that circles. The reason I came to this was because of some of the sketch designs I created, I found that the way I was using my pen to draw the structure of peoples bodies where been show with a sound wave mimic. Let’s check what this looks like.

Ascii Art (Design)

Video ascii art is areal time post processing effect that will transform any video into ASCII art. The effect is created in a shader and use Kickjs engine. You can see the ASCII shader in action on http://www.kickjs.org/example/video_ascii_art/Video_Ascii_Art.html.

This is a lovely example of what someone has done to adapt the Ascii Video to be easily used by other users, simply uploading a video it will develop it to be produced with letters, the main feature of a Ascii.  To me when I look at this I am getting an impression of particles, the slight movements that the letters are making looks like the way I want to particles to react in an image, the only problem being that I think the image is too clear here, you can clearly see what is happening and I don’t want that to be the case. I want people to react to what is being displayed and try and disrupt it.

Here below is what I am working with, I have used my phone as an example of the way the camera and code react to the colours to change them into letters. I’ve added the code below.

/**
* ASCII Video
* by Ben Fry.
*
*
* Text characters have been used to represent images since the earliest computers.
* This sketch is a simple homage that re-interprets live video as ASCII text.
* See the keyPressed function for more options, like changing the font size.
*/

import processing.video.*;

Capture video;
boolean cheatScreen;

// All ASCII characters, sorted according to their visual density
String letterOrder =
" .`-_':,;^=+/"|)\<>)iv%xclrs{*}I?!][1taeo7zjLu" +
"nT#JCwfy325Fp6mqSghVd4EgXPGZbYkOA&8U$@KHDBWNMR0Q";
char[] letters;

float[] bright;
char[] chars;

PFont font;
float fontSize = 1.5;
void setup() {
size(640, 480);

// This the default video input, see the GettingStartedCapture
// example if it creates an error
video = new Capture(this, 160, 120);

// Start capturing the images from the camera
video.start();

int count = video.width * video.height;
//println(count);

font = loadFont("UniversLTStd-Light-48.vlw");

// for the 256 levels of brightness, distribute the letters across
// the an array of 256 elements to use for the lookup
letters = new char[256];
for (int i = 0; i < 256; i++) {
int index = int(map(i, 0, 256, 0, letterOrder.length()));
letters[i] = letterOrder.charAt(index);
}

// current characters for each position in the video
chars = new char[count];

// current brightness for each point
bright = new float[count];
for (int i = 0; i < count; i++) {
// set each brightness at the midpoint to start
bright[i] = 128;
}
}
void captureEvent(Capture c) {
c.read();
}
void draw() {
background(0);

pushMatrix();

float hgap = width / float(video.width);
float vgap = height / float(video.height);

scale(max(hgap, vgap) * fontSize);
textFont(font, fontSize);

int index = 0;
video.loadPixels();
for (int y = 1; y < video.height; y++) {

// Move down for next line
translate(0, 1.0 / fontSize);

pushMatrix();
for (int x = 0; x < video.width; x++) {
int pixelColor = video.pixels[index];
// Faster method of calculating r, g, b than red(), green(), blue()
int r = (pixelColor >> 16) & 0xff;
int g = (pixelColor >> 8) & 0xff;
int b = pixelColor & 0xff;

// Another option would be to properly calculate brightness as luminance:
// luminance = 0.3*red + 0.59*green + 0.11*blue
// Or you could instead red + green + blue, and make the the values[] array
// 256*3 elements long instead of just 256.
int pixelBright = max(r, g, b);

// The 0.1 value is used to damp the changes so that letters flicker less
float diff = pixelBright - bright[index];
bright[index] += diff * 0.1;

fill(pixelColor);
int num = int(bright[index]);
text(letters[num], 0, 0);

// Move to the next pixel
index++;

// Move over for next character
translate(1.0 / fontSize, 0);
}
popMatrix();
}
popMatrix();

if (cheatScreen) {
//image(video, 0, height - video.height);
// set() is faster than image() when drawing untransformed images
set(0, height - video.height, video);
}
}
/**
* Handle key presses:
* 'c' toggles the cheat screen that shows the original image in the corner
* 'g' grabs an image and saves the frame to a tiff image
* 'f' and 'F' increase and decrease the font size
*/
void keyPressed() {
switch (key) {
case 'g': saveFrame(); break;
case 'c': cheatScreen = !cheatScreen; break;
case 'f': fontSize *= 1.1; break;
case 'F': fontSize *= 0.9; break;
}
}

 

This piece of code relates to my general idea to the best that I can find, the position I am in now is take the sketches I have created and the concept idea I have and manipulate this piece of work to the fashion I want it to represent. Lets get drawing.

What Stage Am I At? (The Boring Part)

What has happened so far.

I’ve analysed a concept

What I want to portray with my digital environment piece is the concept of motion; not focusing sourly on humans being the main thing which creates movement in the world. I want to bring to people’s attention that, lets say when you move your arm (you can clearly see your arm moving) there is a reaction around the space the motion is taking place. There is a reaction to everything that happens, even if you can’t see it, it still exists. 

I came to this culture because of the recent essay I have written on ‘Participatory Culture’ where fans of a franchise take the franchise into their own hands and create a piece of work in their own image. The culture creates an action in response of something which already exist, but may not be seen. This added to life experiences of everything you do having a reaction, whether it’s not replying to a text or calling the wrong person there is always a response somewhere those causes.

I’ve analysed the space

Using the Foyer in Weymouth House we have been offered one of the screens to install our systems, this is located in an eye catching spot so people will notice the system and also the camera will be facing where most of the action happens (where people walk through). Already knowing that we have been given access to this location I conducted some research with the use of the “Independent Dorset” brief, allowing me to get an insight to the duration people come into the foyer and leading to me having a wider knowledge of how I will present my digital environment.

I’ve analysed Particle Systems

Using something which already exists “Particle Systems” I did some research into some Systems which have already been written and how they look, the use of this gave me some ideas stretching from an environment which is 100% made out of particles and when motion is detected the particles respond with movement. With further research and advise from Liam my seminar tutor the complexity of writing a Particle System was explained as it may be quite over whelming and not work, but I really wanted to stick to the idea. This came back to the concept I have and the fact that I want this piece of work to be seen as art. 

I analysed the possibilities of Particle Systems

Using research and my note book I went into how I could take this possible particle system and create something on my own, doing sketches and looking into Examples which Processing supply like ‘OpenCV’ and ‘Punktiert’ I analysed how the imagery in particles is portrayed. The results of this was looking how the particles where presented, something being small and big and others being shown with the way they move rather than the dimensions of them, but all where really quite beautiful. Some people even found ways of getting the particles to move in such away they created images, just with how to particles moved around the page. Moving forward this took me to the point of wanting the camera to notice people rather than the entire space just being turned into Particles, this is quite a tricky thing to succeed in as the code is incredibly complex and I didn’t know how to write it, but I did recall that using a Kinect webcam already has the programming installed to recognise people from structure and with the library called ‘SimpleOpenNI’ I could manipulate this. 

The problems I confronted

I thought the possibility of this going wrong where quite minor, but when it came to it the code which had been written was specific for a kinetic camera, rather than the Logitech HD system I had rented, this was not a massive problem I could just rent out a Kinect. Before trying this I tried to manipulate the code I already had so that the output would either be my webcam on the Mac or the webcam I had rented, me copying and pasting basic camera processing into the file I already had I had got to the point where the camera would turn on, but then deactivate itself. Leading to 100% having to use the Kinect. The Kinect did NOT work because of the model number being to recent therefore having to abandon all the ideas I had. 

How I have solved the problem

I did not have enough time to write a particle system with to Kinect camera, putting me into a position where I decided I’m going to create something called “An Almost Particle System” this being a piece of work which is inspired and symbolises what a particle system looks like, but is in-fact not one. Using inspiration from a work called ‘AsciiVideo’ I will create a piece which uses motion, colour and possibly light density to portray action inside the space. Enough said.  

Now what?

The next blog posts are going to be the design process of my system.

 

 

I Found Something

Doing my sketches and scrolling through examples I have come across something.

AsciiVideo

I’ve hit a stroke of luck here, this piece being one of the examples of Processing it has all the aspects I am looking for;

Colour, Motion and Light

Here the colours are being replaced with ‘letters’ and they are hitting against each other to show constant reaction, this is the effect I want to try and portray (showing constant motion). From what I understand (what I see without looking at the code) the letters have slight tints dependents on the lighting in the room, but it’s not a big feature, the biggest feature here is the letters acting as the image. Very similar to the way particles would act.

I am going to develop on these letters and see where it could go. The reason for this being it’s structured way of portraying what the camera can see; it’s showing another way of seeing life and to me it looks quite like a particle system relating back to what I want my Concept to be. I could try and adapt these letters in other formats, replacing them with little dots may give the best portrayal of particles.

 

 

 

You Live and You Learn

The Kinect, mentioned in the previous post as something which would be an absolute life saver I managed to get my hands on one from the university. I was over joyed that they had one and thought all my problems would be solved, I was 100% wrong…just spent the last three hours typing away on Terminal and downloading different programs so that Processing could ready the Kinect and use it as an output, to find that it was NEVER going to work because the model number of the kit is too new. I needed  an older version model, mine being 1473 and the one i need having to be 1414.
As you can imagine this is quite frustrating, putting a stunt on the work I was going to create.

But no worries I can still get round this; the whole Kinect idea is now dropped I am just going to use a normal webcam. The down side to this is the benefit of having the Kinect is it’s ability to recognise depth and people, saving me having to right the code I will find another way round this.

What I’m going to do now

The job now is to find a fix for this problem, already thinking about it whilst I slowly failed at terminal I will have to focus the camera to recognise movement rather that big shapes (like bodies), below is an example of what I wanted to achieve with the Kinect camera. What you are looking at is one picture (the middle) have the persons entire body being the blank space the particles being in motion around, the other is the reverse of this, body being the particles and the blank space being around. I can still do a basic idea of this which is going to be much simpler than what I planned.The Fix

Using;

• Colour
• Motion
• Lighting

I will be able to adapt a similar, but complete different interface.

Colour

• Using the change of colours I will adapt the webcam to base the code of recognising colour, so imagine in the picture below the lines are red and the persons body is blue; when the blue overlaps the red you will see movement. Relating to the way I wanted the particles to move.

Motion

• Having the Particles been replaced with colour (making colour the main focus) the next key aspect is motion; it is important to make sure that Motion is recognised when passers walk by, I will have to think about away of the colour not being static having it in a focus which shows the world moving even if it is not.

Lighting

• The use of lighting is something I could add to give this idea an extra sparking, having different tones of colour to create a different perspective when being output, I will look into this more later.

Time for some more sketches