Play Computer games with hand gesture recognition using Matlab

To do this experiment,just you need an webcam and matlab2012a or 2012b

Now we are gonna do live motion detection,the below live detection code is not written by me,i got it from this link http://arindambose.webs.com/htmls/Motion%20Detection%20in%20LIVE%20Video.html

vidDevice = imaq.VideoDevice('winvideo', 1, 'YUY2_640x480', ...
'ROI', [1 1 640 480], ...
'ReturnedColorSpace', 'rgb');
optical = vision.OpticalFlow('OutputValue', 'Horizontal and vertical components in complex form');

maxWidth = imaqhwinfo(vidDevice,'MaxWidth');
maxHeight = imaqhwinfo(vidDevice,'MaxHeight');
shapes = vision.ShapeInserter;
shapes.Shape = 'Lines';
shapes.BorderColor = 'Custom';
shapes.CustomBorderColor = [255 0 0];
r = 1:5:maxHeight;
c = 1:5:maxWidth;
[Y, X] = meshgrid(c,r);

hVideoIn = vision.VideoPlayer;
hVideoIn.Name = 'Original Video';
hVideoIn.Position = [30 100 640 480];
hVideoOut = vision.VideoPlayer;
hVideoOut.Name = 'Motion Detected Video';
hVideoOut.Position = [700 100 640 480];

nFrames = 0;
while (nFrames < Inf)
rgbData = step(vidDevice);
optFlow = step(optical,rgb2gray(rgbData));
optFlow_DS = optFlow(r, c);
H = imag(optFlow_DS)*50;
V = real(optFlow_DS)*50;
lines = [Y(:)'; X(:)'; Y(:)'+V(:)'; X(:)'+H(:)'];
rgb_Out = step(shapes, rgbData, lines');
step(hVideoIn, rgbData);
step(hVideoOut, rgb_Out);
nFrames = nFrames + 1;
end
release(hVideoOut);
release(hVideoIn);release(vidDevice);

But after this Ive changed this a little bit and  made a different thing,hope you like it.

So the code i got from the above link,well im pasting it here,

Now again use the redbox around the motion

rectangle('Position',cen,'EdgeColor','r','LineWidth',5);

Now perform a mouse click operation whenever a redbox appears in the video,get the code here

So,this is how your computer player will fire or do some other stuff as per game settings

And move the mouse pointer as per box location,code is Here

By this your player will switch it's view,

Now after doing all this,

1.Run your matlab code

2.Minimize the video stream

3.Run the game you want to play

I've tried this with some popular games,like counter strike,IGI 

Well this is a very basic idea,you can implement it further....you can also add some keyboard operation for performing other tasks,Keyboard Code here

try it it will definitely work... Thank u

Press like and leave comment

Vehicle Counting System in Matlab

1. Input a video of vehicles ex. test.avi
2. Now extract the frames and save in a specific folder
    Get the code here, one of my previous post
3. Detect the motion of the vehicle in the video
    Get the code here

Now if everything has gone alright,we can begin.

We just make very simple changes in the previous code to make the thing work perfectly.

Step 1 : Watch the redbox around the moving object.Now insert this code in there

             rectangle('Position',cen,'EdgeColor','r','LineWidth',5);
     
             Here 'cen' is the centroid

Step 2: Now the most important step,insert this code after that

        txt=text(no, strcat('car: ', num2str(round(no))));
        set(txt, 'FontName', 'Arial', 'FontWeight', 'bold', 'FontSize', 12, 'Color', 'black');

It will display the number of cars. 'no' is a variable which is set to '0' at first.But when ever a motion is detected in the video 'no' will be incremented by 1,that is how we get the vehicle count.

But the basic problem what you will face in this method that is,here we are considering frame after frame,so the 'no' variable gets incremented each time,but thats not what we want,we want the actual count,so we are gonna need a FLAG variable which indicates if 'no' is incremented by 1 for a particular vehicle then it wont increment for the specific one again.

like this,
FLAG=0;
if FLAG==0
   no=no+1;
   flag=1;
   frame=frame+1;
end
else
   frame=frame+1;
end

So now it is done,Thank You :)
Press like and leave comment

Screen Capture With Matlab

screencapture.m

clc;
r=java.awt.Robot;
tool=java.awt.Toolkit.getDefaultToolkit();
rectangle=java.awt.Rectangle(tool.getScreenSize());
image=r.createScreenCapture(rectangle);
fileh=java.io.File('scrcap.jpg');
a=javax.imageio.ImageIO.write(image,'jpg',fileh);
disp('done');

Bestfitness matlab code using neural network

    clc;
    a= [39.8878 39.9947 39.9544 40.0285 39.9263 40.0013 40.9078 39.7623 40.9278];
    b=[0.6016 0.6016 0.6045 0.6074 0.6067 0.6074 0.6052 0.5998 0.6056] ;
    c=[100 100 100 100 100 100 100 100 100];
    y=a+ b.*c;
    I=[a; b; c];
    T=y;
    R=[0 1000; 0 1000 ; 0 1000];
    S=[5 1];
    net = newff([0 1000;0 1000 ;0 1000],[4 1],{'tansig','purelin'});
    O=[100.0439 100.5907 100.4028 100.7729 100.5963 100.7457 100.5964 99.7375 100.6877];
    net=train(net,I,O);
    O1=sim(net,I);
    plot(1:9,O,1:9,O1);
     net.IW{1}
     net.LW{2,1}
     BestFitness=sim(net,[39.8595 0.6023 100]');

Making a Photoviewer with GUI using Matlab

Code :


clc;
imr=input('Enter the full path of the image frames:','s');
somefolder=imr;
filelist=dir(somefolder);
for i=1:size(filelist,1)
    if filelist(i).isdir ~=true
        fname=filelist(i).name;
        if strcmp(fname(size(fname,2)-3:size(fname,2)),'.jpg')==1
            tmp=imread([somefolder fname]);
            if size(tmp,3)==3;
                tmp=imresize(tmp,[400,400],'bilinear');
                imshow(tmp);
                pause(1);
                disp([fname,':loaded']);
            end
        end
    end
end

Building GUI :

If you dont know how create basic matlab GUI then you can see my previous post :

Building GUI with Matlab

Step 1 : I have made a very simple interface,drawn just a button  and an axes

Step 2 : Now open the GUI .m file by clicking view and then m-file editor

Step 3: Now paste the above code in the pushbutton_callback function and save

Step 4: Now run the code,press play button,then it should ask for the image folder path,write the path press          enter it will show the images

Building GUI with Matlab

1. Type 'guide' on the console and press enter



2. Now select a guide template and mark 'save on start up as' option then change the name,ex. test.fig
then press ok.

3. A screen will appear like this one...



4. Now you can draw button,use axis for showing images or videos,and make your own GUI.

5. To edit the matlab code do this,



6. Copy the matlab code what you want to execute after pressing a button within that particular button function

7. Then run your code pressing 'f5' it should show the GUI you built.

Virtual Mouse using Hand gestures Matlab Code

1. Take a sample Video

2. Use any motion tracking algorithm like camshift or kalman to detect and track the motion of your hand movements.

Get code here : Motion detection and tracking matlab code

3. Now track the centroids

 like this ----
        bb = stats(object).BoundingBox;
        bc = stats(object).Centroid;

4. Now move the cursor as per centroids last recorded location

use this piece of code to move the cursor


         import java.awt.Robot;
         cursor = Robot;
         cursor.mouseMove(bc(1), bc(2));


5. Clicking mechanism :

Now where you want a clicking operation during motion detection use this code

import java.awt.Robot;
import java.awt.event.*;

cursor = Robot;

cursor.mousePress(InputEvent.BUTTON1_MASK);

For more hardware access help : Click here

Real time Video Recording and Save as AVI matlab code

Here is an easy way to do this using my older Posts :

Just first Run this file  :

Capture and save Frames using Webcam matlab Code

Now run this :

Frames to AVI using Matlab Code

*just combine them both and save as a single file

Guess what... its done... :D

So,what I want to say is that always look at the smaller modules,combine them,Make your own project...

Frames to AVI using Matlab Code

avi = avifile('output1.avi');
avi.COMPRESSION ='None';
for i =1:numOfFrames
   %%%read image
   %%% call imread or related something
   %%%%%%%%%%%%%
   avi = addframe(avi,img);
end
aviobj = close(avi);

Reverse thing,you may also see Video to frames

Capture and save Frames using Webcam matlab Code

clc ; % clearing the command window
n = input(' Enter the number of photos to be taken: ');
intervel = input(' Enter the time(seconds) gap between succeessive photos: ');
photosave = input(' Do you want to save the files(y/n): ','s');
disp('Please wait...');
outputFolder = fullfile(cd, 'frames');
if ~exist(outputFolder, 'dir')
mkdir(outputFolder);
end
obj = videoinput('winvideo',1);
preview(obj);
disp('Press Enter to start after webcam initialization.');
pause;
disp('First shot will taken after 1 second');
pause(1);
for i=1:n
img=getsnapshot(obj);
image(img);
if(photosave == 'y')
outputBaseFileName = sprintf('fr%d.png',i);
outputFullFileName = fullfile(outputFolder, outputBaseFileName);
imwrite(img,outputFullFileName,'jpg');
end
pause(intervel);
end
closepreview;

disp('The program successfully taken the photos');
disp('Done.');

Access Webcam using Matlab

webcam.m

clc;
vid = videoinput('winvideo', 1, 'RGB24_320x240');
preview(vid);