Matlab: Image Processing: twelve-sided dice with numbers

-

(German) This post is like the post before and describes the identification of the numbers on a twelve-sided dice with numbers from 1 to 12. Also automatically rolled by the dice rolling machine of  my brother.

Dice identification:

First of all the images are loaded into Matlab and the size is reduces to 200x200 pixel size. The with rgb2gray the image is reduces to a gray-scaled image and the gradient is used.
The gradient and the normal image is converted into a binare image with a treshold. Then the gradient images is morphologically closed and multiplied with the normal tresholded image and afterwards morphologically dilated.
After this the big for loop over all images is  executed. The the watershed is applied and the middle of the image with my own function is determined, which just calculate the mean of all columns and rows where the image is one. From this middle a big area is chosen that the rolled number is for sure in this area.
Then the biggest area in the watershed transform is searched and if the middle of this region is fully in this area and the deviation to the middle is small then this is the first part of the number. This is repeated for the next four biggest parts with the difference that now the distance to the middle has to be at least a value (so that a nine won't also take the middle in the nine as a part) and the part has to be bigger than a treshold that little deviations are filtered.
After this all parts are sticked together so that one has just the number in the middle itself.
Then the new middle of the image is determined with a smaller area of 51x51 pixels and the number is cut out.
Then the morphology skeleton is applied that one just has a one pixel thick number left which is dilated to a 3 pixel thick number.

The whole process you can see in this graph (first column is the original image, 2: tresholded, 3: treshold times gradient, 4: watershed image, 5: all 5 parts together, 6: the cutted number):

Number identification:

First of all of every number one filtered images is loaded and a in my case 7x7 neighborhood is determined. Then the input numbers are shifted in that neighborhood and saved.
Now a big for-loop of every angle in a circle in steps of 1°, over every number and every shifted number through the neighborhood is substracting the input numbers with the numbers I want to check and the sum of this difference is saved in a matrix with all these values.
After this the minima value of this matrix is taken and the corresponding number is determined. Congrats you have the number of the image. If the difference of the images is too big, so now accordance or the image is empty it returns 0 and the number can't be identified.
Also very noisy images as you see here can be identified and the six and nine, where just the dot is different, is also good determined.
It can be improved by chosing a bigger neighborhood but then it also takes much longer.

Saving:

Now every image is saved in different folders with seperated in numbers and the amounts of all the numbers ist determined:
amount 0: 8
amount 1: 154
amount 2: 192
amount 3: 143
amount 4: 180
amount 5: 138
amount 6: 166
amount 7: 169
amount 8: 140
amount 9: 177
amount 10: 185
amount 11: 193
amount 12: 155

and here are the wrong determined numbers:
1:    6x4, 2x11, 2xf
2:    
3:    1x2, 1x9
4:   
5:    2x9
6:    4x9
7:    5x1, 1x5
8:    1x2, 1x9
9:   
10:    1x2
11:   
12:

27/2000 are not right identified. I woud say this is a quit good result for this code. The time it took is 13600s with the methode, that each image is processed one after another to reduce the time to take hold on the big matrices.

Source Code:

clear all;
clc;
tic;
iptsetpref('UseIPPL',false);
showgraph = 0;
text_iden = 1;
saveimg = 1;
fold = 'NeuronBilder_D12_03_7_2000';

 %________________________________________________
cd('C:\Users\admin\Dropbox\Sparta Kriegspläne\DieImages\TextHandle');
for i = 1:1:12
    file_num{i} = sprintf('n%d.mat', i);
    num_ex(:,:,i) = cell2mat(struct2cell(load(file_num{i},'-mat')));
end
    neighboor = 7^2;
    n_neigh = (sqrt(neighboor)+1)/2;
    num_small(:,:,:) = num_ex(n_neigh:end+1-n_neigh,n_neigh:end+1-n_neigh,:);
    num_8con(:,:,:,:) = zeros(51,51,12,9);

for i=1:1:sqrt(neighboor)
    for j=1:1:sqrt(neighboor)
        num_8con(i:end+i-sqrt(neighboor),j:end+j-sqrt(neighboor),:,j+sqrt(neighboor)*(i-1)) = num_small(:,:,:);
    end
end
   
    %________________________________________________
cd('C:\Users\admin\Dropbox\Sparta Kriegspläne\DieImages\D12_01x2000');
files = dir('*.jpg');

len = length(files);
%len = 100;
s = 1;
start=0;
    searched_num = [1:1:12];

for i = 1:1:len
    name{i}=files(start+i).name;
    img(:,:,:,i) = im2double(imread(name{i}));
    imgarea(:,:,:) = img(1:200,61:260,:,i);
    img_gray(:,:) = rgb2gray(imgarea(:,:,:));
    img_grad(:,:) = imgradient(img_gray(:,:),'sobel');
    sz = size(img_gray);
    img_oli = img_gray;

    img_tresh = img_oli;
    img_tresh(img_tresh<0.7) = 0;
    img_tresh(img_tresh>=0.7) = 1;

    img_grad(img_grad<0.75) = 0;
    img_grad(img_grad>=0.75) = 1;

    closed = imclose(img_grad,ones(5));
    img_multi = closed.*img_tresh;
    img_multi = imdilate(img_multi,ones(3));

    small_trsh = 50;
    dtm = 15;
    clearvars L_sort L_sum index L_unique L_max L_max_2;   
   
    water = zeros(sz(1),sz(2));
    water_2 = zeros(sz(1),sz(2));
    water_3 = zeros(sz(1),sz(2));
    water_4 = zeros(sz(1),sz(2));
    water_5 = zeros(sz(1),sz(2));
   
    img_water_5(:,:) = zeros(sz(1),sz(2));
    img_water_4(:,:) = zeros(sz(1),sz(2));
    img_water_3(:,:) = zeros(sz(1),sz(2));
    img_water_2(:,:) = zeros(sz(1),sz(2));
    img_water_1(:,:) = zeros(sz(1),sz(2));

    grad_between(:,:) = imgradient(img_multi(:,:),'sobel');
    L(:,:) = watershed(grad_between(:,:));
    L_old = L;
    % find the middle
    [pos_x pos_y] = image_mid(img_tresh(:,:));
    pos_x = pos_x - 18;
    width = 32;
    vec(:,:) = [pos_x-width, pos_x+width; pos_y-width, pos_y+width];
    vec(vec<1) = 1;
    vec(vec>200) = 200;
    %calculate the area L
    water = zeros(sz(1),sz(2));
    L(L==0) = 1;
    % find largest areas
    L_unique = unique(L(:));
    for j=1:1:length(L_unique)
        L_sum(j) = length(L(L==j-1));
    end
    [L_sort, index] = sort(L_sum,'descend');
   
    %1__________________________________________
    L_max = index(2)-1;
    i
    %main img
    water(L(:,:) == L_max) = 1;
    img_water(:,:) = water(:,:);   
    %letter 1
    [pos_x_1 pos_y_1] = image_mid(img_water(:,:));
    area = img_water(vec(1,1):vec(1,2),vec(2,1):vec(2,2));
    sum_1(i) = sum(area(:));
    sum_11(i) = sum(water(:));
    dif_mid_1(i) = ((pos_x-pos_x_1)^2+(pos_y-pos_y_1)^2)^(0.5);
    if sum_11(i)==sum_1(i) && dif_mid_1(i)<35
        img_water(:,:) = water(:,:);
    else
        img_water(:,:) = zeros(sz(1),sz(2));
    end
   
    %2__________________________________________
    L_max_2 = index(3)-1; 
    water_2(L(:,:) == L_max_2) = 1;
    img_water_2(:,:) = water_2(:,:);   
    [pos_x_2 pos_y_2] = image_mid(img_water_2(:,:));
    area = img_water_2(vec(1,1):vec(1,2),vec(2,1):vec(2,2));
    sum_2(i) = sum(area(:));
    sum_22(i) = sum(water_2(:));
    dif_mid_2(i) = ((pos_x_1-pos_x_2)^2+(pos_y_1-pos_y_2)^2)^(0.5);
    %second letter
    if sum_2(i)==sum_22(i) && dif_mid_2(i)>dtm && sum_2(i)>small_trsh
        img_water_2(:,:) = water_2(:,:);
    else
        img_water_2(:,:) = zeros(sz(1),sz(2));
    end
   
    %3__________________________________________
    %third letter
    L_max_3 = index(4)-1;
    water_3(L(:,:) == L_max_3) = 1;
    img_water_3(:,:) = water_3(:,:);
    [pos_x_3 pos_y_3] = image_mid(img_water_3(:,:));
    area_3 = img_water_3(vec(1,1):vec(1,2),vec(2,1):vec(2,2));
    sum_3(i) = sum(area_3(:));
    sum_33(i) = sum(water_3(:));
    dif_mid_3(i) = ((pos_x_1-pos_x_3)^2+(pos_y_1-pos_y_3)^2)^(0.5);
    if sum_3(i)==sum_33(i) && dif_mid_3(i)>dtm && sum_3(i)>small_trsh
        img_water_3(:,:) = water_3(:,:);
    else
        img_water_3(:,:) = zeros(sz(1),sz(2));
    end

    %4__________________________________________
    %forth letter
    if length(index)>4
        L_max_4 = index(5)-1;
        water_4(L(:,:) == L_max_4) = 1;
        img_water_4(:,:) = water_4(:,:);
        [pos_x_4 pos_y_4] = image_mid(img_water_4(:,:));
        area_4 = img_water_4(vec(1,1):vec(1,2),vec(2,1):vec(2,2));
        sum_4(i) = sum(area_4(:));
        sum_44(i) = sum(water_4(:));
        dif_mid_4(i) = ((pos_x_1-pos_x_4)^2+(pos_y_1-pos_y_4)^2)^(0.5);
        if sum_4(i)==sum_44(i) && dif_mid_4(i)>dtm && sum_4(i)>small_trsh
            img_water_4(:,:) = water_4(:,:);
        else
            img_water_4(:,:) = zeros(sz(1),sz(2));
        end
    end
    %5__________________________________________
    %fifth letter
    if length(index)>5
        L_max_5 = index(6)-1;
        water_5(L(:,:) == L_max_5) = 1;
        img_water_5(:,:) = water_5(:,:);
        [pos_x_5 pos_y_5] = image_mid(img_water_5(:,:));
        area_5 = img_water_5(vec(1,1):vec(1,2),vec(2,1):vec(2,2));
        sum_5(i) = sum(area_5(:));
        sum_55(i) = sum(water_5(:));
        dif_mid_5(i) = ((pos_x_1-pos_x_5)^2+(pos_y_1-pos_y_5)^2)^(0.5);
        if sum_5(i)==sum_55(i) && dif_mid_5(i)>dtm && sum_5(i)>small_trsh
            img_water_5(:,:) = water_5(:,:);
        else
            img_water_5(:,:) = zeros(sz(1),sz(2));
        end
    end
   
   
    bet_2 = zeros(sz(1),sz(2));
    bet_2(vec(1,1):vec(1,2),vec(2,1):vec(2,2)) = 1;
    img_rect(:,:) = bet_2(:,:);
    rgb(:,:,:) = label2rgb(L_old(:,:),'jet',[.5 .5 .5]);
   
   
    img_together(:,:) = img_water(:,:) + img_water_2(:,:) + img_water_3(:,:)...
        + img_water_4(:,:) + img_water_5(:,:);
   
    %determine fast area
    [pos_x_n pos_y_n] = image_mid(img_together(:,:));
    width = 25;
    vec_n(:,:) = [pos_x_n-width, pos_x_n+width; pos_y_n-width, pos_y_n+width];
    fast_area(:,:) = zeros(51,51);
    vec_n(vec_n<1) = 1;
    vec_n(vec_n>200) = 200;
    en = vec_n(1,2)-vec_n(1,1)+1;
    en2 = vec_n(2,2)-vec_n(2,1)+1;
    if sum(sum(img_together(:,:)))>50
        fast_area(1:en,1:en2) = img_tresh(vec_n(1,1):vec_n(1,2),vec_n(2,1):vec_n(2,2));
    end

    BW3(:,:) = bwmorph(fast_area(:,:),'skel',Inf);
    BW2 = imdilate(BW3,ones(3));
    BW1 = imdilate(BW3,ones(2));
%___________________
 
    if text_iden == 1
   
    img_ocr = fast_area(:,:);
    img_all{i} = img_ocr;
    angstep=1;
    for angle=1:angstep:360
        img_rot = imrotate(img_ocr,angle,'nearest','crop');
        for j=1:1:12
            for k=1:1:neighboor
                diff_img(:,:) = abs(img_rot-num_8con(:,:,j,k));
                img_diff((angle-1)/angstep+1,j,k) = sum(sum(diff_img));
            end
        end

    end
    for k=1:1:neighboor
        differ_k = img_diff(:,:,k);
        [min_val_k(k) min_pos_k(k)] = min(differ_k(:));
    end
    [minima, minima_pos] = min(min_val_k);
    differ = img_diff(:,:,minima_pos);
    [min_val min_pos] = min(differ(:));
    numbers = min_pos-mod(min_pos,360/angstep);
    letter(i) = numbers/(360/angstep)+1;
    if letter(i) == 0
        letter(i) = 12;
    elseif letter(i) == 5 || letter(i) == 8 || letter(i) == 9
        text_grad = imgradient(img_ocr,'sobel');
        L_text = watershed(text_grad);
        text_unique = length(unique(L_text(:)));
        switch text_unique
            case 3
                letter(i) = 5;
        end
    end
   
    if sum(sum(img_ocr))==0
        letter(i) = 0;
    end
    %subplot(10,10,i);imshow(img_ocr,[]);title(letter(i));


    for i=0:1:12
        c = sprintf('count_%d', i);
        assignin('base', c, length(letter(letter==i)));
    end
    end  
end


% saving __________________________________________________________
if saveimg == 1
    cd('C:\Users\admin\Music');
    mkdir(fold);
    cd(fold);
   
    fid = fopen('Ergebnisse_D12.txt','w');
    for i=0:1:12
        fprintf(fid,'Anzahl %d: %d\r\n',i,length(letter(letter==i)));
    end
    fclose(fid);

    for i=0:1:12
        mkdir(sprintf('%d', i));
    end
   
    for i=1:1:len
        i
        filename = sprintf('C:\\Users\\admin\\Music\\%s\\%d\\img%d_%d.jpg',fold,letter(i),letter(i), i);
        imwrite(img(:,:,:,i),filename);
    end
   
end

time_dur = toc

%_________________________________________________________%
function [pos_x pos_y] = image_mid(image);

[x y] = find(image==1);
pos_x = round(mean(x));
pos_y = round(mean(y));

end

Kommentare

Kommentar veröffentlichen

Beliebte Posts aus diesem Blog

Matlab: 3D Coordinate System Rotations with Vectors

-
Bild
This post describes how to switch/transform between corrdinate systems in Matlab and how this can be represented. There are 3 functions that are crucial here: grid3D(startVec, stopVec, varargin) ... Creates an orthogonal grid in 3D depending on start and end point (diagonal of a cube) and draws ticks, labels and MinorGrids. Rotations can be performed here to show how to transform between Cartesian corrdinate systems. point3D(startVec, stopVec, varargin) ... Is a function to create 3D vectors based on start and end points. It includes options like naming the points, displaying them as arrows, as well as settings for color and thickness. arrow3D(startVec, stopVec, varargin) ... This function creates an arrow in 3D that can be scaled by the options. The functions including example can be downloaded here: Matlab-Functions/CartesianCorrdinateTransformation at main · PhysIngo/Matlab-Functions (github.com) First of all an example where the 1 coordinate system with 2 vectors is rotated 2 times...
Mehr anzeigen

Matlab: Cone/Arrow in 3D

-
Cone/Arrow in 3D To plot a cone or arrow in 3D you can use this function: function arrow3D(startVec, stopVec, varargin) The startVec is the beginning of the 3D cone and the stopVec the end of the cone. Ypi can use the optional arguments as 'Color','Thickness','Length' to customize your cone. Source code : function arrow3D(startVec,stopVec,varargin) % Input arguments: % startVec ... input vector of the beginning of the point in [x,y,z] % stopVec ... input vector of the ending of the point in [x,y,z] % % Optional input argument (varargin): % 'Color',color ... define the color of the arrow/point % 'Thickness',thick ... thickness of the arrow % 'Length',length ... Length of the arrow % 'Axes',ax ... Input axes     if isempty(varargin)         varargin{1} = '';     end          [x] = startVec(1);      [y] = startVec(2);      [z] = startVec(3);      x2 = stopVec(1);   ...
Mehr anzeigen

Matlab: Points and vectors in 3D-plots

-
Bild
Points and Vectors in 3D-Plots (German) This matlabfunction creates vectors and points for the previous post in the 3D cartesian coordinate system. function punkt3D(x1,y1,z1,x2,y2,z2,sign,name,color,dx,dy,dz,true) Therefor "startVec" are the initial values of the vector and "stopVec" the final values. You need the function ( Cone/Arrow ) and  ( Grid3D ). The optional arguments are shown below. For example you can use it as: figure;hold on;axis off; grid3D([0,0,0], [1,1,1], 'Amount', [4,4,4], 'Axes', gca, 'MinorGrid','Arrow','AxTicks',[{['']},{['']},{['']}],...     'AxLabels',["";"";""]); P = [0.5,0.5,1]; Q = [1,0.5,1]; point3D([0,0,0],P,'Marker','arrow','Label','P(0.5|0.5|1)','Color',[0.7 0.5 0]); point3D([0,0,0],Q,'Marker','arrow','Label','Q(1|0.5|1)','Color',[0.0 0.4 0.74],'D...
Mehr anzeigen