Getting Started with Data set

Data set used in this project is downloaded from Cricsheet. I am planning to do the analysis in two phases. Phase 1 is based on Team wise evaluation and Phase 2 is player wise evaluation. First we will focus on team wise evaluation. Chosen team for the evaluation is Sri Lanka. You can download the data set used in this tutorial here (See diagram below).



In any kind of data mining analysis, the very first process is preparing the data. Missing values, repeated data, erroneous data, data type mismatch are some of the common things to be considered when working with raw data. So in this post I will guide you through in the process of getting ready with our data.

Our data set is in YAML format (Cricsheet provides experimental CSV and  XML formats at the time of this post being written). As they state their data is experimental, I recommend you guys to work with YAML format. However there is no functionality to  directly load YAML data set in R. As a solution I have written a Java program which converts above YAML data set into CSV format. You can download converted CSV file here. If someone is interested in the Java program, please find it through my Github Profile. 

Steps in Preparing the data

  1. Load the CSV file.
  2. Convert loaded file into a data frame.
  3. Convert columns into appropriate data type (Character, Numeric, Boolean).
  4. Fill empty cell with values  
  5. (There are some professional methods to deal with missing data, however in this data set we cannot use something like average value or mean value due to the nature of the problem domain. Therefore missing values are represented as "unknown").
  6.  Remove white spaces within data sets (This is not necessary in a general data preparing process. By removing white spaces it was able to make "Sri Lanka" and " Sri Lanka" into one attribute value. You can see that the latter term is containing a space before, probably a typo. If we have not performed this step, attribute value "Sri Lanka" is different from value " Sri Lanka" which might cause erroneous results).
  7. Remove repeated data.

Herewith I have attached my R Script. 

Congratulations!!! You are about to start the Cricket Sabermetrics analysis. 

Comments

Post a Comment

Popular posts from this blog

What is Cricket Sabermetrics?

In the 1980s, work by Bill James on statistical analysis of Baseball named Sabermetrics gained a major popularity. Sabermetrics is study of the game baseball through scientific and objective analysis. The term was derived from the acronym SABR which stands for Society of American Baseball Research. Recognition of this work led to a revolution in the game of Baseball. Oakland Athletics baseball team went on to set the record for twenty consecutive wins in the 2002 season using Sabermetrics. Many teams and clubs adapted this approach in the following years and got succeeded. Many sports including cricket tend to follow similar approaches in their games after witnessing the transformation of game baseball through sabermetrics and its positive outcomes.  Nowadays analysis on a particular subject area is often interrelated with data science and machine learning.  Modern Sabermetrics can be defined as application of machine learning in Cricket data. I recommend to watch criti...
Read more

What is this Blog about?

This blog will describe on how to apply Sabermetrics approach in Cricket using machine learning techniques. This will be an easy step by step guide on applying machine learning concepts to model the Cricket game. Language used for programming and analysis is R .  Ultimate goal of this project is proposing a data mining system for selecting a Winning Cricket Team!
Read more