Project 3: Association Rule Mining

Courtesy PROF. CAROLINA RUIZ with modification

Project Assignment

• THOROUGHLY READ AND FOLLOW THE PROJECT GUIDELINES. These guidelines contain detailed information about how to structure your project, and how to prepare your written and oral reports.

  Your written report should be at most 10 pages long (including all the graphs, figures, and appendices). The font size should be no smaller than 11pts.

• Data Mining Technique(s): We will run experiments using the following techniques:
  • Pre-processing Techniques: Feature selection, feature creation, dimensionality reduction, noise reduction, attribute discretization, ...
  • Association rule mining: Apriori algorithm.

• Dataset(s): In this project, we will use two datasets:

  • The 1995 Data Analysis Exposition. This dataset contains college data taken from the U.S. News & World Report's Guide to America's Best Colleges. The necessary files are:
    • readme
    • usnews.data
    • usnews.doc

  • A dataset of your choice depending on your own interests. This dataset should contain enough instances (at least 200 instances) and several attributes (at least 10). Ideally it should contain a good mix of numeric and nominal attributes.
    Here are some possibilites:
    • A dataset you are working with for your research or your job.
    • A dataset on more up-to-date college data that you collect/find online.
    • A dataset from a data repository listed in the online resources, or other online data repository.
    • Other data source of your choice.

• Performance Metric(s):Support, confidence, lift, leverage, and conviction. Include in your report a definition (using a precise formula) and a description of the meaning of each of these metrics. Also, for extra credit you are encouraged (but not required) to implement in Weka other association rule metrics defined in Section 6.7 of the textbook (e.g., interest factor, correlation analysis, IS measure, ...), and experiment with them.

• General Comments: In constrast with our previous classification and regression projects, we won't use any evaluation protocol (e.g., 10-fold cross validation) for the association analysis of this project, as we're not using the rules for prediction. Focus instead on experimenting with different ways of preprocessing the data, varying the parameters of the Apriori algorithm, and providing your own method to evaluate the resulting collections of association rules. Remember to experiment with car (that is, classification association rules) and to compare its classification performance to that of decision trees, and addition to non-car rules.

• Advanced Topic(s) (30 points): Investigate in more depth (experimentally, theoretically, or both) a topic of your choice that is related to association rule mining and that is not covered already in this project. This association rule mining -related topic might be something that was described or mentioned in the textbook or in class, or that comes from your own research, or that is related to your interests, or that appears in a research paper that you find intriguing.