You will use R to mine actual data for a problem of interest. These could be data from a problem from your current job if you have one, something of interest to the School of Management or College, data acquired from the web, etc. You will design the data mining task, mine the data, and describe your results. You also will research existing solutions to the problem, if any have been proposed or documented. Your own data and results need not be on a par with actual industry results; the goal is for you to get as realistic a hands-on experience as possible, given the constraints of what you have learned.
In writing up/presenting your research, think of yourself as ananalyst employed by or retained by a company (large or small) or by a funding source (e.g., a venture capital (VC) firm or incubator), who wants to understand the state of the art for using data mining for the task in question. Review what has been done to date on your problem. Consider as an example predictive analytics for on-line advertising: A VC firm considering funding on-line ad networks or ad-tech startups would need to understand the state of the art in using data mining for targeting on-line advertising, when considering an idea for applying data mining. Don’t worry too much about coming up with a novel idea. It is more important to develop the idea well (within the scope of what we’ve discussed in class).
You should use the CRISP-DM data mining process to structure your research and report. Keep in mind that it may be ineffective simply to proceed linearly through the steps, and this may need to be reflected in your analysis. You should interact with me from the preparation of your initial ideas through to the preparation of your report, as a consultant would interact with a firm or funding source in preparing a research report. Use your imagination, prior experience, or ask for help to fill in any gaps between the material available and what you would be able to find out if you actually could interact with the client firm.
This assignment will have a phased submission of work, as follows:
Submission 1: On February29th, you will submit a proposal for your project via Moodle. This should give as much detail as possible on your ideas, so that I can give you brief feedback. Include in your proposal your ideas about: What is the exact business problem? What is the use scenario? What precisely is the data mining problem? Is it supervised or unsupervised? What is a data instance? What might be the target variable? What features would be useful? How exactly would it add business value? Etc.
Submission2: On March21styou will submit your final report which should be about 1500 words, plus any appendices you would like to include. Use external sources where appropriate, and provide clear citations and bibliography. You should also submit your data file and a working R script which I can run on it.
You will get the most out of the project if you interact with me during the development of your ideas. Please feel free to come talk to me about your ideas as often as you’d like — my office hours are on Mondays and Tuesdays 11.30-12.30 and there is a notice on the board outside my office (MX116 on the first floor of the Moore Annexe) where you can sign for an appointment.
Your report should include the information detailed below, in approximately the order given. Your report need not have corresponding sections or bullet points, but I should be able to find the information without searching too hard. Be as precise/specific as you can.
Business Understanding (take this seriously)
• Identify, define, and motivate the business problem that you are addressing.
• How (precisely) will a data mining solution address the business problem?
(NB: I’d like to see a good definition/motivation of the business problem and a precise statement of how a data mining solution will address the problem. It’s not so important that the hands-on results match perfectly. It’s more important that you have the experience of working through a realistic problem definition.)
• Identify and describe the data (and data sources) that will support data mining to address the business problem. Include those aspects of the data that we talk about in class and/or in the quizzes.
• Specify how these data are integrated to produce the format required for data mining.
(NB: data preparation can be time consuming. Get started early. Talk to me if you need advice.)
• Specify the type of model(s) built and/or patterns mined.
• Discuss choices for data mining algorithm: what are alternatives, and what are the pros and cons?
• Discuss why and how this model should “solve” the business problem (i.e., improve along some dimension of interest to the firm).
• Discuss how the result of the data mining is/should be evaluated. How should a business case be developed to project expected improvement? ROI? If this is impossible/very difficult, explain why and identify any viable alternatives.
• Discuss how the result of the data mining will be deployed.
• Discuss any issues the firm should be aware of regarding deployment.
• Are there important ethical considerations?
• Identify the risks associated with your proposed plan and how you would mitigate them.