meta data for this page
Back to Main Menu | Ins. Research Menu | Next Ad-Hoc Queries
RISK PREDICTION & RETENTION
How to Access: Institutional Research > Risk Scoring
Purpose: Advanced Machine Learning for at-risk student detection. How does it work? We will process and analyze your data automatically, create several statistical models, and pick the one that best fits your students.
Abstract: The Risk Scoring system is a key tool in AccuCampus that helps detect students at risk of dropping out. Using AI technology, AccuCampus analyzes student data from its database and from other sources to try to detect patterns and keep staff members in contact with those students. In this document, we review how Risk Scoring works in-depth. We will look at what the recommended configurations are for it and its limitations.
If this is the first time in this feature please click CREATE YOUR FIRST MODEL to start, otherwise, a list of Models will be presented:
Risk Assessment Models List
Top button Options
- Create New - Click this button to start a new Model. (see details in the next chapter).
- Refresh - Click this button to update the list and refresh it.
List options Please notice that for each Model(row) in the list there are 4 buttons on the right-hand side.
- Re-analyze- Click this button to analyze again this Model, this is especially useful if there have been changes in Accucampus and you would like to analyze again the Model.
- View analysis Results- A graphical representation of each feature that is included in the model will appear. Use these graphs to check your data for accuracy.
- Process- Clicking Process will begin the machine learning algorithm. You will start seeing risk scores for your users within a week of starting the model.
- Delete- Use this button to delete the current Model, a confirmation message will show up to make sure you are not deleting by mistake.
Viewing Risk Scores
User risk scores are found on the individual user’s profile page. The risk score will be updated weekly as new information is added to the system through imports and through the user’s behaviors within the system.
Create New Risk Assessment Model Buttons
- Name -Enter a Name for the model. If you will have more than one model, we recommend using a unique and specific name so that other users will understand what the model is for.
- Exclude features - Choose which demographic features to exclude when making predictions. These will be the unique keys used in your user profiles.
- Criteria to exclude users - Choose to exclude user types from your model. (example age<18)
New features
- New features - If desired, create new features to include when making predictions. These are typically created by combining features.(example visits_tm [0]:= visits_tutoring + visits_mentoring)
Semester precedences
- Semesters - Add semester precedences. These tell the system how semesters proceed from one to the next. We recommend including at least Spring and Fall semesters, but you can include all semesters if desired. Keep in mind that not all students will proceed through the semesters the same.
- When the current Semester is - Please set a Semester here so you can specify what Semester follows next.
- The next semester is - Please set the Semester that follows the Semester above.
- Add precedence - Please click this button to ADD the precedence specified above.
- Load default precedences -
- Save and analyze - If you are ready to analyze your model’s results, click Save and Analyze.
- Save - Save current data to return to the model for further modifications later.
- Cancel - Cancel current operation without saving.
Analysis Results
Once you click Save and Analyze, the system will begin analyzing the data that will be included in your model to come up with the best algorithmic model to use. The Analysis screen will show you each feature that was included in the model as a graph with a visual breakdown of the student population for that factor as well as a graph that shows you the weighting that the system has applied for each feature. These graphs can be used to 1) identify any issues with the data that might have been missed prior to importing and 2) help you determine if the model should be adjusted before processing. Tweaking the model would include excluding features or creating new combinations of features to include. Once satisfied, you can then run the model by clicking “Process”. This will start the machine learning system.
Limitations There are some known limitations in the AccuCampus’ risk scoring system. The first and most obvious one is that AccuCampus does not know the individual situation of each student; the analysis is being done based on the information available and there are several factors that cannot be represented within a database. The understanding of the student situation will only be as good as the information available. The engine currently works to detect whether the student will drop out or not. The engine design was not for nor can it predict a specific outcome on the performance in a specific course or in specific learning areas. Furthermore, information from a few semesters is required to begin training the model in AccuCampus. The longer the institution uses AccuCampus, the more accurate the prediction will be. Information from action items and student activity on the AccuCampus website is not currently affecting the risk score.
Important documentation
This PDF explains in detail the process, this is a must-read.