Widget: Assessment Graph
The Assessment Graph Widget displays a line graph of the available student results averaged per month.
Depending on what is to be plotted, the graph may show single or multiple lines. If there are multiple lines, a legend will appear underneath the graph to identify each line displayed. The horizontal axis will be labelled for each month, with time increasing from left to right. The vertical axis displays the markscheme labels (which are imported from SIMS). If baselines have been configured in the toolkit they can also be displayed on the graph.
Configuring the Widget
To configure the widget click the small black Edit button that appears in the top right corner of the widget in the Widget editor. You can also double click the Nested Page Component that holds the Widget if editing through the Toolkit.
The Assessment Graph Widget has 11 fields you can set in the Widget options, these are:
- Filter tag - This is the Filter Tag name to use for this Widget.
- Width - This is the display width for the graph in pixels.
- Height - This is the display height for the graph in pixels.
- Group by - This filter allows you to set whether the graph displays data by either Classification, Subject, User or Group.
- Include baselines - If this check box is selected then any defined Base Lines will also be displayed on the graph.
- Include markscheme scale - If this check box is selected then the vertical axis of the graph will be populated with the relevant mark scheme.
- Only in classification - This allows you to filter the data so that only data for a specific Classification is used in the graph.
- Only in group - This allows you to filter the data so that only data for a specific Group is used in the graph.
- Only in last num days - This allows you to limit how far back the data on the graph goes in days.
- Only in subject - This allows you to filter the data so that only data for a specific Subject is used in the graph.
- Only in type - This allows you to filter the data so that only data for a specific Classification Type is used in the graph.
This Widget uses Filter Tags. For more information about Filter Tags see the article Filter Tags Explained.
The Assessment Graph widget can be used to display trends in a student's behaviour and results over time. This section is designed to help when the graph doesn't do what you expect. The graphing widget is a powerful tool, but because of its flexibility it needs to be carefully configured in order to accurately and reliably reflect a child's progress.
Identifying the Problem
The graph widget collates results per month per user.
It starts off with set of data results. It then averages those results for each month creating set points for that month for each line it needs to plot on the graph. Months with no results available are skipped. It is this process of collating results that can potentially cause the most problems.
- All marks for a particular month are merged. This process can be problematic if the marks we are trying to merge come from more than one markscheme (how do you merge A,B,C grades with key stage levels for example?).
- Depending on the graph configuration, target and behaviour results might be merging with current attainment results: this can give a misleading and confusing view of the child's progress.
- Graphs are by their nature numeric – before being plotted all marks need to be converted to numerical values:
- Some markschemes are already numeric, or the mapping from text labels to numeric values is defined in SIMS; others are not. Results can be skipped and not plotted on the graph if they cannot be converted to numerical values.
- Sometimes a markscheme looks like it is numeric already, but in fact it is not. A good example is a markscheme where '1' equates to excellent, '2' to good, '3' to average and '4' to poor. This appears to be numeric but works the reverse way round (results will potentially be plotted upsidedown!).
Sometimes data to make sense of the marks on a graph is simply missing: some schools assign marks to an aspect in SIMS without associating a markscheme with that aspect. This makes it more difficult for Frog because we don't have any information on what those marks are or how they should be plotted.
Deciding What to Plot
Many of the problems with the graph can be solved by being a bit more selective with what you are trying to plot. You can either use filters within the Widget user fields to restrict the data, or use the suite of assessment filter widgets.
- Your problem is caused by students attainment/target and effort results all being merged together giving spurious results. It doesn't make sense to merge these so you need to decide on one type to display: you open the Widget properties and change the graph to plot only the current attainment. Problem solved.
- Your problem is caused by the graph plotting a mixture of GCSE and Key Stage results over a period of years. These schemes do not merge correctly and resulting graph is a bit odd. To solve this we define two new Frog assessment groups in the toolkit and assign the aspects to each group accordingly. We then add an assessment group select above the graph: the user now chooses to view either the Key Stage results or the GCSE results.
Hopefully the above two examples illustrate that problems are often caused by trying to plot too much data on a single graph. It is often better to add multiple graphs to a page that show particular trends than trying to cram it all on a single graph.
SIMS enforces limitations in that an aspect can only be associated with a single markscheme. In the Assessment Graph Widget we are collating data across a number of different aspects which might all have different markschemes. If necessary, the graph will attempt to merge those markschemes together.
For example, consider a list of results which are a mixture of the markschemes which are defined in SIMS:
- GCSE Markscheme - Numeric Labels A*=10, A=8, B=6, C=4
- Homework Markscheme - Numeric Labels E (Excellent)=3, G (Good)=2, P (Poor)=1
Given a list of marks in both markschemes Frog will:
- Choose a primary markscheme. This will be the markscheme with the highest number of different grades (as it gives the most detailed mark breakdown). In this case, the primary markscheme will be the GCSE Markscheme.
- For each of the grades in the Homework Markscheme, Frog will map that grade to what it perceives to be a numerical equivalent position in the primary markscheme. So for example, the 'G' grade from the homework markscheme occurs exactly in the middle of the 1-3 numeric range of that markscheme, so it will be mapped to the middle of the numeric range of the primary markscheme (value 7). In other words the merged markscheme becomes:
- Merged Markscheme - Numeric LAbels A* or E=10, A=8, G=7, B=6, C or P=4
In this case we have merged two markschemes, but the graph may need to merge many markschemes back onto the primary one. The scale on the left hand side of your graph always shows the primary markscheme values. Depending on your own markschemes, this merging process may work well, or not so, and this may be the cause of problems within your graph. If you are having problems with markschemes merging, you have two options to solve the issue:
- Update the definitions of your markschemes within SIMs. Frog imports the markschemes and uses the data to perform merges. If the merge is not working well, it might be solved by updating/adding more detail to the markscheme definitions in SIMS.
- Divide and conquer: as discussed in the “Deciding what to Plot” section you can solve your problem by restricting what the graph plots so the incorrect markscheme merge never occurs (i.e. the data plotted does not have multiple markschemes).
Arriving at Numerical Values
Not all marks or markschemes are numerical. Any marks from non-numerical markschemes are simply skipped and not included on the graph. A markscheme is classes as numeric if numerical values for each mark is configured (in SIMs), if so these values are then used as the numeric marks. If there are no numeric values defined in SIMs, but the markscheme labels are all numeric, then the labels will be used as the numerical values. If neither of these conditions are met, the results from this markscheme are not plotted on the graph. This algorithm to determine whether a markscheme is numeric can lead to errors: take for example the markscheme:
1=Excellent, 2=Good, 3=Poor
This markscheme has no numeric values defined in SIMs, but it is still classes as a numerical scheme because the labels are numeric. However, the graph will plot these values the wrong way round since it will treat poor (3) as being above excellent (1). To solve this issue, you would need to add some numeric values in SIMS to the markscheme definition.
3=Excellent, 2=Good, 3=Poor
So as you can see with a little bit of consideration to how your markschemes are defined, and the numeric values associated with them the graph will then be able to properly average the values and the data you see will have some meaning.
In a World Without Markschemes
Most schools define and associate SIMs markschemes to each aspect, but some schools have no defined markschemes, or markschemes attached to only some of their aspects.
When there is no markscheme it is possible to define a minimum and maximum numerical mark in SIMs and Frog will import this to use as the marking range. Frog will treat all marks without a markscheme as numeric.
Be aware the process of merging marks with a markscheme and those without a markscheme will be problematic and should be avoided by being more specific about what to plot on the graph where possible.